Model determination method and electronic equipment

Abstract

The invention discloses a model determination method and electronic equipment. The method comprises the following steps: dividing a target scene into a plurality of block regions; acquiring scene image data, wherein the scene image data comprises a plurality of scene images corresponding to the target scene and a camera position corresponding to each scene image; a first area and a second area in the target scene are determined, the first area is one of the multiple block areas, the second area comprises the first area, and the second area is larger than the first area; based on the camera position corresponding to each scene image, a target scene image corresponding to the second area is determined from the multiple scene images, and the camera position of the target scene image is located in the second area; and determining a three-dimensional model corresponding to the first area based on the target scene image.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional models, and in particular to a model determination method and electronic equipment. Background Art

[0002] Currently, the scene can be divided into multiple sub-blocks, and a 3D model is trained for each sub-block, and then the 3D model of the scene is obtained by merging them. However, the existing sub-block 3D model training method does not consider the contribution of images outside the sub-block range to the current sub-block, so the trained sub-block 3D model lacks stability. Summary of the invention

[0003] In view of this, an embodiment of the present application provides a model determination method and an electronic device.

[0004] According to the first aspect of the present application, an embodiment of the present application provides a model determination method, including: dividing a target scene into multiple block areas; acquiring scene image data, the scene image data including multiple scene images corresponding to the target scene and a camera position corresponding to each scene image; determining a first area and a second area in the target scene, the first area being one of the multiple block areas, the second area including the first area, and the second area being larger than the first area; based on the camera position corresponding to each scene image, determining a target scene image corresponding to the second area from the multiple scene images, the camera position of the target scene image being located within the second area; and determining a three-dimensional model corresponding to the first area based on the target scene image.

[0005] Optionally, before determining the first area and the second area in the target scene, the model determination method further includes:

[0006] Based on the camera position corresponding to each scene image, two target camera positions that meet the distance condition are determined;

[0007] The directions of the coordinate axes in the coordinate system of the target scene are adjusted according to the directions corresponding to the lines connecting the two target camera positions.

[0008] Optionally, based on a camera position corresponding to each scene image, determining a target scene image corresponding to the second area from a plurality of scene images includes:

[0009] Based on the camera position corresponding to each scene image, determine a candidate scene image corresponding to the second area from the multiple scene images, the camera position of the candidate scene image being located within the second area;

[0010] A target scene image is determined from the candidate scene images, the number of target three-dimensional feature points corresponding to the target scene image is greater than a set threshold, the target three-dimensional feature points are three-dimensional feature points corresponding to the first area in the target point cloud, and the target point cloud is determined based on multiple scene images.

[0011] Optionally, determining the three-dimensional model corresponding to the first area based on the target scene image includes:

[0012] Performing a first screening process on the target scene image where the camera position is outside the first area to obtain a first screened image;

[0013] A three-dimensional model corresponding to the first area is determined based on the first screening image and the target scene image where the camera position is within the first area.

[0014] Optionally, determining the three-dimensional model corresponding to the first area based on the first screening image and the target scene image with the camera position located within the first area includes:

[0015] Determining a peripheral scene image from the plurality of scene images; the camera position of the peripheral scene image is outside the second area;

[0016] Performing a second screening process on the peripheral scene image to obtain a second screened image, wherein the selected ratio corresponding to the second screening process is lower than the selected ratio corresponding to the first screening process;

[0017] A three-dimensional model corresponding to the first area is determined based on the first screening image, the second screening image, and the target scene image where the camera position is within the first area.

[0018] Optionally, the model determination method further includes:

[0019] Map the camera position corresponding to each scene image to the target plane to obtain the mapped camera position;

[0020] Determine a first plane area and a second plane area in the target plane, the first plane area is used to determine the first area, and the second plane area is used to determine the second area;

[0021] Based on the camera position corresponding to each scene image, a target scene image corresponding to the second area is determined from multiple scene images, including: based on the mapping camera position corresponding to each scene image, the target scene image corresponding to the second area is determined from multiple scene images, and the mapping camera position of the target scene image is located within the second plane area.

[0022] Optionally, the target scene is divided into a plurality of block areas, including:

[0023] Adjust the scale information of the target scene to the target scale;

[0024] The target scene is divided into multiple block areas according to the preset segmentation size.

[0025] Optionally, after determining the three-dimensional model corresponding to the first area based on the target scene image, the model determination method further includes:

[0026] The three-dimensional model corresponding to the first area and the three-dimensional model corresponding to at least one third area are spliced ​​to obtain a three-dimensional model corresponding to the target scene, the third area is at least one of the block areas, and the third area is different from the first area.

[0027] Optionally, determining the three-dimensional model corresponding to the first area based on the target scene image includes:

[0028] Obtaining an initialization point cloud corresponding to the first region, wherein the initialization point cloud includes a plurality of Gaussian volumes to be trained;

[0029] The initialization point cloud is trained based on the target scene image to obtain a three-dimensional model corresponding to the first area.

[0030] According to a second aspect of the present application, an embodiment of the present application provides an electronic device, including:

[0031] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the model determination method as in the first aspect or any embodiment of the first aspect.

[0032] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a flow chart of a model determination method in an embodiment of the present application;

[0034] Figure 2 A schematic diagram of a process for adjusting the camera position corresponding to each scene image to the camera position in a new coordinate system in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of a process for determining a first plane area and a second plane area in a target plane in an embodiment of the present application;

[0036] Figure 4 A schematic diagram of a flow chart of another model determination method in an embodiment of the present application;

[0037] Figure 5 A schematic diagram of a flow chart of another model determination method in an embodiment of the present application;

[0038] Figure 6 This is a schematic diagram of the hardware structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0040] The embodiment of the present application provides a model determination method, which can be applied to the scene 3D model determination based on NeRF (neural radiant field), the scene 3D model determination based on 3D Gaussians Splatting (3DGS), or the scene 3D model determination based on other 3D modeling technologies. Figure 1 As shown, including:

[0041] S101, dividing the target scene into a plurality of block areas.

[0042] In this embodiment, the target scene is a virtual scene corresponding to the real scene.

[0043] In this embodiment, the target scene may be divided into a plurality of block regions of the same size, or the target scene may be divided into a plurality of block regions of different sizes.

[0044] S102, acquiring scene image data, where the scene image data includes a plurality of scene images corresponding to the target scene and a camera position corresponding to each scene image.

[0045] In this embodiment, the scene image is an image obtained by capturing an image of a real scene.

[0046] During specific implementation, images of a real scene may be captured at different locations of the real scene to obtain a plurality of scene images, and the camera position corresponding to each scene image may be recorded to obtain scene image data.

[0047] S103, determining a first area and a second area in the target scene, wherein the first area is one of a plurality of block areas, the second area includes the first area, and the second area is larger than the first area.

[0048] In this embodiment, when performing three-dimensional modeling on the target scene, three-dimensional modeling can be performed on each block area of ​​the target scene separately, and then the three-dimensional models of each block area are spliced ​​together to obtain the three-dimensional model of the target scene. In this way, when the three-dimensional model of the target scene is constructed, the requirements for the memory and graphics processing unit (GPU) of the electronic device will not be too high, and will not be limited by the hardware resources of the electronic device.

[0049] In this embodiment, when three-dimensional modeling is performed on any block area among multiple block areas, the block area can be used as the first area, and then a second area is determined based on the position of the first area, so that the second area includes the first area and the second area is larger than the first area.

[0050] In specific implementation, after determining the position of the first area, the position can be expanded to a certain size range in all directions to obtain the second area corresponding to the first area. For example, the size of the first area is s, and after determining the position of the first area, the position is expanded to a size of s / 2 in all directions, and the area corresponding to the expanded position is the second area.

[0051] S104, based on the camera position corresponding to each scene image, determining a target scene image corresponding to the second area from the multiple scene images, wherein the camera position of the target scene image is located within the second area.

[0052] In this embodiment, the target scene image is a scene image whose camera position is located within the second area and has a greater contribution to the construction of the three-dimensional model of the first area, that is, the target scene image is a valid scene image when the three-dimensional model of the first area is constructed. The target scene image includes a valid scene image whose camera position is located within the first area, and a valid scene image whose camera position is located within the second area and outside the first area.

[0053] In this embodiment, based on the camera position corresponding to each scene image, it can be determined whether each scene image is located in the second area and whether it is a valid scene image corresponding to the first area.

[0054] S105: Determine a three-dimensional model corresponding to the first area based on the target scene image.

[0055] In this embodiment, after the target scene image is determined, a three-dimensional model of the first area can be constructed based on the target scene image to obtain a three-dimensional model corresponding to the first area.

[0056] The model determination method provided by the embodiment of the present application divides the target scene into multiple block areas; obtains scene image data, the scene image data includes multiple scene images corresponding to the target scene and a camera position corresponding to each scene image; determines a first area and a second area in the target scene, the first area is one of the multiple block areas, the second area includes the first area, and the second area is larger than the first area; based on the camera position corresponding to each scene image, determines the target scene image corresponding to the second area from the multiple scene images, the camera position of the target scene image is located within the second area; determines the three-dimensional model corresponding to the first area based on the target scene image; in this way, when constructing the three-dimensional model corresponding to the first area, not only the three-dimensional model of the first area is adopted The target scene image with the camera position within the first area and the target scene image with the camera position outside the first area are also used, thereby taking into account the contribution of scene images at different camera positions to the construction of the three-dimensional model of the first area, and can effectively eliminate invalid scene images collected in the first area, and add valid scene images collected outside the first area, so that the three-dimensional model of the first area has high adaptability and stability; and the target scene is divided into multiple block areas, and corresponding three-dimensional models are established respectively, which can effectively reduce the upper limit requirements for hardware resources when training the scene three-dimensional model. The three-dimensional model of each block area is also easier to converge to a good effect. At the same time, it is also suitable for distributed training on multiple machines and multiple cards to reduce training time.

[0057] In an optional embodiment, step S101, dividing the target scene into a plurality of block areas, includes:

[0058] The scale information of the target scene is adjusted to the target scale; and the target scene is divided into a plurality of block areas according to a preset segmentation size.

[0059] In this embodiment, since the calibrated scale of the target scene may not correspond to the absolute scale of the real scene, or the scale space corresponding to the calibrated scale of the target scene is smaller or larger, when dividing the target scene into multiple block areas, the scale information of the target scene can be first adjusted to the target scale, and then the adjusted scale target scene can be divided into multiple block areas according to a preset segmentation size.

[0060] In specific implementation, if the calibration scale of the target scene does not correspond to the absolute scale of the real scene, the calibration scale of the target scene can be adjusted to the absolute scale of the real scene, so that the calibration scale of the target scene is the same as the absolute scale of the real scene, which is convenient for dividing the target scene according to the actual segmentation size. If the scale space corresponding to the calibration scale of the target scene is small, the scale space corresponding to the calibration scale of the target scene can be enlarged to adjust the scale space corresponding to the calibration scale of the target scene to the target scale, which is convenient for dividing the target scene. If the scale space corresponding to the calibration scale of the target scene is large, the scale space corresponding to the calibration scale of the target scene can be reduced to adjust the scale space corresponding to the calibration scale of the target scene to the target scale, which is convenient for dividing the target scene.

[0061] In this embodiment, the scale information of the target scene is adjusted to the target scale to facilitate the division of the target scene, and then the target scene is divided into multiple block areas according to the preset segmentation size, so that the target scene can be divided into multiple uniform blocks, each of which has the same size.

[0062] In an optional embodiment, before determining the first area and the second area in the target scene in step S103, the model determination method further includes:

[0063] Based on the camera position corresponding to each scene image, two target camera positions that meet the distance condition are determined; and the directions of the coordinate axes in the coordinate system of the target scene are adjusted according to the directions corresponding to the lines connecting the two target camera positions.

[0064] In this embodiment, the distance condition may be: two camera positions with the largest spatial distance are used as the two target camera positions.

[0065] In this embodiment, after adjusting the direction of the coordinate axis in the coordinate system of the target scene according to the direction corresponding to the line between the two target camera positions, the camera position corresponding to each scene image can be adjusted to the new coordinate system, and the new coordinate system is the coordinate system obtained after the direction of the coordinate axis is adjusted.

[0066] In specific implementation, the spatial distance between each two camera positions can be calculated based on the camera position corresponding to each scene image, and then the two camera positions with the largest spatial distance can be determined as the two target camera positions. Then, the line between the two target camera positions is used as the direction of one of the coordinate axes, such as the direction of the x-axis, to rotate the coordinate system to obtain a new coordinate system. Then, the camera position corresponding to each scene image is adjusted to the new coordinate system. For example, the camera positions corresponding to all scene images are as follows: Figure 2As shown in (a), by connecting the positions of each scene image, a convex hull can be obtained. The longest connecting line direction of the convex hull vertices is used as the direction of the x-axis, and the coordinate system is rotated to obtain a new coordinate system. Then the position of each scene image can be adjusted to the position in the new coordinate system, as shown in Figure 2 (b) as shown.

[0067] In this embodiment, by determining two target camera positions that meet the distance condition and adjusting the direction of the coordinate axis in the coordinate system of the target scene according to the direction corresponding to the line connecting the two target camera positions, the positions of all scene images can be distributed more evenly in the target scene, thereby avoiding the situation where the positions of all scene images are concentratedly or unevenly distributed in the target scene.

[0068] In an optional embodiment, before determining the first area and the second area in the target scene in step S103, the model determination method further includes:

[0069] The camera position corresponding to each scene image is mapped to the target plane to obtain the mapped camera position; a first plane area and a second plane area are determined in the target plane, the first plane area is used to determine the first area, and the second plane area is used to determine the second area.

[0070] Step S104, based on the camera position corresponding to each scene image, determining the target scene image corresponding to the second area from the multiple scene images, including: based on the mapping camera position corresponding to each scene image, determining the target scene image corresponding to the second area from the multiple scene images, the mapping camera position of the target scene image is located within the second plane area.

[0071] In this embodiment, the target plane includes but is not limited to an xz plane, an xy plane or a yz plane.

[0072] In specific implementation, the camera position corresponding to each scene image can be mapped to the target plane to obtain the mapped camera position, such as Figure 3 As shown in (a), the camera position corresponding to each scene image is converted from a three-dimensional space coordinate to a two-dimensional plane coordinate, and the number of camera positions corresponding to all scene images remains unchanged. Then, the rectangular bounding box that maps the camera position on the target plane can be determined, and the rectangular bounding box can be divided into multiple sub-rectangular bounding boxes according to the preset segmentation size s. The first plane area is determined to be any sub-rectangular bounding box, such as Figure 3 As shown in (b), for the determined first plane area, it is extended to the surroundings by a preset size, such as s / 2, to obtain the second plane area. Thus, the first area can be obtained by longitudinally extending along the first plane area, and the second area can be obtained by longitudinally extending along the second plane area.

[0073] Accordingly, when determining the target scene image corresponding to the second area from multiple scene images based on the camera position corresponding to each scene image, the target scene image corresponding to the second area can be determined from multiple scene images based on the mapping camera position corresponding to each scene image, and the mapping camera position of the target scene image is located within the second plane area.

[0074] In this embodiment, the camera position corresponding to each scene image is mapped to the target plane to obtain a mapped camera position; a first plane area and a second plane area are determined in the target plane, the first plane area is used to determine the first area, and the second plane area is used to determine the second area; thereby, the scene image corresponding to the first plane area is consistent with the scene image corresponding to the first area, and the scene image corresponding to the second plane area is consistent with the scene image corresponding to the second area, so that it is possible to determine the target scene image corresponding to the second area from multiple scene images based on the camera position of each scene image on the plane, and the computational complexity of determining the target scene image can be simplified.

[0075] In an optional embodiment, step S104, based on the camera position corresponding to each scene image, a target scene image corresponding to the second area is determined from the multiple scene images, such as Figure 4 As shown, including:

[0076] S1041, based on the camera position corresponding to each scene image, determine a candidate scene image corresponding to the second area from a plurality of scene images, the camera position of the candidate scene image being located within the second area.

[0077] S1042, determining a target scene image from the candidate scene images, the number of target three-dimensional feature points corresponding to the target scene image is greater than a set threshold, the target three-dimensional feature points are three-dimensional feature points corresponding to the first area in the target point cloud, and the target point cloud is determined based on multiple scene images.

[0078] In this embodiment, for step S1041, the candidate scene images may be all scene images whose camera positions are located in the second area. In specific implementation, by comparing the camera position of each scene image with the position of the second area, it is possible to determine whether the position of the scene image is located in the second area, thereby determining whether the scene image is a candidate scene image for the second area.

[0079] For step S1042, the target point cloud corresponding to the target scene can be determined through multiple scene images, and the three-dimensional feature points corresponding to each alternative scene image can be determined. Then, based on the position of the first area, the target three-dimensional feature points corresponding to each alternative scene image can be determined. Then, based on the target three-dimensional feature points corresponding to each alternative scene image, the number of target three-dimensional feature points corresponding to each alternative scene image can be determined. If the number of target three-dimensional feature points corresponding to the current alternative scene image is greater than a set threshold, the current alternative scene image is the target scene image. If the number of target three-dimensional feature points corresponding to the current alternative scene image is less than or equal to the set threshold, the current alternative scene image is not the target scene image.

[0080] In this embodiment, by setting the number of target three-dimensional feature points corresponding to the target scene image to be greater than a set threshold, scene images that have a greater contribution to the three-dimensional modeling of the first area can be selected from the scene images whose camera position is within the second area, and invalid scene images collected within the second area can be effectively eliminated, so that the three-dimensional model of the first area has higher adaptability and stability.

[0081] In an optional embodiment, step S105, a three-dimensional model corresponding to the first area is determined based on the target scene image, such as Figure 5 As shown, including:

[0082] S1051, performing a first screening process on the target scene image whose camera position is outside the first area to obtain a first screened image.

[0083] S1052: Determine a three-dimensional model corresponding to the first area based on the first screening image and the target scene image whose camera position is within the first area.

[0084] In this embodiment, with respect to S1051, the target scene image includes the target scene image whose camera position is located within the first area, and the target scene image whose camera position is located outside the first area.

[0085] For target scene images whose camera position is within the first area, all of them can be used to determine the three-dimensional model corresponding to the first area. For target scene images whose camera position is outside the first area, a first screening process can be performed to obtain a first screening image, which is used to determine the three-dimensional model corresponding to the first area.

[0086] In specific implementation, for each target scene image whose camera position is outside the first area, the corresponding first probability can be calculated, and each target scene image is screened based on the first probability of each target scene image. If selected, the target scene image is the first screened image. The calculation formula of the first probability is:

[0087] Among them, N vis is the number of target 3D feature points corresponding to the target scene image, the target 3D feature points are the 3D feature points corresponding to the first region in the target point cloud, and the target point cloud is determined based on multiple scene images. N whole are all the three-dimensional feature points corresponding to the target scene image.

[0088] With respect to S1052, once the first screening image is determined, a three-dimensional model corresponding to the first area can be determined based on the first screening image and the target scene image whose camera position is within the first area.

[0089] In some implementations, the number n of the first screening images and the target scene images whose camera positions are within the first area may be counted, and if n>T max , then randomly remove some scene images to make the number of scene images equal to T max , where T max It is determined by the performance of the electronic device used to train the three-dimensional model corresponding to the first area; if n <T min , it is considered that the scene contained in the first area is too small and the first area is deleted.

[0090] In this embodiment, a first screening image is obtained by performing a first screening process on the target scene image whose camera position is outside the first area; a three-dimensional model corresponding to the first area is determined based on the first screening image and the target scene image whose camera position is within the first area; instead of selecting all target scene images outside the first area for three-dimensional modeling of the first area, the proportion of target scene images outside the first area can be reduced, thereby improving the stability of the three-dimensional model of the first area.

[0091] In an optional embodiment, step S1052, determining a three-dimensional model corresponding to the first area based on the first screening image and the target scene image with the camera position located within the first area, includes:

[0092] Determine a peripheral scene image from multiple scene images; the camera position of the peripheral scene image is located outside the second area; perform a second screening process on the peripheral scene image to obtain a second screened image, and the selection ratio corresponding to the second screening process is lower than the selection ratio corresponding to the first screening process; determine the three-dimensional model corresponding to the first area based on the first screening image, the second screening image and the target scene image with the camera position within the first area.

[0093] In this embodiment, when determining the three-dimensional model corresponding to the first area, a scene image may also be selected from scene images where the camera position is outside the second area, that is, a peripheral scene image is selected.

[0094] In a specific implementation, based on the camera position corresponding to each scene image, the scene image whose camera position is outside the second area can be determined, and then for each scene image whose camera position is outside the second area, if the number of target three-dimensional feature points corresponding to the scene image is greater than a first set threshold, the scene image is determined to be a peripheral scene image. The target three-dimensional feature points are three-dimensional feature points corresponding to the first area in the target point cloud, the target point cloud is determined based on multiple scene images, and the first preset threshold is less than the set threshold in the above embodiment.

[0095] Then, for each peripheral scene image whose camera position is outside the second area, the corresponding second probability can be calculated, and each peripheral scene image is screened based on the second probability of each peripheral scene image. If selected, the peripheral scene image is the second screened image. The calculation formula of the second probability is:

[0096] Among them, N vis is the number of target 3D feature points corresponding to the peripheral scene image, the target 3D feature points are the 3D feature points corresponding to the first area in the target point cloud, and the target point cloud is determined based on multiple scene images. N whole are all the three-dimensional feature points corresponding to the peripheral scene image.

[0097] In this embodiment, a peripheral scene image is determined from a plurality of scene images; the camera position of the peripheral scene image is located outside the second area; a second screening process is performed on the peripheral scene image to obtain a second screened image, and the selection ratio corresponding to the second screening process is lower than the selection ratio corresponding to the first screening process; a three-dimensional model corresponding to the first area is determined based on the first screening image, the second screening image and the target scene image whose camera position is located within the first area; thereby, scene images that have a greater contribution to the three-dimensional modeling of the first area can be selected from outside the second area for use in the three-dimensional modeling of the first area, thereby taking into account the contribution of scene images at different camera positions to the construction of the three-dimensional model of the first area, and improving the stability of the three-dimensional model of the first area; and, a second screening process is performed on the peripheral scene image, and the selection ratio corresponding to the second screening process is lower than the selection ratio corresponding to the first screening process, and thus the ratio of the target scene images outside the second area can be reduced, and the stability of the three-dimensional model of the first area can be further improved.

[0098] In an optional embodiment, in step S105, after determining the three-dimensional model corresponding to the first area based on the target scene image, the model determination method further includes:

[0099] The three-dimensional model corresponding to the first area and the three-dimensional model corresponding to at least one third area are spliced ​​to obtain a three-dimensional model corresponding to the target scene, the third area is at least one of the block areas, and the third area is different from the first area.

[0100] In this embodiment, the three-dimensional model corresponding to the first area and the three-dimensional model corresponding to the at least one third area may be spliced ​​based on the positions of the first area and the at least one third area in the target scene.

[0101] In this embodiment, a complete three-dimensional model corresponding to the target scene can be obtained by splicing the three-dimensional modeling corresponding to the first area and the three-dimensional model corresponding to at least one third area.

[0102] In an optional embodiment, step S105, determining the three-dimensional model corresponding to the first area based on the target scene image, includes:

[0103] An initialization point cloud corresponding to the first area is obtained, wherein the initialization point cloud includes a plurality of Gaussian volumes to be trained; and the initialization point cloud is trained based on the target scene image to obtain a three-dimensional model corresponding to the first area.

[0104] In this embodiment, based on multiple scene images, a target point cloud can be determined. Then, based on the position of the first area and the target point cloud, all three-dimensional feature points corresponding to the first area can be determined to obtain an initialization point cloud corresponding to the first area. Then, the initialization point cloud can be trained based on the target scene image to obtain a three-dimensional model corresponding to the first area.

[0105] In this embodiment, an initialization point cloud corresponding to the first area is obtained; the initialization point cloud is trained based on the target scene image to obtain a three-dimensional model corresponding to the first area; in this way, the three-dimensional model of the first area is constructed using the three-dimensional Gaussians Splatting (3DGS) technology, and a three-dimensional model of the first area with better quality can be obtained.

[0106] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0107] Figure 6 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0108] like Figure 6As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0109] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0110] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the model determination method. For example, in some embodiments, the model determination method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the model determination method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the model determination method in any other appropriate manner (e.g., by means of firmware).

[0111] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0112] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0113] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0114] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0115] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0116] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0117] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0118] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0119] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model determination method, comprising: Divide the target scene into multiple block areas; Acquire scene image data, where the scene image data includes a plurality of scene images corresponding to a target scene and a camera position corresponding to each scene image; Determine a first area and a second area in a target scene, wherein the first area is one of the plurality of block areas, the second area includes the first area, and the second area is larger than the first area; Based on the camera position corresponding to each scene image, determine a target scene image corresponding to the second area from the multiple scene images, the camera position of the target scene image being located within the second area; A three-dimensional model corresponding to the first area is determined based on the target scene image.

2. The model determination method according to claim 1, before determining the first area and the second area in the target scene, further comprising: Based on the camera position corresponding to each scene image, two target camera positions that meet the distance condition are determined; The directions of the coordinate axes in the coordinate system of the target scene are adjusted according to the direction corresponding to the line connecting the two target camera positions.

3. The model determination method according to claim 1, wherein determining the target scene image corresponding to the second area from the plurality of scene images based on the camera position corresponding to each scene image comprises: Based on the camera position corresponding to each scene image, determine a candidate scene image corresponding to the second area from the multiple scene images, the camera position of the candidate scene image being located within the second area; The target scene image is determined from the candidate scene images, the number of target three-dimensional feature points corresponding to the target scene image is greater than a set threshold, the target three-dimensional feature points are three-dimensional feature points in a target point cloud corresponding to the first area, and the target point cloud is determined based on the multiple scene images.

4. The model determination method according to claim 1, wherein determining the three-dimensional model corresponding to the first area based on the target scene image comprises: Performing a first screening process on the target scene image whose camera position is outside the first area to obtain a first screened image; A three-dimensional model corresponding to the first area is determined based on the first screening image and the target scene image with a camera position within the first area.

5. The model determination method according to claim 4, wherein determining the three-dimensional model corresponding to the first area based on the first screening image and the target scene image with the camera position located within the first area comprises: determining a peripheral scene image from the plurality of scene images; The camera position of the peripheral scene image is outside the second area; Performing a second screening process on the peripheral scene image to obtain a second screened image, wherein the selected ratio corresponding to the second screening process is lower than the selected ratio corresponding to the first screening process; A three-dimensional model corresponding to the first area is determined based on the first screening image, the second screening image, and the target scene image with a camera position within the first area.

6. The model determination method according to claim 1, further comprising: Map the camera position corresponding to each scene image to the target plane to obtain the mapped camera position; Determine a first plane area and a second plane area in the target plane, wherein the first plane area is used to determine the first area, and the second plane area is used to determine the second area; The step of determining a target scene image corresponding to the second area from the multiple scene images based on a camera position corresponding to each scene image comprises: determining a target scene image corresponding to the second area from the multiple scene images based on a mapping camera position corresponding to each scene image, wherein the mapping camera position of the target scene image is located within the second plane area.

7. The model determination method according to claim 1, wherein dividing the target scene into a plurality of block areas comprises: Adjusting the scale information of the target scene to a target scale; The target scene is divided into a plurality of block areas according to a preset segmentation size.

8. The model determination method according to claim 1, after determining the three-dimensional model corresponding to the first area based on the target scene image, the method further comprises: The three-dimensional model corresponding to the first area and the three-dimensional model corresponding to at least one third area are spliced ​​to obtain a three-dimensional model corresponding to the target scene, the third area is at least one of the block areas, and the third area is different from the first area.

9. The model determination method according to claim 1, determining the three-dimensional model corresponding to the first area based on the target scene image, comprising: Obtaining an initialization point cloud corresponding to the first area, wherein the initialization point cloud includes a plurality of Gaussian volumes to be trained; The initialization point cloud is trained based on the target scene image to obtain a three-dimensional model corresponding to the first area.

10. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the model determination method as described in any one of claims 1-9.

Images