A three-dimensional scene reconstruction method, device, equipment and storage medium

Abstract

The present application provides a three-dimensional scene reconstruction method, device, equipment and storage medium, which relate to the field of image processing technology. The method includes: extracting features and matching features of multiple frames of images corresponding to the scene to be reconstructed, and obtaining feature information of each frame of image, the feature information including: the number of feature points included in each frame of image, and the number of matching feature points between each frame of image and adjacent frame of image; determining the scene type of the scene to be reconstructed from multiple preset scene types according to the feature information of each frame of image; determining the target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed; reconstructing the scene to be reconstructed based on the target scene reconstruction strategy, and obtaining a three-dimensional model of the scene to be reconstructed. By using the embodiments of the present application, a three-dimensional reconstruction method can be selected in a targeted manner to improve the accuracy of three-dimensional reconstruction.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a three-dimensional scene reconstruction method, device, equipment and storage medium. Background Art

[0002] Three-dimensional reconstruction technology mainly includes the steps of preliminary model data acquisition and preprocessing, point cloud data registration and fusion, point cloud data surface reconstruction, etc., and finally converts the real target scene into a mathematical model that can be displayed by the computer.

[0003] Currently, after acquiring image data of a scene to be reconstructed, a certain 3D reconstruction method may be selected to construct a 3D model of the scene to be reconstructed. The 3D reconstruction method may be, for example, a traditional SIFT feature extraction reconstruction method or a deep learning method.

[0004] However, there may be deviations between the three-dimensional model constructed by existing technologies and the real information of the scene to be reconstructed, making it difficult to ensure the quality of the three-dimensional model. Summary of the invention

[0005] The purpose of the present application is to provide a three-dimensional scene reconstruction method, device, equipment and storage medium to address the deficiencies in the above-mentioned prior art, which can selectively select a three-dimensional reconstruction method and improve the accuracy of three-dimensional reconstruction.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, an embodiment of the present application provides a three-dimensional scene reconstruction method, the method comprising:

[0008] Performing feature extraction and feature matching on multiple frames of images corresponding to the scene to be reconstructed to obtain feature information of each frame of image, wherein the feature information includes: the number of feature points included in each frame of image, and the number of matching feature points between each frame of image and adjacent frame of image;

[0009] Determining the scene type of the scene to be reconstructed from a plurality of preset scene types according to feature information of each frame of image;

[0010] Determining a target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed;

[0011] The scene to be reconstructed is reconstructed based on the target scene reconstruction strategy to obtain a three-dimensional model of the scene to be reconstructed.

[0012] Optionally, determining the scene type of the scene to be reconstructed from a plurality of preset scene types according to the feature information of each frame image includes:

[0013] According to the number of feature points included in each frame image and the number of matching feature points between each frame image and an adjacent frame image, a matching success rate corresponding to each frame image is obtained;

[0014] Determine the image type of each frame image according to the matching success rate corresponding to each frame image, wherein the image type includes: a pure repetitive texture type or a non-pure repetitive texture type;

[0015] According to the image type of each frame image, the scene type of the scene to be reconstructed is determined from a plurality of preset scene types.

[0016] Optionally, determining the image type of each frame of image according to the matching success rate corresponding to each frame of image includes:

[0017] If the matching success rate corresponding to the current image is less than the first preset matching success rate, determining that the image type of the current image is the pure repetitive texture type; otherwise, determining that the image type of the current image is the non-pure repetitive texture type;

[0018] The current image is any one of the frames.

[0019] Optionally, determining the scene type of the scene to be reconstructed from a plurality of preset scene types according to the image type of each frame of image includes:

[0020] If the quantity and position information of the first images belonging to the pure repetitive texture type in each frame image meets the first preset condition, the scene type of the scene to be reconstructed is determined to be a pure repetitive texture scene among the multiple preset scene types, and the quantity and position information includes at least one of the following: the number of the first images and whether there is an adjacent relationship between the first images.

[0021] Optionally, when the scene to be reconstructed is not a pure repetitive texture scene, determining the scene type of the scene to be reconstructed from a plurality of preset scene types according to the image type of each frame image includes:

[0022] Acquire at least one second image belonging to the non-purely repeated texture type in each frame image;

[0023] Performing grid division on each of the second images to obtain a plurality of grids corresponding to each of the second images;

[0024] Determining the number of matching sub-feature points included in each grid in each second image according to position information of matching feature points between each second image and an adjacent frame image in each second image;

[0025] Determining a matching success rate corresponding to each grid in each second image according to the number of matching sub-feature points included in each grid in each second image and the number of matching feature points in the feature information of each second image;

[0026] Determining the image subtype of each second image according to the matching success rate corresponding to each grid in each second image and the second preset matching success rate, wherein the image subtype includes: a semi-repeated texture type and a rich texture type;

[0027] According to the image subtype of each of the second images, the scene type of the scene to be reconstructed is determined from a plurality of preset scene types.

[0028] Optionally, determining the image subtype of each second image according to the matching success rate corresponding to each grid in each second image and a second preset matching success rate includes:

[0029] Binarize each grid in the current second image according to the matching success rate corresponding to each grid in the current second image and the second preset matching success rate, and determine the initial pixel value of each grid, wherein the current second image is any one of the second images;

[0030] The image subtype of the current second image is determined according to the initial pixel values ​​of each grid.

[0031] Optionally, determining the image subtype of the current second image according to the initial pixel values ​​of each grid includes:

[0032] Determine the number of grids in each grid whose initial pixel value is a preset value;

[0033] If the number of grids is greater than or equal to the preset number of grids, the image subtype of the current second image is determined to be the semi-repeating texture type; otherwise, the image subtype of the current second image is determined to be the rich texture type.

[0034] Optionally, determining the image subtype of the current second image according to the initial pixel values ​​of each grid includes:

[0035] According to the initial pixel value of each grid and the preset connected domain, the initial pixel value of each grid is corrected to obtain the target pixel value of each grid;

[0036] The image subtype of the current second image is determined according to the target pixel value of each grid.

[0037] Optionally, determining the image subtype of the current second image according to the target pixel value of each grid includes:

[0038] Merge each grid into multiple blocks according to the preset number of blocks;

[0039] According to the target pixel value of each grid, determine the pixel value corresponding to each block;

[0040] The image subtype of the current second image is determined according to the pixel values ​​corresponding to each block.

[0041] Optionally, the determining the scene type of the to-be-reconstructed scene from a plurality of preset scene types according to the image subtype of each of the second images includes:

[0042] If the number of images belonging to the semi-repeating texture type in each of the second images is greater than the number of images belonging to the rich texture type in each of the second images, the scene type of the scene to be reconstructed is determined to be a semi-repeating texture scene among the multiple preset scene types; otherwise, the scene type of the scene to be reconstructed is determined to be a rich texture scene among the multiple preset scene types.

[0043] Optionally, determining the target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed includes:

[0044] According to the scene type of the scene to be reconstructed and the corresponding relationship between the scene type and the scene reconstruction strategy, a target scene reconstruction strategy corresponding to the scene to be reconstructed is determined.

[0045] In a second aspect, an embodiment of the present application further provides a three-dimensional scene reconstruction device, the device comprising:

[0046] A feature processing module is used to extract and match features of multiple frames of images corresponding to the scene to be reconstructed, and obtain feature information of each frame of image, wherein the feature information includes: the number of feature points included in each frame of image, and the number of matching feature points between each frame of image and adjacent frame of image;

[0047] A first determination module, used to determine the scene type of the scene to be reconstructed from a plurality of preset scene types according to feature information of each frame image;

[0048] A second determination module, used to determine a target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed;

[0049] A reconstruction module is used to reconstruct the scene to be reconstructed based on the target scene reconstruction strategy to obtain a three-dimensional model of the scene to be reconstructed.

[0050] Optionally, the first determination module is specifically used to obtain the matching success rate corresponding to each frame image according to the number of feature points included in each frame image and the number of matching feature points between each frame image and adjacent frame images; determine the image type of each frame image according to the matching success rate corresponding to each frame image, and the image type includes: pure repetitive texture type or non-pure repetitive texture type; determine the scene type of the scene to be reconstructed from multiple preset scene types according to the image type of each frame image.

[0051] Optionally, the first determination module is further specifically used to determine that the image type of the current image is the pure repetitive texture type if the matching success rate corresponding to the current image is less than a first preset matching success rate; otherwise, determine that the image type of the current image is the non-pure repetitive texture type; wherein the current image is any one frame image among each frame image.

[0052] Optionally, the first determination module is further specifically used to determine that the scene type of the scene to be reconstructed is a pure repetitive texture scene among the multiple preset scene types if the quantity and position information of the first images belonging to the pure repetitive texture type in each frame image meets a first preset condition, and the quantity and position information includes at least one of the following: the number of the first images and whether there is an adjacent relationship between the first images.

[0053] Optionally, when the scene to be reconstructed is not a pure repetitive texture scene, the first determination module is further used to obtain at least one second image belonging to the non-pure repetitive texture type in each frame image; grid-divide each second image to obtain a plurality of grids corresponding to each second image; determine the number of matching sub-feature points included in each grid in each second image according to the position information of the matching feature points between each second image and the adjacent frame image in each second image; determine the matching success rate corresponding to each grid in each second image according to the number of matching sub-feature points included in each grid in each second image and the number of matching feature points in the feature information of each second image; determine the image subtype of each second image according to the matching success rate corresponding to each grid in each second image and the second preset matching success rate, the image subtype including: semi-repetitive texture type and rich texture type; determine the scene type of the scene to be reconstructed from a plurality of preset scene types according to the image subtype of each second image.

[0054] Optionally, the first determination module is further specifically used to binarize each grid in the current second image according to the matching success rate corresponding to each grid in the current second image and the second preset matching success rate, and determine the initial pixel value of each grid, wherein the current second image is any one of the second images; and determine the image subtype of the current second image according to the initial pixel value of each grid.

[0055] Optionally, the first determination module is further specifically used to determine the number of grids in each grid whose initial pixel value is a preset value; if the number of grids is greater than or equal to the preset number of grids, the image subtype of the current second image is determined to be the semi-repeating texture type; otherwise, the image subtype of the current second image is determined to be the rich texture type.

[0056] Optionally, the first determination module is further specifically used to correct the initial pixel value of each grid according to the initial pixel value of each grid and a preset connected domain to obtain a target pixel value of each grid; and determine the image subtype of the current second image according to the target pixel value of each grid.

[0057] Optionally, the first determination module is further specifically used to merge each grid into multiple blocks according to a preset number of blocks; determine the pixel value corresponding to each block according to the target pixel value of each grid; and determine the image subtype of the current second image according to the pixel value corresponding to each block.

[0058] Optionally, the first determination module is further specifically used to determine that the scene type of the scene to be reconstructed is a semi-repeating texture scene among the multiple preset scene types if the number of images belonging to the semi-repeating texture type in each of the second images is greater than the number of images belonging to the rich texture type in each of the second images; otherwise, determine that the scene type of the scene to be reconstructed is a rich texture scene among the multiple preset scene types.

[0059] Optionally, the second determination module is specifically configured to determine a target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed and a correspondence between the scene type and the scene reconstruction strategy.

[0060] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the three-dimensional scene reconstruction method of the first aspect above.

[0061] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the three-dimensional scene reconstruction method of the first aspect are executed.

[0062] The beneficial effects of this application are:

[0063] The embodiments of the present application provide a three-dimensional scene reconstruction method, apparatus, device and storage medium, the method comprising: performing feature extraction and feature matching on multiple frame images corresponding to a scene to be reconstructed, to obtain feature information of each frame image, the feature information comprising: the number of feature points included in each frame image, and the number of matching feature points between each frame image and adjacent frame images; determining the scene type of the scene to be reconstructed based on the feature information of each frame image; determining a target scene reconstruction strategy corresponding to the scene to be reconstructed from a plurality of preset scene types based on the scene type of the scene to be reconstructed; and reconstructing the scene to be reconstructed based on the target scene reconstruction strategy to obtain a three-dimensional model of the scene to be reconstructed.

[0064] By adopting the three-dimensional scene reconstruction method provided in the embodiment of the present application, the scene type of the scene to be reconstructed is determined by acquiring the feature information of each frame image corresponding to the scene to be reconstructed, and the target scene reconstruction strategy corresponding to the scene to be reconstructed can be determined according to the mapping relationship between the scene type and the scene reconstruction strategy. Then, the scene to be reconstructed can be reconstructed according to the target scene reconstruction strategy corresponding to the scene to be reconstructed to obtain the three-dimensional model of the scene to be reconstructed. In this way, the blind selection of scene reconstruction strategy can be avoided, which will cause the poor quality of the constructed three-dimensional model, thereby improving the accuracy of three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 A schematic diagram of the structure of a three-dimensional scene reconstruction system provided in an embodiment of the present application;

[0067] Figure 2 A schematic diagram of a flow chart of a three-dimensional scene reconstruction method provided in an embodiment of the present application;

[0068] Figure 3 A schematic diagram of a flow chart of another three-dimensional scene reconstruction method provided in an embodiment of the present application;

[0069] Figure 4 A schematic diagram of a flow chart of another three-dimensional scene reconstruction method provided in an embodiment of the present application;

[0070] Figure 5 A schematic diagram of a flow chart of another three-dimensional scene reconstruction method provided in an embodiment of the present application;

[0071] Figure 6A schematic diagram of a flow chart of another three-dimensional scene reconstruction method provided in an embodiment of the present application;

[0072] Figure 7 A schematic diagram of the structure of a three-dimensional scene reconstruction device provided in an embodiment of the present application;

[0073] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0075] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0076] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0077] Before explaining the embodiments of the present application in detail, the application scenario of the present application is first introduced. The application scenario can be specifically the three-dimensional scene reconstruction of farmland, buildings or other scenes to be reconstructed. It should be noted that the present application does not limit the content of the scene to be reconstructed.

[0078] Figure 1 A schematic diagram of the structure of a three-dimensional scene reconstruction system provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the system may include an image acquisition device 101 and a data processing device 102, and the image acquisition device 101 is communicatively connected with the data processing device 102. The image acquisition device 101 may capture a scene to be reconstructed to obtain a series of images, and the specific form of the image acquisition device 101 may be a camera, such as a monocular camera, a binocular camera, or a trinocular camera. In an exemplary embodiment, the image acquisition device 101 may be pre-set on a drone, and the drone may perform a shooting task on the scene to be reconstructed according to a preset route based on pre-set flight parameters and shooting parameters.

[0079] The image acquisition device 101 sends the captured images to the data processing device 102. After acquiring the images corresponding to the scene to be reconstructed, the data processing device 102 can process the multiple images according to the following embodiments to determine the scene type of the scene to be reconstructed. The data processing device 102 can also perform three-dimensional reconstruction of the scene to be reconstructed using a target scene reconstruction strategy corresponding to the scene type of the scene to be reconstructed.

[0080] The data processing device 102 may be a general-purpose computer or a computer device composed of a dedicated integrated circuit, etc., and the present application embodiment does not limit this. For example, the data processing device 110 may be a mobile terminal device such as a tablet computer, or may also be a personal computer (PC). Those skilled in the art may know that the number of the above-mentioned data processing devices 102 may be one or more, and their types may be the same or different. The present application embodiment does not limit the number and device type of the data processing devices 102.

[0081] It should be noted that the data processing device 102 can be set on a drone or on the ground, and this application does not limit it.

[0082] The three-dimensional scene reconstruction method mentioned in the present application is illustrated as follows with reference to the accompanying drawings. Figure 2 A schematic diagram of a three-dimensional scene reconstruction method provided in an embodiment of the present application. Figure 2 As shown, the method may include:

[0083] S201 , performing feature extraction and feature matching on multiple frames of images corresponding to a scene to be reconstructed, to obtain feature information of each frame of image.

[0084] The feature information includes: the number of feature points included in each frame image, and the number of matching feature points between each frame image and the adjacent frame image. For a frame image, the feature information of the image includes the number of feature points included in the image and the number of matching feature points between the image and the adjacent images. For example, assuming that multiple frames of images include image 1, image 2, image 3, etc., the feature information of image 1 includes the number of feature points in image 1 and the number of matching feature points between image 1 and image 2, the feature information of image 2 includes the number of feature points in image 2 and the number of matching feature points between image 2 and image 3, and so on.

[0085] It can be understood that the image acquisition device can capture a series of images in the scene to be reconstructed. For example, continuous multiple frames of images can be extracted from the series of images according to actual needs, and then the data processing device can process the extracted continuous multiple frames of images to obtain the characteristic information of the preset multiple frames of continuous images (such as 20 frames), and then determine the scene type of the scene to be reconstructed according to the characteristic information of the images.

[0086] The feature extraction mentioned above can be understood as using the detector to extract points in the image with relatively large neighborhood changes, including corner points, edge points, intersection points and other points with rich image content. These points are called feature points, so that the number of feature points included in each frame image can be obtained. After the feature points are extracted, it is necessary to use the neighborhood information of the feature points to calculate and describe them in order to form a descriptor feature vector that is easy to match and has good stability. Among them, the descriptor can be SIFT (Scale-invariant feature transform), of course, it can be other types of descriptors, and this application does not limit it. Feature matching can be understood as matching based on the similarity of the descriptor feature vector. This application uses adjacent frame images for matching, and the number of matching feature points between each frame image and the adjacent frame image can be obtained. It should be noted that the detailed content of feature extraction and feature matching can refer to the relevant description, and will not be explained here.

[0087] S202: Determine a scene type of a scene to be reconstructed from a plurality of preset scene types according to feature information of each frame of image.

[0088] Among them, multiple preset scene types may include pure repetitive texture scenes, semi-repetitive texture scenes, rich texture scenes, and of course other texture scenes, which are not limited in this application. As an exemplary example, the scene types may be divided into multiple preset scene types in advance according to the complexity of the scene texture. It can be understood that the higher the complexity of the scene texture, the more it matches the pure repetitive texture scene, and the higher the complexity of the scene texture, the more it matches the rich texture scene. In other words, the pure repetitive texture scene is used to indicate that the scene to be reconstructed is a scene with strong texture repetitiveness, such as lawns and farmland; the semi-repetitive texture scene is used to indicate that the scene to be reconstructed is a scene including repetitive texture areas and non-repetitive texture areas, such as a scene with roads; the rich texture scene is used to indicate that the scene to be reconstructed is a scene with weak texture repetitiveness, such as a building.

[0089] According to the above description, each frame image corresponds to the number of feature points and the number of matching feature points. Therefore, the scene type of the scene to be reconstructed can be determined according to the number of feature points and the number of matching feature points corresponding to each frame image. It can be determined whether the scene to be reconstructed is a pure repetitive texture scene. When the scene to be reconstructed is not a pure repetitive texture scene, it can be determined whether the scene type of the scene to be reconstructed that is not a pure repetitive texture scene is a semi-repetitive texture scene or a rich texture scene.

[0090] S203: Determine a target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed.

[0091] S204: Reconstruct the scene to be reconstructed based on the target scene reconstruction strategy to obtain a three-dimensional model of the scene to be reconstructed.

[0092] In an exemplary embodiment, the scene type and the scene reconstruction strategy have a preset corresponding relationship. After the scene type of the scene to be reconstructed is determined, the scene reconstruction strategy corresponding to the scene type of the scene to be reconstructed can be determined based on the preset corresponding relationship between the scene type and the scene reconstruction strategy. This scene reconstruction strategy is called the target scene reconstruction strategy.

[0093] Another exemplary method is that after the scene type of the scene to be reconstructed is determined, the optimal scene reconstruction strategy corresponding to the scene type of the scene to be reconstructed can be statistically calculated based on historical data. The optimal scene reconstruction strategy is called the target scene reconstruction strategy. The optimal scene reconstruction strategy can be understood as a three-dimensional modeling method in which the constructed three-dimensional model best matches the real information of the scene to be reconstructed.

[0094] After the target scene reconstruction strategy is determined, the target scene reconstruction strategy can be used to perform three-dimensional scene reconstruction on the scene to be reconstructed to obtain a three-dimensional model of the scene to be reconstructed. It should be noted that the specific details of the three-dimensional scene reconstruction using the target scene reconstruction strategy can be referred to the relevant description and will not be explained here.

[0095] To summarize, in the three-dimensional scene reconstruction method provided in the embodiment of the present application, the scene type of the scene to be reconstructed is determined by acquiring the feature information of each frame image corresponding to the scene to be reconstructed, and the target scene reconstruction strategy corresponding to the scene to be reconstructed can be determined according to the mapping relationship between the scene type and the scene reconstruction strategy. Then, the scene to be reconstructed can be reconstructed according to the target scene reconstruction strategy corresponding to the scene to be reconstructed to obtain a three-dimensional model of the scene to be reconstructed. This can avoid the phenomenon of blindly selecting a scene reconstruction strategy, which results in poor quality of the constructed three-dimensional model.

[0096] At the same time, according to the feature information of each frame image, not only can it be determined whether the scene to be reconstructed is a pure repetitive texture scene, but also when it is not a pure repetitive texture scene, it can be determined whether the scene to be reconstructed is a semi-repetitive texture scene or a rich texture scene. In this way, on the premise of improving the accuracy of the scene type of the scene to be reconstructed, the accuracy of the three-dimensional model reconstructed using the target scene reconstruction strategy can be further improved.

[0097] Figure 3 A schematic diagram of another 3D scene reconstruction method provided in an embodiment of the present application. Figure 3 As shown, the above-mentioned determining the scene type of the scene to be reconstructed from a plurality of preset scene types according to the feature information of each frame image includes:

[0098] S301 : Obtain a matching success rate corresponding to each frame image according to the number of feature points included in each frame image and the number of matching feature points between each frame image and an adjacent frame image.

[0099] In an exemplary embodiment, it is assumed that multiple frames of images are extracted from a series of images captured by an image acquisition device and feature extraction and feature matching are performed on the multiple frames of images, thereby obtaining feature information of 20 consecutive frames of images, and the scene type of the scene to be reconstructed is determined using the feature information of the 20 consecutive frames of images. Feature extraction and feature matching are performed on each frame of images to obtain the number of feature points corresponding to each frame of images and the number of matching feature points with each adjacent frame of images. For a frame of image, assuming that the number of feature points corresponding to the frame of image is S, and the number of matching feature points between the frame of image and the adjacent frame of image is P, then the matching success rate corresponding to the image can be expressed as: (P / S)*100%. The matching success rates corresponding to other frame images can refer to the above description and will not be explained here. Finally, the matching success rate corresponding to each frame of image can be obtained.

[0100] S302: Determine the image type of each frame of image according to the matching success rate corresponding to each frame of image.

[0101] The image types include: pure repetitive texture type or non-pure repetitive texture type. It is understandable that for pure repetitive texture scenes, such as corn fields and wheat fields in farmland scenes, the images corresponding to pure repetitive texture scenes have very close neighborhood information and relatively low differentiation, making it difficult to extract feature points. Therefore, the matching success rate values ​​of the images corresponding to pure repetitive texture scenes are very small.

[0102] In an achievable embodiment, the matching success rate corresponding to each frame image is compared with the first preset matching success rate, and the image type of each frame image is determined based on the comparison result. Taking the current image in each frame image as an example, the current image can be any frame image in each frame image. Assuming that the first preset matching success rate is N%, N can be set according to actual needs, and this application does not limit it. The matching success rate corresponding to the current image is compared with N%. If the matching success rate corresponding to the current image is less than N%, the image type of the current image is a pure repetitive texture type; if the matching success rate corresponding to the current image is greater than or equal to N%, the image type of the current image is a non-pure repetitive texture type. Referring to the above description, the image type corresponding to each frame image can be obtained in the end.

[0103] S303: Determine the scene type of the scene to be reconstructed from a plurality of preset scene types according to the image type of each frame of image.

[0104] In an exemplary manner, the image identification of each frame of image can be determined according to the image type of each frame of image. Assuming that the image identification of an image whose image type is a pure repetitive texture type is set to 1, the image identification of an image whose image type is a non-pure repetitive texture type is 0. The scene type of the scene to be reconstructed can be determined according to the image identification of each frame of image. Specifically, the scene type of the scene to be repeated can be determined according to the number of images whose image identification is 1 and the position relationship. When the number of images whose image identification is 1 and the position relationship meet the first preset condition, the scene type of the scene to be repeated is a pure repetitive texture scene among multiple preset scene types, otherwise it is a non-pure repetitive texture scene among multiple preset scene types.

[0105] Optionally, the above-mentioned determining the scene type of the scene to be reconstructed from multiple preset scene types based on the image type of each frame image includes: if the quantity position information of the first images belonging to the pure repetitive texture type in each frame image meets the first preset condition, then determining that the scene type of the scene to be reconstructed is a pure repetitive texture scene among multiple preset scene types, the quantity position information includes at least one of the following: the number of first images, and whether there is an adjacent relationship between the first images.

[0106] According to the above description, each frame of image corresponds to an image type, and an image whose image type is a pure repetitive texture type can be called a first image. It should be noted that the present application does not limit the number of images whose image type is a pure repetitive texture type. Continuing with the above example, in an exemplary manner, the number of images with an image identifier of 1 is counted. When the number of images with an image identifier of 1 meets the quantity requirement in the first preset condition, such as when the number of images with an image identifier of 1 occupies at least half (such as 10 frames) of the selected images (such as 20 frames) corresponding to the scene to be reconstructed, the scene type of the scene to be reconstructed can be determined as a pure repetitive texture scene. In another exemplary manner, when the number of images with an image identifier of 1 meets the quantity requirement in the first preset condition, and the images with an image identifier of 1 meet the position requirement in the first preset condition, such as when the number of images with an image identifier of 1 occupies at least half (such as 10 frames) of the selected images (such as 20 frames) corresponding to the scene to be reconstructed, and the position relationship of the images with an image identifier of 1 is continuous, that is, there is an adjacent relationship between the first images, then the scene type of the scene to be reconstructed is a pure repetitive texture scene among multiple preset scene types.

[0107] If the quantity and position information of the first images belonging to the pure repetitive texture type in each frame image does not meet the first preset condition, it is determined that the scene to be reconstructed is not a pure repetitive texture scene, that is, the scene to be reconstructed is a non-pure repetitive texture scene, and the non-pure repetitive texture scene includes a semi-repetitive texture scene and a rich texture scene. The following embodiment continues to explain the process of determining whether the scene type of the scene to be reconstructed that is a non-pure repetitive texture scene is actually a semi-repetitive texture scene among multiple preset scene types, or a rich texture scene.

[0108] Figure 4 A flowchart of another three-dimensional scene reconstruction method provided in an embodiment of the present application is shown below. Figure 4 As shown, the above-mentioned determining the scene type of the scene to be reconstructed from a plurality of preset scene types according to the image type of each frame image includes:

[0109] S401 , obtaining at least one second image of a non-purely repeated texture type in each frame of images.

[0110] S402: Divide each second image into grids to obtain a plurality of grids corresponding to each second image.

[0111] When the scene to be reconstructed is a non-pure repetitive texture scene, images of the non-pure repetitive texture type can be extracted from each frame of images according to the image type corresponding to each frame of images. The images of the non-pure repetitive texture type are called second images. It should be noted that the present application does not limit the number of second images. Take one of the second images as an example. Assume that the length of the second image is r, the width is c, and the preset grid unit length and width are w respectively. Then the length direction of the second image will have r n = r / w grids, there will be c in the width direction n = c / w grids, that is, there are D grids on the second image, D = r n *c n。

[0112] It should be noted that the present application does not limit the number of grids included in the second image, and the grids corresponding to each second image can be finally obtained by referring to the above description.

[0113] S403: Determine the number of matching sub-feature points included in each grid in each second image according to the position information of the matching feature points between each second image and the adjacent frame image in each second image.

[0114] S404: Determine a matching success rate corresponding to each grid in each second image according to the number of matching sub-feature points included in each grid in each second image and the number of matching feature points in the feature information of each second image.

[0115] According to the above description, the feature information of each frame image includes the matching feature point information of itself and the adjacent frame image, and the matching feature point information includes the position information and quantity information of the matching feature points. Continuing to take a second image as an example, the position information of the matching feature points in the second image is determined according to the feature information of the second image, and then the matching feature points located on each grid in the second image can be counted, and the matching feature points located on each grid can be called matching sub-feature points, and the number of matching sub-feature points included in each grid can be counted.

[0116] Take any grid in the second image for illustration, assuming that the number of matching sub-feature points included in the grid is S1, and the number of matching feature points between the second image and the adjacent frame image is P, then the matching success rate corresponding to the grid can be expressed as: (S1 / P)*100%. Similarly, the matching success rate corresponding to each grid in each second image can be obtained.

[0117] S405: Determine the image subtype of each second image according to the matching success rate corresponding to each grid in each second image and a second preset matching success rate.

[0118] Among them, the image subtype includes: semi-repeating texture type and rich texture type. Whether the image subtype of the second image is a semi-repeating texture type or a rich texture type can be achieved through the following embodiments. Continuing to explain with a second image as an example, the image subtype determination process of other second images is similar. According to the above description, each grid included in the second image corresponds to a matching success rate, and the matching success rate corresponding to each grid is compared with the second preset matching success rate. The grid with a matching success rate greater than the second preset matching success rate can be marked as 1, otherwise the grid is marked as 0. If the number of grids marked as 0 corresponding to the second image is greater than or equal to half of the total number of grids included, the image subtype of the second image is determined to be a semi-repeating texture type, otherwise the image subtype of the second image is determined to be a rich texture type.

[0119] It should be noted that the second preset matching success rate can be set according to actual needs, such as 5%, and this application does not limit it.

[0120] S406: Determine a scene type of a scene to be reconstructed from a plurality of preset scene types according to the image subtypes of each second image.

[0121] Finally, the image subtype corresponding to each second image can be determined. As an exemplary example, if the number of images belonging to the semi-repeating texture type in each second image is greater than the number of images belonging to the rich texture type in each second image, then the scene type of the scene to be reconstructed is determined to be a semi-repeating texture scene among multiple preset scene types; otherwise, the scene type of the scene to be reconstructed is determined to be a rich texture scene among multiple preset scene types.

[0122] It can be seen that when the scene to be reconstructed is not a pure repetitive texture scene, by performing grid division and other processing on the second image of the non-pure repetitive texture type, it is determined whether the image subtype of the second image is a semi-repetitive texture type or a rich texture type, and then the scene type of the scene to be reconstructed is determined according to the image subtype of the second image, whether it is a semi-repetitive texture scene among multiple preset scene types, or a rich texture scene. In this way, the scene type of the scene to be reconstructed can be divided more finely. On the premise of improving the accuracy of the scene type of the scene to be reconstructed, the accuracy of the scene reconstruction strategy for three-dimensional modeling of the scene to be reconstructed can be further improved, and the three-dimensional model obtained by using the target scene reconstruction strategy corresponding to the scene type of the scene to be reconstructed can be made to better match the scene to be reconstructed.

[0123] Figure 5 A flowchart of another three-dimensional scene reconstruction method provided in an embodiment of the present application is shown below. Figure 5 As shown, the above-mentioned determining the image subtype of each second image according to the matching success rate corresponding to each grid in each second image and the second preset matching success rate includes:

[0124] S501. Binarize each grid in the current second image according to the matching success rate corresponding to each grid in the current second image and a second preset matching success rate, and determine an initial pixel value of each grid.

[0125] Among them, the current second image is any second image among the second images. Taking one of the second images as an example for explanation, the second image can be called the current second image, and the strategy for binarizing each grid in the current second image is: compare the matching success corresponding to each grid in the current second image with the second preset matching success rate, and binarize each grid according to the comparison result of each grid. Specifically, the initial pixel value of the grid whose matching success rate is greater than the second preset matching success rate can be set to 1, and the initial pixel value of the grid whose matching success rate is less than or equal to the second preset matching success rate can be set to 0, so that the initial pixel value of each grid in the current second image can be obtained.

[0126] S502: Determine the image subtype of the current second image according to the initial pixel values ​​of each grid.

[0127] Optionally, the number of grids in each grid whose initial pixel value is a preset value is determined. If the number of grids is greater than the preset number of grids, the image subtype of the current second image is determined to be a semi-repeating texture type; otherwise, the image subtype of the current second image is determined to be a rich texture type.

[0128] In an exemplary embodiment, assuming that the preset value is 0, the preset number of grids is half of the number of grids included in the current second image. The number of grids with initial pixel values ​​of 0 is counted, and if the number of grids with initial pixel values ​​of 0 corresponding to the current second image is greater than or equal to the preset number of grids, the image subtype of the current second image is determined to be a semi-repeated texture type, otherwise it is determined to be a rich texture type.

[0129] Figure 6 A schematic diagram of another 3D scene reconstruction method provided in an embodiment of the present application. Figure 6 As shown, the above-mentioned determining the image subtype of the current second image according to the initial pixel values ​​of each grid includes:

[0130] S601 , correcting the initial pixel value of each grid according to the initial pixel value of each grid and the preset connected domain to obtain the target pixel value of each grid.

[0131] S602: Determine the image subtype of the current second image according to the target pixel value of each grid.

[0132] Each grid of the current second image can be described in the form of a matrix. Assume that the current second image is a matrix Q with 2 rows and 3 columns, that is, the current second image includes 6 grids. Each initial element in the matrix Q is equivalent to the initial pixel value corresponding to the above-mentioned grid, such as Q(1,1) is 0, Q(1,2) is 1, Q(1,3) is 0, Q(2,1) is 1, Q(2,2) is 1, and Q(2,3) is 0. The preset connected domain can be understood as the area to the left or directly below each element. If the connected domain is composed of the left area and the area directly below, then the connected domain corresponding to Q(1,1) is composed of the areas Q(1,2) and Q(2,1), and the connected domain corresponding to Q(1,2) is composed of the areas Q(1,3) and Q(2,2). Since there is no element to the left of Q(1,3), the corresponding element of Q(1,3) is not revised. For a grid having a left region and a region directly below, the initial pixel value of the grid may be corrected by multiplying the elements of the left region and the region directly below.

[0133] Take the grid corresponding to Q(1,1) as an example. The initial element on Q(1,1) is 0, and the elements on the connected domains Q(1,2) and Q(2,1) corresponding to Q(1,1) are 1 respectively. Then the element of Q(1,1) is corrected to 1, that is, the initial pixel value of the grid corresponding to Q(1,1) is corrected to 1, that is, the target pixel value corresponding to the grid is 1. If the connected domain corresponding to the grid is a region, then the target pixel value of the grid is still the initial pixel value, so the target pixel value of each grid can be obtained. It should be noted that the strategy for correcting the initial pixel value of the grid can be set according to actual needs, and this application does not limit it.

[0134] After the target pixel value of each grid is determined, the number of grids in each grid whose target pixel value is a preset value (such as 0) is determined. If the number of grids is greater than half of the number of grids included in the current second image, the image subtype of the current second image is determined to be a semi-repeating texture type; otherwise, the image subtype of the current second image is determined to be a rich texture type.

[0135] The initial pixel values ​​of each grid are corrected by presetting a connected domain, and the target pixel values ​​of each grid obtained after correction are used to determine the image subtype of the current second image. This can avoid the phenomenon of feature loss in the image caused by the binarization operation, and highlight the clustered characteristics of the successfully matched feature points, which can improve the accuracy of determining the image subtype of the image.

[0136] Optionally, the above-mentioned determining the image subtype of the current second image based on the target pixel value of each grid includes: merging each grid into a plurality of blocks according to a preset number of blocks; determining the pixel value corresponding to each block based on the target pixel value of each grid; and determining the image subtype of the current second image based on the pixel value corresponding to each block.

[0137] Among them, merging each grid into multiple blocks according to the preset number of blocks can be understood as dividing the current second image into the form of a preset matrix, which includes a preset number of blocks, such as 4, that is, the preset matrix is ​​a matrix with 2 rows and 2 columns, and the current second image can be divided into 4 regions. Each region in the current second image can be composed of multiple grids. For example, taking one of the regions of the current second image as an example, the pixel value corresponding to the region is equal to the sum of the target pixel values ​​of the included grids. The number of regions with pixel values ​​of 0 in the current second image is counted. If the number of regions with pixel values ​​of 0 is at least 2, that is, it accounts for at least half of the second image, then the image subtype of the current second image is a semi-repeating texture type. If the number of regions with pixel values ​​of 0 is less than 2, then the image subtype of the current second image is a rich texture type.

[0138] Optionally, the above-mentioned determining the target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed includes: determining the target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed and the correspondence between the scene type and the scene reconstruction strategy.

[0139] In general, the scene type and the scene reconstruction strategy may have the following correspondence: the scene reconstruction strategy corresponding to the pure repetitive texture scene and the semi-repetitive texture scene is the deep learning reconstruction method, and the scene reconstruction strategy corresponding to the rich texture scene is the traditional SIFT feature extraction and reconstruction method. Among them, the deep learning reconstruction method can be understood as using a pre-trained model to construct a three-dimensional model of the scene to be reconstructed, and the traditional SIFT feature extraction and reconstruction method can be understood as extracting the SIFT feature points of the image, generating the data association relationship between multiple frames of images based on the feature points, and then estimating the relative motion between the images, and finally using the Bundle Adjustment to optimize the camera posture and construct the three-dimensional model of the scene to be reconstructed. It should be noted that this embodiment is only an example, and the correspondence between the scene type and the scene reconstruction strategy can be set according to the actual situation, and this application does not limit it.

[0140] If the scene type of the scene to be reconstructed is a pure repetitive texture scene or a semi-repetitive texture scene, the deep learning reconstruction method can be used to construct the three-dimensional model of the scene to be reconstructed. If the scene type of the scene to be reconstructed is a rich texture scene, the traditional SIFT feature extraction reconstruction method as described above can be used to construct the three-dimensional model of the scene to be reconstructed.

[0141] Figure 7 This is a schematic diagram of the structure of a three-dimensional scene reconstruction device provided in an embodiment of the present application. Figure 7 As shown, the device comprises:

[0142] The feature processing module 701 is used to extract and match features of multiple frames of images corresponding to the scene to be reconstructed, and obtain feature information of each frame of image;

[0143] A first determination module 702, configured to determine the scene type of the scene to be reconstructed from a plurality of preset scene types according to feature information of each frame image;

[0144] The second determination module 703 is used to determine a target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed;

[0145] The reconstruction module 704 is used to reconstruct the scene to be reconstructed based on the target scene reconstruction strategy to obtain a three-dimensional model of the scene to be reconstructed.

[0146] Optionally, the first determination module 702 is specifically used to obtain the matching success rate corresponding to each frame image according to the number of feature points included in each frame image and the number of matching feature points between each frame image and the adjacent frame images; determine the image type of each frame image according to the matching success rate corresponding to each frame image, and the image type includes: pure repetitive texture type or non-pure repetitive texture type; determine the scene type of the scene to be reconstructed from multiple preset scene types according to the image type of each frame image.

[0147] Optionally, the first determination module 702 is also specifically used to determine that the image type of the current image is a pure repetitive texture type if the matching success rate corresponding to the current image is less than a first preset matching success rate, otherwise, determine that the image type of the current image is a non-pure repetitive texture type; wherein the current image is any frame image among the frame images.

[0148] Optionally, the first determination module 702 is further specifically used to determine that the scene type of the scene to be reconstructed is a pure repetitive texture scene among the multiple preset scene types if the quantity and position information of the first images belonging to the pure repetitive texture type in each frame image meets a first preset condition, and the quantity and position information includes at least one of the following: the number of first images and whether there is an adjacent relationship between the first images.

[0149] Optionally, when the scene to be reconstructed is not a pure repetitive texture scene, the first determination module 702 is further used to obtain at least one second image of a non-pure repetitive texture type in each frame image; grid-divide each second image to obtain a plurality of grids corresponding to each second image; determine the number of matching sub-feature points included in each grid in each second image based on the position information of the matching feature points between each second image and the adjacent frame image in each second image; determine the matching success rate corresponding to each grid in each second image based on the number of matching sub-feature points included in each grid in each second image and the number of matching feature points in the feature information of each second image; determine the image subtype of each second image based on the matching success rate corresponding to each grid in each second image and the second preset matching success rate, the image subtype including: semi-repetitive texture type and rich texture type; determine the scene type of the scene to be reconstructed from a plurality of preset scene types based on the image subtype of each second image.

[0150] Optionally, the first determination module 702 is further specifically used to binarize each grid in the current second image according to the matching success rate corresponding to each grid in the current second image and the second preset matching success rate, and determine the initial pixel value of each grid, wherein the current second image is any one of the second images; and determine the image subtype of the current second image according to the initial pixel value of each grid.

[0151] Optionally, the first determination module 702 is further specifically used to determine the number of grids in each grid whose initial pixel value is a preset value; if the number of grids is greater than or equal to the preset number of grids, the image subtype of the current second image is determined to be a semi-repeating texture type; otherwise, the image subtype of the current second image is determined to be a rich texture type.

[0152] Optionally, the first determination module 702 is further specifically used to correct the initial pixel value of each grid according to the initial pixel value of each grid and a preset connected domain to obtain a target pixel value of each grid; and determine the image subtype of the current second image according to the target pixel value of each grid.

[0153] Optionally, the first determination module 702 is further specifically used to merge each grid into multiple blocks according to a preset number of blocks; determine the pixel value corresponding to each block according to the target pixel value of each grid; and determine the image subtype of the current second image according to the pixel value corresponding to each block.

[0154] Optionally, the first determination module 702 is further specifically used to determine that the scene type of the scene to be reconstructed is a semi-repeating texture scene among multiple preset scene types if the number of images belonging to the semi-repeating texture type in each second image is greater than the number of images belonging to the rich texture type in each second image; otherwise, determine that the scene type of the scene to be reconstructed is a rich texture scene among multiple preset scene types.

[0155] Optionally, the second determination module 703 is specifically configured to determine a target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed and a correspondence between the scene type and the scene reconstruction strategy.

[0156] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0157] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0158] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 801, a storage medium 802 and a bus 803, the storage medium 802 stores machine-readable instructions executable by the processor 801, when the electronic device is running, the processor 801 communicates with the storage medium 802 via the bus 803, and the processor 801 executes the machine-readable instructions to perform the steps of the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0159] Optionally, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method embodiment are executed.

[0160] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0161] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0163] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0164] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0165] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A three-dimensional scene reconstruction method, characterized in that: The method comprises: Performing feature extraction and feature matching on multiple frames of images corresponding to the scene to be reconstructed to obtain feature information of each frame of image, wherein the feature information includes: the number of feature points included in each frame of image, and the number of matching feature points between each frame of image and adjacent frame of image; Determining the scene type of the scene to be reconstructed from a plurality of preset scene types according to feature information of each frame of image; Determining a target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed; The scene to be reconstructed is reconstructed based on the target scene reconstruction strategy to obtain a three-dimensional model of the scene to be reconstructed.

2. The method according to claim 1, characterized in that The step of determining the scene type of the scene to be reconstructed from a plurality of preset scene types according to the feature information of each frame image includes: According to the number of feature points included in each frame image and the number of matching feature points between each frame image and an adjacent frame image, a matching success rate corresponding to each frame image is obtained; Determine the image type of each frame image according to the matching success rate corresponding to each frame image, wherein the image type includes: a pure repetitive texture type or a non-pure repetitive texture type; According to the image type of each frame of image, the scene type of the scene to be reconstructed is determined from a plurality of preset scene types.

3. The method according to claim 2, characterized in that The step of determining the image type of each frame of image according to the matching success rate corresponding to each frame of image includes: If the matching success rate corresponding to the current image is less than the first preset matching success rate, determining that the image type of the current image is the pure repetitive texture type; otherwise, determining that the image type of the current image is the non-pure repetitive texture type; The current image is any one of the frames.

4. The method according to claim 2, characterized in that: The step of determining the scene type of the scene to be reconstructed from a plurality of preset scene types according to the image type of each frame of image includes: If the quantity and position information of the first images belonging to the pure repetitive texture type in each frame image meets the first preset condition, the scene type of the scene to be reconstructed is determined to be a pure repetitive texture scene among the multiple preset scene types, and the quantity and position information includes at least one of the following: the number of the first images and whether there is an adjacent relationship between the first images.

5. The method according to claim 2, characterized in that: When the scene to be reconstructed is not a pure repetitive texture scene, determining the scene type of the scene to be reconstructed from a plurality of preset scene types according to the image type of each frame image includes: Acquire at least one second image belonging to the non-purely repeated texture type in each frame image; Performing grid division on each of the second images to obtain a plurality of grids corresponding to each of the second images; Determining the number of matching sub-feature points included in each grid in each second image according to position information of matching feature points between each second image and an adjacent frame image in each second image; Determining a matching success rate corresponding to each grid in each second image according to the number of matching sub-feature points included in each grid in each second image and the number of matching feature points in the feature information of each second image; Determining the image subtype of each second image according to the matching success rate corresponding to each grid in each second image and the second preset matching success rate, wherein the image subtype includes: a semi-repeated texture type and a rich texture type; According to the image subtype of each of the second images, the scene type of the scene to be reconstructed is determined from a plurality of preset scene types.

6. The method according to claim 5, characterized in that The determining the image subtype of each second image according to the matching success rate corresponding to each grid in each second image and a second preset matching success rate includes: Binarize each grid in the current second image according to the matching success rate corresponding to each grid in the current second image and the second preset matching success rate, and determine the initial pixel value of each grid, wherein the current second image is any one of the second images; The image subtype of the current second image is determined according to the initial pixel values ​​of each grid.

7. The method according to claim 6, characterized in that The step of determining the image subtype of the current second image according to the initial pixel values ​​of each grid includes: Determine the number of grids in each grid whose initial pixel value is a preset value; If the number of grids is greater than or equal to the preset number of grids, the image subtype of the current second image is determined to be the semi-repeating texture type; otherwise, the image subtype of the current second image is determined to be the rich texture type.

8. The method according to claim 6, characterized in that The step of determining the image subtype of the current second image according to the initial pixel values ​​of each grid includes: According to the initial pixel value of each grid and the preset connected domain, the initial pixel value of each grid is corrected to obtain the target pixel value of each grid; The image subtype of the current second image is determined according to the target pixel value of each grid.

9. The method according to claim 8, characterized in that The step of determining the image subtype of the current second image according to the target pixel values ​​of each grid includes: Merge each grid into multiple blocks according to the preset number of blocks; According to the target pixel value of each grid, determine the pixel value corresponding to each block; The image subtype of the current second image is determined according to the pixel values ​​corresponding to each block.

10. The method according to any one of claims 5 to 9, characterized in that: The step of determining the scene type of the to-be-reconstructed scene from a plurality of preset scene types according to the image subtype of each of the second images includes: If the number of images belonging to the semi-repeating texture type in each of the second images is greater than the number of images belonging to the rich texture type in each of the second images, the scene type of the scene to be reconstructed is determined to be a semi-repeating texture scene among the multiple preset scene types; otherwise, the scene type of the scene to be reconstructed is determined to be a rich texture scene among the multiple preset scene types.

11. The method according to claim 1, characterized in that: The step of determining a target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed includes: According to the scene type of the scene to be reconstructed and the corresponding relationship between the scene type and the scene reconstruction strategy, a target scene reconstruction strategy corresponding to the scene to be reconstructed is determined.

12. A three-dimensional scene reconstruction device, characterized in that: The device comprises: A feature processing module is used to extract and match features of multiple frames of images corresponding to the scene to be reconstructed, and obtain feature information of each frame of image, wherein the feature information includes: the number of feature points included in each frame of image, and the number of matching feature points between each frame of image and adjacent frame of image; A first determination module, used to determine the scene type of the scene to be reconstructed from a plurality of preset scene types according to feature information of each frame image; A second determination module, used to determine a target scene reconstruction strategy corresponding to the scene to be reconstructed according to the scene type of the scene to be reconstructed; A reconstruction module is used to reconstruct the scene to be reconstructed based on the target scene reconstruction strategy to obtain a three-dimensional model of the scene to be reconstructed.

13. An electronic device, characterized in that: include: A controller, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the controller. When the electronic device is running, the controller communicates with the storage medium via the bus, and the controller executes the machine-readable instructions to perform the steps of the three-dimensional scene reconstruction method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the controller, the steps of the three-dimensional scene reconstruction method according to any one of claims 1 to 11 are executed.

Images