Three-dimensional reconstruction method, device, equipment, medium and program product

Abstract

The present disclosure provides a three-dimensional reconstruction method, device, equipment, medium and program product, which relates to the field of computer technology, and in particular to the field of image processing. The specific implementation scheme is: divide the image to be matched of the three-dimensional object into image rings to obtain at least two image rings; determine the intra-ring matching results of the image rings according to at least two images to be matched in the image rings; determine the inter-ring matching results between different image rings, and perform three-dimensional reconstruction of the three-dimensional object according to the intra-ring matching results of the image rings and the inter-ring matching results between different image rings, and the inter-ring matching results are used to characterize the overlapping relationship of the three-dimensional objects in different image rings. The scheme of the embodiment of the present disclosure improves the accuracy of three-dimensional reconstruction.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of image processing, and specifically to a three-dimensional reconstruction method, device, equipment, medium and program product. Background Art

[0002] 3D panoramic photography refers to shooting an object from different angles and reconstructing it in three dimensions based on multiple images to generate a picture with a 3D effect. The viewer can use the mouse to rotate the object in the image to see the whole picture of the object.

[0003] Image matching is required in the 3D reconstruction process, and feature-based image matching is a commonly used image matching method. Feature-based image matching methods require feature point extraction and feature point matching of images collected at different angles. Improving the accuracy of feature point matching is very important for 3D reconstruction. Summary of the invention

[0004] The present disclosure provides a three-dimensional reconstruction method, apparatus, device, medium and program product.

[0005] According to one aspect of the present disclosure, a three-dimensional reconstruction method is provided, comprising:

[0006] Dividing the image to be matched of the three-dimensional object into image rings to obtain at least two image rings;

[0007] Determine an intra-ring matching result of the image ring according to at least two images to be matched in the image ring;

[0008] The inter-ring matching results between different image rings are determined, and the three-dimensional object is reconstructed based on the intra-ring matching results of the image rings and the inter-ring matching results between different image rings; the inter-ring matching results are used to characterize the overlapping relationship of the three-dimensional objects in different image rings.

[0009] According to another aspect of the present disclosure, there is provided a three-dimensional reconstruction device, comprising:

[0010] An image ring division module is used to divide the image to be matched of the three-dimensional object into image rings to obtain at least two image rings;

[0011] An intra-ring matching result determination module, used to determine an intra-ring matching result of the image ring according to at least two images to be matched in the image ring;

[0012] The three-dimensional reconstruction module is used to determine the inter-ring matching results between different image rings, and to perform three-dimensional reconstruction of three-dimensional objects based on the intra-ring matching results of the image rings and the inter-ring matching results between different image rings; the inter-ring matching results are used to characterize the overlapping relationship of three-dimensional objects in different image rings.

[0013] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to at least one processor; wherein,

[0016] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the three-dimensional reconstruction method of any embodiment of the present disclosure.

[0017] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the three-dimensional reconstruction method of any embodiment of the present disclosure.

[0018] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the three-dimensional reconstruction method of any embodiment of the present disclosure is implemented.

[0019] The disclosed embodiments can improve the accuracy of three-dimensional reconstruction.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0022] Figure 1a is a schematic diagram of a three-dimensional reconstruction method provided according to an embodiment of the present disclosure;

[0023] Figure 1b is a schematic diagram of in-ring image matching provided according to an embodiment of the present disclosure;

[0024] Figure 1c is another schematic diagram of in-ring image matching provided according to an embodiment of the present disclosure;

[0025] Figure 2a is a schematic diagram of a three-dimensional reconstruction method provided according to an embodiment of the present disclosure;

[0026] Figure 2b is a schematic diagram of inter-ring image matching provided according to an embodiment of the present disclosure;

[0027] Figure 3 is a schematic diagram of a three-dimensional reconstruction method provided according to an embodiment of the present disclosure;

[0028] Figure 4 is a schematic diagram of a three-dimensional reconstruction device provided according to an embodiment of the present disclosure;

[0029] Figure 5 It is a block diagram of an electronic device used to implement the 3D reconstruction method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] Figure 1a This is a flowchart of a 3D reconstruction method according to an embodiment of the present disclosure. This embodiment can be applied to the case where 3D reconstruction is performed through the intra-ring matching results and inter-ring matching results of the image to be matched. The method of this embodiment can be executed by a 3D reconstruction device, which can be implemented in software and / or hardware, and is specifically configured in an electronic device with certain data computing capabilities. The electronic device can be a client device or a server device, and the client device can be a mobile phone, a tablet computer, a vehicle terminal, a desktop computer, etc.

[0032] S110 , dividing the image to be matched of the three-dimensional object into image rings to obtain at least two image rings.

[0033] The 3D object is an object that needs to be reconstructed in 3D, and the image to be matched of the 3D object is an image obtained by photographing the 3D object, which is used as the material for 3D reconstruction. Specifically, the image to be matched is an image of the object obtained by continuously photographing the rotating object.

[0034] In a specific example, a three-dimensional object is placed in the center of a turntable, and the turntable is controlled to rotate. Every time the turntable rotates ten degrees, a camera captures an image. When the turntable rotates one circle, a group (36 images) of images to be matched are obtained as the material for three-dimensional reconstruction. Of course, after obtaining a group of images to be matched, the posture of the object on the turntable is changed, and the next group of images is taken. Finally, the multiple groups of images to be matched are used as the material for three-dimensional reconstruction. Each group of images to be matched can form an image ring according to the shooting order.

[0035] It is worth noting that when taking each set of pictures, the obtained images are named according to the set configuration information, so that the image ring to which the image to be matched belongs and the position of the image to be matched in the current image ring can be judged according to the name later. Specifically, the image to be matched is named xx_yy.jpg, where xx is used to indicate the image ring to which the current image to be matched belongs, and yy is used to indicate the position of the current image to be matched in the image ring.

[0036] In the disclosed embodiment, after acquiring multiple images to be matched of a three-dimensional object, the images to be matched are divided into at least two image rings. Each image ring includes multiple images to be matched obtained by photographing the three-dimensional object in a circle, and the three-dimensional object is placed in a different posture during the photographing of each image ring. By dividing the images to be matched into image rings, the matching relationship of the same posture of the three-dimensional object at different shooting angles can be obtained in one image ring. Moreover, the matching relationship between different postures of the three-dimensional object can be obtained in different image rings.

[0037] Specifically, the image to be matched is divided into image rings according to the shooting time of the image to be matched, and the multiple images to be matched contained in each image ring are arranged in order according to the shooting time. The image ring to which the image to be matched belongs and the position of the image to be matched in the image ring can also be obtained by parsing the name of the image to be matched.

[0038] In a specific example, the image to be matched is obtained by shooting once every 2 seconds, and a circle of ring objects is collected every 70 seconds, obtaining 36 images to be matched; further, the image is stopped for 30 seconds, the posture of the three-dimensional object is transformed, and the above-mentioned ring object collection process is continued. In this case, the images to be matched with an interval of 2 seconds can be divided into an image ring according to the collection time of the images to be matched. Furthermore, in each image ring, the images to be matched are arranged in sequence according to the collection time of the images to be matched to obtain an image ring.

[0039] In another specific example, during the shooting process, the obtained image is named according to the set configuration information. Specifically, the image to be matched is named xx_yy.jpg, where xx is used to indicate the image ring to which the current image to be matched belongs, and yy is used to indicate the position of the current image to be matched in the image ring. When the name of the current image to be matched is 02_35.jpg, by parsing the name, it can be obtained that the current image to be matched belongs to the second image ring, and the arrangement order in the second image ring is the 36th (in the shooting of a ring object, the image sequence number is from 00 to 35).

[0040] S120: Determine an intra-ring matching result of the image ring according to at least two to-be-matched images in the image ring.

[0041] The intra-loop matching result is used to characterize the overlapping relationship of three-dimensional objects in the same image loop. The intra-loop matching result is the result obtained by performing image matching on the images to be matched in the same image loop. Exemplarily, exhaustive matching is performed on the images to be matched in the same image loop to obtain the intra-loop matching result. Another exemplary method is to perform sequential matching on the images to be matched in the same image loop to obtain the intra-loop matching result.

[0042] In the disclosed embodiment, after the image to be matched is divided into at least two image rings, for each image ring, the intra-ring matching result of the image ring can be determined based on at least two images to be matched contained in the image ring. Specifically, the images to be matched in the same image ring are matched in pairs to obtain the number of matching feature points between each pair of images to be matched (composed of two images to be matched) and the coordinates of the matching feature points in the images to be matched. Finally, the set of matching results of each pair of images to be matched is used as the intra-ring matching result of the image ring.

[0043] Specifically, the in-loop matching result can also be obtained by sequential matching. Specifically, for each image loop, each image to be matched in the image loop is sequentially used as a target image to be matched, and one or more images to be matched are obtained on both sides of the target image to be matched as adjacent images according to the arrangement order of the images to be matched in the image loop. Further, the target image to be matched is image matched with each adjacent image to obtain a matching result for the target image to be matched. Finally, the matching results of all images in the image loop are used as the in-loop matching result.

[0044] In a specific example, Figure 1b As shown, an image ring includes 11 images to be matched. The image to be matched with sequence number 0 can be used as the target image to be matched, and the target image to be matched can be matched with other images to be matched in the image ring to obtain a matching result. Furthermore, the image to be matched with sequence number 1 can be used as the target image to be matched, and the target image to be matched can be matched with other images to be matched in the image ring. This process can be repeated until the matching process of all images to be matched in the image ring is completed, and the matching results of all images to be matched can be used as the matching results within the ring.

[0045] In another specific example, Figure 1cAs shown, an image ring includes 11 images to be matched. First, the image to be matched with sequence number 0 can be used as the target image to be matched, and two images to be matched are determined on both sides of the target image to be matched as adjacent images. Further, the target image to be matched is matched with the four adjacent images to be matched, respectively, to obtain a matching result. Further, the image to be matched with sequence number 1 is used as the target image to be matched, and the above image matching operation is repeated. And so on, until the matching process of all images to be matched in the image ring is completed, and the matching results of all images to be matched are obtained as the matching results within the ring.

[0046] S130, determining the inter-ring matching results between different image rings, and performing three-dimensional reconstruction on the three-dimensional object according to the intra-ring matching results of the image rings and the inter-ring matching results between different image rings, wherein the inter-ring matching results are used to characterize the overlapping relationship of the three-dimensional objects in different image rings.

[0047] In the scenario where multiple image rings are obtained, in order to improve the accuracy of 3D reconstruction, in addition to obtaining the matching results of the images to be matched within each image ring, it is also necessary to further determine the inter-ring matching results between the image rings. Among them, the inter-ring matching results are used to characterize the overlapping relationship of 3D objects in different image rings. For example, the 3D object is a cup, including a cup body and a handle. The image to be matched in the first image ring is obtained by photographing the cup placed vertically around the object, and the image to be matched in the second image ring is obtained by photographing the cup placed horizontally around the object. The inter-ring matching results can characterize the matching relationship between the handle in the first image in the first image ring and the handle in the second image in the second image ring at the same position.

[0048] In the disclosed embodiment, multiple image rings can be combined in pairs to obtain multiple image ring pairs, that is, the image ring pair includes a first image ring and a second image ring. The image to be matched included in the first image ring and the image to be matched included in the second image ring in the image ring pair are exhaustively matched to obtain an inter-ring matching result between the image rings. Specifically, each image to be matched in the first image ring and each image to be matched in the second image ring are compared in pairs to obtain an inter-ring matching result.

[0049] After completing the above exhaustive matching process, it is also possible to continue screening in the exhaustive matching results, and use the screened matching results as the inter-ring matching results. Specifically, a pair of images to be matched with the largest number of matching feature points can be found in the exhaustive matching results, and the images to be matched include a first image belonging to the first image ring and a second image belonging to the second image ring. The first image and the second image are used as bridge points between the two image rings. Finally, the matching result between the two bridge points is used as the inter-ring matching result. It is also possible to determine at least one adjacent image on both sides of the two bridge points to form two image sets centered on the bridge point. Finally, the matching result between the images to be matched in the two image sets is used as the inter-ring matching result.

[0050] After obtaining the matching results of the images to be matched inside and between the image rings of the three-dimensional object, the intra-ring matching results of the image ring and the inter-ring matching results of different image rings can be used as inputs for three-dimensional reconstruction.

[0051] In a specific example, for the images to be matched of a three-dimensional object, three image rings are obtained, namely image rings A, B and C, each of which contains 36 images to be matched. First, the three image rings are divided into three image ring pairs, namely image rings A and B, image rings A and C, and image rings B and C. Furthermore, an exhaustive match is performed for each image ring pair. Taking image rings A and B as an example, each image to be matched in image ring A is matched with each image to be matched in image ring B, and 36*36 groups of matching results are obtained. Similarly, other image ring pairs are also matched according to the above process, and finally the set of matching results of each image ring pair can be directly used as the inter-ring matching result.

[0052] Considering that some three-dimensional objects with the same texture on different surfaces are used to avoid mismatching. After obtaining the above 36*36 groups of matching results, a group of matching results with the largest number of matching feature points can be further searched in these matching results. For example, the number of matching feature points between the image 6 to be matched in image ring A and the image 6 to be matched in image ring B is the largest. At this time, the image 6 to be matched in image ring A can be used as the first bridge image, and the image 6 to be matched in image ring B can be used as the second bridge image. Finally, the matching result between the first bridge image and the second bridge image is used as the inter-ring matching result. It is also possible to obtain three adjacent images on both sides of the first bridge point in image ring A. Specifically, the adjacent images to the image 6 to be matched in image ring A are images 3, 4, 5, 7, 8 and 9 to be matched, and images 3 to 8 to be matched in image ring A are used as the first image set. Similarly, in image ring B, three images to be matched are obtained on both sides of the second bridge point, and images 3 to 8 to be matched in image ring B are used as the second image set. Finally, the matching result between each to-be-matched image in the first image set and each to-be-matched image in the second image set is taken as the inter-ring matching result.

[0053] The technical solution of the disclosed embodiment is to divide the image to be matched of the three-dimensional object into image rings to obtain at least two image rings, and determine the intra-ring matching results of the image rings based on at least two images to be matched in the image rings, and further determine the inter-ring matching results between different image rings. The inter-ring matching results are used to characterize the overlapping relationship of the three-dimensional objects in different image rings, and finally perform three-dimensional reconstruction of the three-dimensional object based on the intra-ring matching results of the image rings and the inter-ring matching results between different image rings. By dividing the image to be matched into image rings and performing three-dimensional reconstruction based on the intra-image ring matching results and the inter-ring matching results, the accuracy of three-dimensional reconstruction is improved.

[0054] Figure 2a is a schematic diagram of a 3D reconstruction method in an embodiment of the present disclosure, which further refines the above embodiment and provides specific steps for determining the inter-ring matching results between different image rings. Figure 2a A three-dimensional reconstruction method provided by an embodiment of the present disclosure is described, including the following:

[0055] S210 , dividing the image to be matched of the three-dimensional object into image rings to obtain at least two image rings.

[0056] S220: Determine an intra-ring matching result of the image ring according to at least two to-be-matched images in the image ring.

[0057] S230: Determine an exhaustive matching result between the first image to be matched in the first image ring and the second image to be matched in the second image ring.

[0058] In an embodiment of the present disclosure, a matching method between different image rings is provided: first, a first image to be matched in a first image ring and a second image to be matched in a second image ring need to be matched in pairs to obtain an exhaustive matching result between the first image ring and the second image ring.

[0059] Exemplarily, the first image ring includes 36 first images to be matched, and the second image ring includes 18 second images to be matched. Each first image to be matched in the first image ring can be matched with each second image to be matched in the second image ring to obtain an exhaustive matching result, which includes matching results of 36*18 groups of images to be matched.

[0060] S240 . Select a first bridge image from the first image ring and a second bridge image from the second image ring according to the exhaustive matching result.

[0061] It is worth noting that for some three-dimensional objects with repeated textures on different surfaces, the above-mentioned exhaustive matching method may result in incorrect matching. For example, the three-dimensional object is a cuboid with the same texture on the front and bottom surfaces. The first image ring for the three-dimensional object contains a first image to be matched corresponding to the front surface, and the second image ring contains a second image to be matched corresponding to the bottom surface. At this time, if the exhaustive matching method is used, some feature points in the first image to be matched and the second image to be matched corresponding to different parts of the three-dimensional object may be successfully matched, resulting in image matching errors.

[0062] In view of the above-mentioned repeated texture mismatching, the first bridge image can be selected in the first image ring and the second bridge image can be selected in the second image ring according to the exhaustive matching results of the first image ring and the second image ring. Specifically, the first bridge image can be determined in the first image ring and the second bridge image can be determined in the second image ring according to the number of matching feature points of each image to be matched in the exhaustive matching results. Exemplarily, in the exhaustive matching results, a pair of images to be matched with the largest number of matching feature points is selected. And the image belonging to the first image ring among the images to be matched is used as the first bridge image, and the image belonging to the second image ring is used as the second bridge image.

[0063] Optionally, based on the exhaustive matching result, a first bridging image is selected from the first image ring, and a second bridging image is selected from the second image ring, including: obtaining a pair of images to be matched with the most matching feature points in the exhaustive matching result, and using the first image to be matched in the pair of images to be matched as the first bridging image, and using the second image to be matched in the pair of images to be matched as the second bridging image.

[0064] In this optional embodiment, a specific method is provided for selecting a first bridge image from the first image ring and a second bridge image from the second image ring according to the exhaustive matching result: obtaining a pair of images to be matched with the most matching feature points in the exhaustive matching result, and using the first image to be matched in the pair of images to be matched as the first bridge image, and using the second image to be matched in the pair of images to be matched as the second bridge image. The first bridge image and the second bridge image are used to establish an association relationship between the first image ring and the second image ring, which can improve the robustness of image matching in a scene with multiple image rings.

[0065] For example, Figure 2b As shown in the figure, the number of matching feature points between the image to be matched with sequence number 0 in the first image ring and the image to be matched with sequence number 0 in the second image ring is 300, which is the pair of images to be matched with the largest number of matching feature points in the exhaustive matching result. At this time, the image to be matched with sequence number 0 in the first image ring can be used as the first bridge image, and the image to be matched with sequence number 0 in the second image ring can be used as the second bridge image.

[0066] S250: Determine an inter-ring matching result between the first image ring and the second image ring according to the first bridge image and the second bridge image.

[0067] In the embodiment of the present disclosure, after determining the bridge image, the inter-ring matching result between the first image ring and the second image ring is determined based on the first bridge image and the second bridge image. Specifically, the matching result between the first bridge image and the second bridge image can be directly used as the inter-ring matching result. In addition, two adjacent images adjacent to the first bridge image on the left and right can be determined in the first image ring, and the adjacent images and the first bridge image constitute the first image set. Similarly, in the second image ring, two adjacent images adjacent to the second bridge image on the left and right can be determined, and the adjacent images and the second bridge image constitute the second image set. Finally, in the exhaustive matching result, the matching result between each image to be matched in the first image set and each image to be matched in the second image set is obtained as the inter-ring matching result. Among them, the first image set and the second image set both include 3 images to be matched, and the final inter-ring matching result includes the matching results of 9 pairs of images to be matched. By obtaining the first bridge image and the second bridge image, the association relationship between different image rings can be established to improve the robustness of image matching.

[0068] For example, Figure 2bAs shown, the first bridge image of the first image ring is the image to be matched with sequence number 0, and in the second image ring, the second bridge image is also the image to be matched with sequence number 0. In the first image ring, it is determined that the two images adjacent to the first bridge image are images to be matched 1 and 10. In the second image ring, it is determined that the two images adjacent to the second bridge image are images to be matched 1 and 10. At this time, the images to be matched 1, 0 and 10 in the first image ring and the images to be matched 1, 0 and 10 in the second image ring can be matched in pairs as the final inter-ring matching result.

[0069] Optionally, an inter-ring matching result between the first image ring and the second image ring is determined based on the first bridging image and the second bridging image, including: in the first image ring, a first image set is formed by the first bridging image and at least two first images to be matched adjacent to the first bridging image; in the second image ring, a second image set is formed by the second bridging image and at least two second images to be matched adjacent to the second bridging image; and an exhaustive matching result between each first image to be matched in the first image set and each second image to be matched in the second image set is used as the inter-ring matching result.

[0070] In this optional embodiment, a specific method is provided for determining the inter-ring matching result between the first image ring and the second image ring based on the first bridge image and the second bridge image: first, in the first image ring, a first image set is formed by a first bridge point and at least two first images to be matched adjacent to the first bridge point. Further, a second image set is formed by a second bridge point and at least two second images to be matched adjacent to the second bridge point. Finally, the exhaustive matching result of each first image to be matched in the first image set and each second image to be matched in the second image set is used as the inter-ring matching result. Only retaining the matching results between the first image set and the second image set can avoid matching errors caused by the same surface texture of different three-dimensional objects, improve matching accuracy, and thus optimize the three-dimensional reconstruction effect.

[0071] In a specific example, the first image ring and the second image ring each include 36 images to be matched, and the images to be matched are arranged in the order of 0 to 35 to form an image ring connected end to end. The first bridge point is the image 10 to be matched in the first image ring, and the second bridge point is the image 6 to be matched in the second image ring. Further, in the first image ring, three images to be matched are determined on both sides of the first bridge point, that is, the images 7, 8 and 9 to be matched adjacent to the image 10 to be matched, and the images 11, 12 and 13 to be matched are determined on both sides of the image 10 to be matched, and the first image set is formed by the first bridge point and the adjacent image points to be matched, that is, the first image set includes the images 7 to 13 to be matched in the first image ring. Furthermore, in the second image ring, three images to be matched are determined on both sides of the second bridge point, that is, images 3, 4 and 5 to be matched adjacent to the image 6 to be matched, and images 7, 8 and 9 to be matched are determined on both sides of the image 6 to be matched, and the second bridge point and the adjacent image points to be matched constitute the second image set, that is, the second image set includes images 3 to 9 to be matched in the second image ring. Finally, in the exhaustive matching results, the matching results between the images to be matched in the first image set and the second image set are selected as the final inter-ring matching results. The above-mentioned first image set and the second image set both contain 7 images to be matched, and the final inter-ring matching results include 7*7 groups of matching results.

[0072] S260 , performing three-dimensional reconstruction of the three-dimensional object according to the intra-ring matching results of the image rings and the inter-ring matching results between different image rings.

[0073] The technical solution of the embodiment of the present disclosure first determines the exhaustive matching result between the first image to be matched in the first image ring and the second image to be matched in the second image ring, and then selects the first bridging image from the first image ring and the second bridging image from the second image ring based on the exhaustive matching result, and determines the inter-ring matching result between the first image ring and the second image ring based on the first bridging image and the second bridging image, so as to obtain the correlation relationship between the multiple image rings, improve the robustness of image matching, and improve the three-dimensional reconstruction accuracy at the same time.

[0074] Figure 3 This is a schematic diagram of a 3D reconstruction in an embodiment of the present disclosure, which is further refined on the basis of the above embodiment, and provides specific steps of dividing the image to be matched of the 3D object into image rings to obtain at least two image rings, and determining the inner ring matching result of the image ring according to at least two images to be matched in the image ring. Figure 3 A three-dimensional reconstruction method provided by an embodiment of the present disclosure is described, including the following:

[0075] S310: Determine the image ring to which the image to be matched belongs according to the first two characters in the name of the image to be matched.

[0076] When the image acquisition device acquires the image to be matched, it will name the image to be matched according to the set configuration information, and the name can represent the scene of the image to be matched. Specifically, the scene to be matched can include the number of rounds of the object shooting process in which the current image to be matched was shot, and the number of shots in the round of shooting in which the current image to be matched was taken.

[0077] According to the above naming principle, the image to be matched can be named xx_yy.jpg, where xx is used to indicate the image ring (i.e., shooting round) to which the current image to be matched belongs, and yy is used to indicate the position of the current image to be matched in the image ring (i.e., shooting time in the round).

[0078] Based on the above naming principle, the name of the image to be matched is parsed. First, the first two characters of the name of the image to be matched are read, and the image ring to which the image to be matched belongs is determined based on the first two characters. For example, if the name of the image to be matched is 01_12.jpg, it is determined that the current image to be matched belongs to the first image ring. Similarly, all images to be matched are divided into their corresponding image rings.

[0079] S320: Determine the position of the image to be matched within the image ring according to the last two characters of the name of the image to be matched.

[0080] Similarly, based on the above naming principle, the name of the image to be matched is further parsed. The last two characters of the name of the image to be matched are read, and the position of the image to be matched in the image ring is determined based on the last two characters. For example, if the name of the image to be matched is 01_12.jpg, it is determined that the current image to be matched belongs to the 13th image in the first image ring. The images in each image ring are arranged in a ring structure in order starting from 00.

[0081] S330 . Construct at least two image rings according to the image ring to which the image to be matched belongs and the position of the image to be matched in the image ring.

[0082] After obtaining the image ring to which the image to be matched belongs and the position of the image to be matched in the image ring, firstly, all the images to be matched are divided into rings according to the image ring to which the image to be matched belongs. Furthermore, according to the position of the images to be matched in the same image ring, the images to be matched are arranged in sequence to form image rings connected end to end. The image rings can be quickly divided according to the names of the images to be matched, thereby improving the efficiency of image ring division. In addition, the image ring division process does not require information such as image acquisition time, and does not require additional storage of other information other than the images to be matched, thus saving storage space.

[0083] S340 , determining at least one neighboring image adjacent to the image to be matched on both sides of each image to be matched according to the arrangement order of the images to be matched in the image ring.

[0084] In order to perform three-dimensional reconstruction, the images to be matched in each image ring are first matched. For each image to be matched in the image ring, at least one adjacent image to be matched can be determined on both sides of the image to be matched according to the arrangement order of the images to be matched in the image ring.

[0085] Exemplarily, the current image ring includes 36 images to be matched arranged in sequence from 00 to 35. Taking the image to be matched 00 as an example, two adjacent images, namely 01, 02, and 34, 35, can be determined on both sides of the image to be matched 00.

[0086] S350: Match the image to be matched with the adjacent image to obtain an intra-ring matching result of the image ring.

[0087] After the adjacent images are determined, the image to be matched is matched with the adjacent images, and the matching result of the image to be matched, and finally the set of matching results of all the images to be matched, can be used as the in-loop matching result.

[0088] Still taking the image ring containing 36 images to be matched as an example, after determining that the neighboring images of the image 00 to be matched are 01, 02, 34 and 35, the image 00 to be matched is matched with these four neighboring images respectively to obtain the matching result for the image 00 to be matched. Similarly, the matching results of all the images to be matched in the image ring can be obtained, and finally the set of the matching results of all the images to be matched is taken as the matching result within the ring. By sequentially matching the images to be matched in the image ring, the matching efficiency can be improved compared with the exhaustive matching method. In addition, the image ring is a head-to-tail structure. During the matching process, the images to be matched at the beginning and end of the ring will be matched to form a closed loop, which can reduce the cumulative error of subsequent incremental reconstruction.

[0089] S360: Determine inter-ring matching results between different image rings, where the inter-ring matching results are used to characterize overlapping relationships between three-dimensional objects in different image rings.

[0090] S370 , reconstructing the three-dimensional object in three dimensions according to the intra-ring matching results of the image rings and the inter-ring matching results between different image rings.

[0091] The technical solution of the disclosed embodiment is to divide the image to be matched of the three-dimensional object into image rings to obtain at least two image rings, and sequentially match the images to be matched in the image rings to determine the intra-ring matching results of the image rings, and further determine the inter-ring matching results between different image rings. The inter-ring matching results are used to characterize the overlapping relationship of the three-dimensional objects in different image rings. Finally, based on the intra-ring matching results of the image rings and the inter-ring matching results between different image rings, the three-dimensional object is reconstructed. By sequentially matching the images within the rings and matching the ring heads and ring tails, the efficiency of image matching is improved, and the accuracy of three-dimensional reconstruction is also improved.

[0092] According to an embodiment of the present disclosure, Figure 4 3D reconstruction apparatus in the embodiment of the present disclosure is applicable to the case where 3D reconstruction is performed by using the intra-ring matching results and inter-ring matching results of the image to be matched. The apparatus is implemented by software and / or hardware and is specifically configured in an electronic device with certain data computing capabilities.

[0093] like Figure 4 The three-dimensional reconstruction device 400 shown in the figure comprises: an image ring division module 410, an intra-ring matching result determination module 420 and a three-dimensional reconstruction module 430; wherein,

[0094] An image ring division module 410 is used to divide the image to be matched of the three-dimensional object into image rings to obtain at least two image rings;

[0095] An intra-ring matching result determination module 420 is used to determine an intra-ring matching result of the image ring according to at least two images to be matched in the image ring;

[0096] The three-dimensional reconstruction module 430 is used to determine the inter-ring matching results between different image rings, and to perform three-dimensional reconstruction of the three-dimensional object based on the intra-ring matching results of the image rings and the inter-ring matching results between different image rings. The inter-ring matching results are used to characterize the overlapping relationship of the three-dimensional objects in different image rings.

[0097] The technical solution of the disclosed embodiment is to divide the image to be matched of the three-dimensional object into image rings to obtain at least two image rings, and determine the intra-ring matching results of the image rings based on at least two images to be matched in the image rings, and further determine the inter-ring matching results between different image rings. The inter-ring matching results are used to characterize the overlapping relationship of the three-dimensional objects in different image rings, and finally perform three-dimensional reconstruction of the three-dimensional object based on the intra-ring matching results of the image rings and the inter-ring matching results between different image rings. By dividing the image to be matched into image rings and performing three-dimensional reconstruction based on the intra-image ring matching results and the inter-ring matching results, the accuracy of three-dimensional reconstruction is improved.

[0098] Furthermore, the 3D reconstruction module 430 includes:

[0099] An exhaustive matching result determining unit, used to determine an exhaustive matching result between a first image to be matched in the first image ring and a second image to be matched in the second image ring;

[0100] a bridging image determining unit, configured to select a first bridging image from the first image ring and a second bridging image from the second image ring according to the exhaustive matching result;

[0101] The inter-ring matching result determining unit is used to determine the inter-ring matching result between the first image ring and the second image ring according to the first bridge image and the second bridge image.

[0102] Further, the bridge image determination unit comprises:

[0103] The bridging image determination subunit is used to obtain the to-be-matched image pair with the most matching feature points in the exhaustive matching result, and use the first to-be-matched image in the to-be-matched image pair as the first bridging image, and use the second to-be-matched image in the to-be-matched image pair as the second bridging image.

[0104] Furthermore, the inter-ring matching result determination unit includes:

[0105] a first image set determining subunit, configured to form a first image set in the first image ring from the first bridging image and at least two first to-be-matched images adjacent to the first bridging image;

[0106] a second image set determining subunit, configured to form a second image set in the second image ring from the second bridging image and at least two second images to be matched adjacent to the second bridging image;

[0107] The inter-ring matching result determination subunit is used to use the exhaustive matching result of each first image to be matched in the first image set and each second image to be matched in the second image set as the inter-ring matching result.

[0108] Furthermore, the image ring division module 410 includes:

[0109] An image ring determination unit, used for determining the image ring to which the image to be matched belongs according to the first two characters in the name of the image to be matched;

[0110] The ring position determination unit is used to determine the position of the image to be matched in the image ring according to the last two characters of the name of the image to be matched;

[0111] The image ring division unit is used to form at least two image rings according to the image ring to which the image to be matched belongs and the position of the image to be matched in the image ring.

[0112] Furthermore, the in-loop matching result determination module 420 includes:

[0113] An adjacent image determining unit, configured to determine at least one adjacent image adjacent to the image to be matched on both sides of each image to be matched according to the arrangement order of the images to be matched in the image ring;

[0114] The in-ring matching result determining unit is used to match the image to be matched with the adjacent image to obtain an in-ring matching result of the image ring.

[0115] The three-dimensional reconstruction device provided in the embodiments of the present disclosure can execute the three-dimensional reconstruction method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0118] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0119] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0120] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0121] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as a three-dimensional reconstruction method. For example, in some embodiments, the three-dimensional reconstruction method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the three-dimensional reconstruction method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the three-dimensional reconstruction method in any other appropriate manner (e.g., by means of firmware).

[0122] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0124] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0127] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0128] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0129] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A three-dimensional reconstruction method, comprising: Dividing the image to be matched of the three-dimensional object into image rings to obtain at least two image rings; Determine an intra-ring matching result of the image ring according to at least two images to be matched in the image ring; Determine an exhaustive matching result between a first image to be matched in the first image ring and a second image to be matched in the second image ring; According to the exhaustive matching result, selecting a first bridge image from the first image ring, and selecting a second bridge image from the second image ring; In the first image ring, the first bridge image and at least two first to-be-matched images adjacent to the first bridge image form a first image set; In the second image ring, the second bridge image and at least two second images to be matched adjacent to the second bridge image form a second image set; Using an exhaustive matching result of each first image to be matched in the first image set and each second image to be matched in the second image set as an inter-ring matching result; The three-dimensional object is three-dimensionally reconstructed according to the intra-ring matching results of the image ring and the inter-ring matching results between different image rings; the inter-ring matching results are used to characterize the overlapping relationship of the three-dimensional objects in different image rings.

2. The method according to claim 1, wherein: According to the exhaustive matching result, selecting a first bridge image from the first image ring and selecting a second bridge image from the second image ring comprises: The pair of images to be matched with the most matching feature points in the exhaustive matching result is obtained, and the first image to be matched in the pair of images to be matched is used as the first bridging image, and the second image to be matched in the pair of images to be matched is used as the second bridging image.

3. The method according to claim 1, wherein: The image to be matched of the three-dimensional object is divided into image rings to obtain at least two image rings, including: Determine the image ring to which the image to be matched belongs according to the first two characters in the name of the image to be matched; Determine the position of the image to be matched within the image ring according to the last two characters of the name of the image to be matched; At least two image rings are formed according to the image ring to which the image to be matched belongs and the position of the image to be matched in the image ring.

4. The method according to claim 1, wherein: Determining an intra-ring matching result of the image ring according to at least two to-be-matched images in the image ring includes: According to the arrangement order of the images to be matched in the image ring, at least one adjacent image adjacent to the image to be matched is determined on both sides of each image to be matched; The image to be matched is matched with the adjacent image to obtain an intra-ring matching result of the image ring.

5. A three-dimensional reconstruction device, comprising: An image ring division module is used to divide the image to be matched of the three-dimensional object into image rings to obtain at least two image rings; An intra-ring matching result determination module, used to determine an intra-ring matching result of the image ring according to at least two images to be matched in the image ring; A three-dimensional reconstruction module, comprising: an exhaustive matching result determination unit, a bridge image determination unit and an inter-ring matching result determination unit; The exhaustive matching result determining unit is used to determine an exhaustive matching result between a first image to be matched in the first image ring and a second image to be matched in the second image ring; The bridge image determination unit is configured to select a first bridge image from the first image ring and a second bridge image from the second image ring according to the exhaustive matching result; The inter-ring matching result determination unit includes: a first image set determination subunit, a second image set determination subunit and an inter-ring matching result determination subunit; The first image set determination subunit is configured to form a first image set in the first image ring from the first bridging image and at least two first to-be-matched images adjacent to the first bridging image; The second image set determination subunit is configured to form a second image set in the second image ring from the second bridging image and at least two second images to be matched adjacent to the second bridging image; The inter-ring matching result determination subunit is used to use the exhaustive matching results of each first image to be matched in the first image set and each second image to be matched in the second image set as the inter-ring matching results; perform three-dimensional reconstruction of the three-dimensional object based on the intra-ring matching results of the image ring and the inter-ring matching results between different image rings; the inter-ring matching results are used to characterize the overlapping relationship of three-dimensional objects in different image rings.

6. The device according to claim 5, wherein: The bridge image determination unit comprises: The bridging image determination subunit is used to obtain the pair of images to be matched with the most matching feature points in the exhaustive matching result, and use the first image to be matched in the pair of images to be matched as the first bridging image, and use the second image to be matched in the pair of images to be matched as the second bridging image.

7. The device according to claim 5, wherein: The image ring division module comprises: An image ring determination unit, used for determining the image ring to which the image to be matched belongs according to the first two characters in the name of the image to be matched; The ring position determination unit is used to determine the position of the image to be matched in the image ring according to the last two characters of the name of the image to be matched; The image ring division unit is used to form at least two image rings according to the image ring to which the image to be matched belongs and the position of the image to be matched in the image ring.

8. The device according to claim 5, wherein: The in-loop matching result determination module includes: An adjacent image determining unit, configured to determine at least one adjacent image adjacent to the image to be matched on both sides of each image to be matched according to the arrangement order of the images to be matched in the image ring; The in-ring matching result determining unit is used to match the image to be matched with the adjacent image to obtain an in-ring matching result of the image ring.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the three-dimensional reconstruction method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the three-dimensional reconstruction method according to any one of claims 1 to 4.

11. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the three-dimensional reconstruction method according to any one of claims 1 to 4.

Images