Three-dimensional reconstruction method, device, equipment and medium based on multi-view geometry

Abstract

The embodiments of the present application disclose a three-dimensional reconstruction method, device, equipment and medium based on multi-view geometry, which relates to the field of aircraft manufacturing. The method realizes three-dimensional reconstruction by converting different sensor data into five-dimensional functions, constructing matching cost volumes using the principle of pixel offset, and extracting features. Among them, by generating a five-dimensional function including two-dimensional angle coordinates, two-dimensional image coordinates and the number of image channels from an image array, different sensor data can be standardized, and then the method can be applied to data collected by different multi-view sensors to realize three-dimensional reconstruction, thereby improving the universality of the method.

Description

Technical Field

[0001] The present application relates to the field of aircraft manufacturing technology, and in particular to a three-dimensional reconstruction method, device, equipment and medium based on multi-view geometry. Background Art

[0002] With the development of intelligent sensing technology, digital imaging technology and artificial intelligence technology, 3D reconstruction technology has also made rapid progress and has been continuously applied in the consumer market, industrial field and national defense industry. For non-contact passive 3D measurement methods, they mainly rely on ambient light, perform multi-view imaging of the object to be measured, and realize 3D reconstruction based on the principle of multi-view geometry. They have the characteristics of simple structure, high measurement accuracy and high measurement efficiency.

[0003] In the related art, the method of realizing three-dimensional reconstruction based on multi-view geometry only realizes three-dimensional reconstruction for data from a single multi-view sensor, and cannot realize three-dimensional reconstruction for data collected by different multi-view sensors, resulting in great limitations in the applicability of this method. Summary of the invention

[0004] The main purpose of the present application is to provide a three-dimensional reconstruction method, device, equipment and medium based on multi-view geometry, aiming to solve the problem in the prior art of realizing three-dimensional reconstruction based on data collected by different multi-view sensors, which leads to large applicability limitations.

[0005] To achieve the above objectives, the present application provides a 3D reconstruction method based on multi-view geometry, comprising:

[0006] Performing image decoding on target data collected by the multi-view sensor at different viewpoints of the target object to obtain an image array;

[0007] Generate a five-dimensional function according to the image array; wherein the five-dimensional function includes a two-dimensional angle coordinate, a two-dimensional image coordinate and the number of image channels;

[0008] Performing a pixel shift operation on the five-dimensional function to obtain a matching cost volume;

[0009] extracting a matching cost feature of the matching cost volume;

[0010] Based on the matching cost feature, the three-dimensional reconstruction of the target object is completed.

[0011] Optionally, the step of generating a five-dimensional function according to the image array comprises:

[0012] Performing epipolar correction on the image array to obtain a corrected image array;

[0013] A five-dimensional function is generated according to the rectified image array.

[0014] Optionally, the step of extracting the matching cost feature of the matching cost volume includes:

[0015] A multi-layer attention mechanism is used to extract cross-scale matching features of the matching cost volume.

[0016] Optionally, the step of extracting cross-scale matching features of the matching cost volume using a multi-layer attention mechanism includes:

[0017] A channel attention mechanism and a spatial attention mechanism are used to extract cross-scale matching features of the matching cost volume.

[0018] Optionally, the step of completing the three-dimensional reconstruction of the target object based on the matching cost feature includes:

[0019] Perform weighted summation on the matching cost features extracted by the multi-layer attention mechanism to obtain the weighted matching cost features;

[0020] Based on the weighted matching cost feature, the three-dimensional reconstruction of the target object is completed.

[0021] Optionally, the step of completing the three-dimensional reconstruction of the target object based on the weighted matching cost feature includes:

[0022] Based on the weighted matching cost feature, obtaining a disparity result;

[0023] Based on the parallax result, three-dimensional reconstruction of the target object is completed.

[0024] Optionally, before the step of extracting the matching cost features of the matching cost volume using the multi-layer attention mechanism, the step further includes:

[0025] A three-dimensional convolution operation is used to extract matching features of the matching cost volume.

[0026] In addition, to achieve the above-mentioned purpose, the present application also provides a three-dimensional reconstruction device based on multi-view geometry, comprising:

[0027] An image decoding module is used to decode the target data collected by the multi-view sensor at different viewpoints of the target object to obtain an image array;

[0028] A data specification module, used to generate a five-dimensional function according to the image array; wherein the five-dimensional function includes a two-dimensional angle coordinate, a two-dimensional image coordinate and the number of image channels;

[0029] An offset operation module, used for performing a pixel offset operation on the five-dimensional function to obtain a matching cost volume;

[0030] A feature extraction module, used to extract matching cost features of the matching cost volume;

[0031] The three-dimensional reconstruction module is used to complete the three-dimensional reconstruction of the target object based on the matching cost feature.

[0032] In addition, to achieve the above-mentioned purpose, the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned method.

[0033] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement the above-mentioned method.

[0034] The beneficial effects that can be achieved by this application.

[0035] The embodiment of the present application proposes a three-dimensional reconstruction method, device, equipment and medium based on multi-view geometry. The method includes: decoding the target data collected by the multi-view sensor at different viewpoints for the target object to obtain an image array; generating a five-dimensional function according to the image array; wherein the five-dimensional function includes two-dimensional angle coordinates, two-dimensional image coordinates and the number of image channels; performing pixel shift operation on the five-dimensional function to obtain a matching cost volume; extracting the matching cost features of the matching cost volume; and completing the three-dimensional reconstruction of the target object based on the matching cost features. That is, the method realizes three-dimensional reconstruction by converting different sensor data into five-dimensional functions, constructing a matching cost volume using the principle of pixel shift, and extracting features. Among them, by generating a five-dimensional function including two-dimensional angle coordinates, two-dimensional image coordinates and the number of image channels from the image array, different sensor data can be standardized, and then the method can be applied to data collected by different multi-view sensors to realize three-dimensional reconstruction, thereby improving the universality of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of the computer device structure of the hardware operating environment involved in the embodiments of the present application;

[0037] Figure 2 A schematic diagram of a flow chart of a 3D reconstruction method based on multi-view geometry according to an embodiment of the present application;

[0038] Figure 3 This is a diagram showing a process of decoding target data into a camera array image according to an embodiment of the present application;

[0039] Figure 4 This is a schematic diagram of the functional modules of a three-dimensional reconstruction device based on multi-view geometry according to an embodiment of the present application.

[0040] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0041] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] The main solution of the embodiments of the present application is: to provide a three-dimensional reconstruction method, device, equipment and medium based on multi-view geometry, the method comprising: performing image decoding on target data collected by a multi-view sensor at different viewpoints for a target object to obtain an image array; generating a five-dimensional function based on the image array; wherein the five-dimensional function comprises two-dimensional angular coordinates, two-dimensional image coordinates and the number of image channels; performing a pixel offset operation on the five-dimensional function to obtain a matching cost volume; extracting matching cost features of the matching cost volume; and completing three-dimensional reconstruction of the target object based on the matching cost features.

[0043] After analysis, it was found that typical 3D reconstruction technologies include contact and non-contact. Among them, the three-dimensional coordinate measuring machine is a widely used contact 3D reconstruction method, but this method has a slow measurement speed and can only obtain relatively sparse point cloud data, which is difficult to meet the growing needs of 3D measurement. Non-contact 3D measurement methods are divided into active and passive. Structured light, lidar, laser tracker, etc. actively project stripes or lasers onto the surface to be measured. They are typical active 3D measurement methods and are widely used in the industrial field. However, active methods require active projection of light and have complex structures, which limits their use environment.

[0044] Passive three-dimensional measurement methods mainly rely on ambient light, perform multi-view imaging of the object to be measured, and realize three-dimensional reconstruction based on the principle of multi-view geometry. They have the characteristics of simple structure, high measurement accuracy and high measurement efficiency. Typical passive measurement systems include binocular vision systems, multi-view vision systems and light field imaging. In related technologies, the influence of reflective areas on the accuracy of three-dimensional reconstruction can be solved by setting three modules: light source control, reflective area acquisition and image fusion, so as to realize three-dimensional reconstruction of target objects without reflective areas. A multi-view visual imaging system can also be built using multiple cameras, bases and four-way brackets, which can obtain image arrays of different perspectives of the target to be measured, and can realize three-dimensional reconstruction in combination with corresponding processing algorithms. Although the above methods all realize multi-view three-dimensional reconstruction, these methods are only for single multi-view sensor data, and cannot realize three-dimensional reconstruction for data collected by different multi-view sensors, resulting in large limitations in the applicability of this method.

[0045] To this end, the present application provides a solution, a three-dimensional reconstruction method, device, equipment and medium based on multi-view geometry, which realizes three-dimensional reconstruction by converting different sensor data into five-dimensional functions, constructing matching cost volumes using the principle of pixel offset, and extracting features. Among them, by generating a five-dimensional function including two-dimensional angle coordinates, two-dimensional image coordinates and the number of image channels from an image array, different sensor data can be standardized, and then the method can be applied to data collected by different multi-view sensors to realize three-dimensional reconstruction, thereby improving the universality of the method.

[0046] Reference Figure 1 , Figure 1 A schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application.

[0047] like Figure 1 As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0048] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0049] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a memory, a network communication module, a user interface module, and a three-dimensional reconstruction device based on multi-view geometry.

[0050] exist Figure 1In the computer device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the computer device of the present invention can be set in the computer device, and the computer device calls the three-dimensional reconstruction device based on multi-view geometry stored in the memory 1005 through the processor 1001, and executes the three-dimensional reconstruction method based on multi-view geometry provided in the embodiment of the present application.

[0051] Reference Figure 2 Based on the hardware device of the aforementioned embodiment, the embodiment of the present application provides a 3D reconstruction method based on multi-view geometry, the method comprising:

[0052] S20, performing image decoding on target data collected by the multi-view sensor at different viewpoints of the target object to obtain an image array;

[0053] In the specific implementation process, the multi-view sensor refers to a multi-view visual sensor, which refers to the image data collected from different angles of an object at the same time. The multi-view visual sensor can quickly switch the shooting angle and can stitch images from different angles together to finally generate a stereo image. The target object refers to the object of three-dimensional reconstruction, which can be any three-dimensional object. For example, aircraft parts. The target data refers to the image data collected at different viewpoints using a multi-view sensor.

[0054] Specifically, after obtaining the target data, the method of this embodiment uses the existing decoding method to decode it into a camera array image. Figure 3 , Figure 3 A diagram showing a process of decoding target data into a camera array image. The target data in the figure is the image data of a statue.

[0055] S40, generating a five-dimensional function according to the image array; wherein the five-dimensional function includes a two-dimensional angle coordinate, a two-dimensional image coordinate, and the number of image channels;

[0056] In the specific implementation process, the five-dimensional function can be expressed as f(T)=L(u, v, x, y, c),

[0057] Where T represents the data space of different multi-view imaging sensors; L(u, v, x, y, c) represents a five-dimensional function, (u, v) represents the angle coordinate system used to index images in the image array, (x, y) represents the image coordinate system used to index pixels in each image; c represents the number of image channels.

[0058] It is understandable that when capturing images of the same target object from different viewpoints, in addition to different shooting angles, the captured data may be inconsistent due to different multi-view sensors used (or different parameters set for the same multi-view sensor, for example, different number of image channels), making it impossible to perform three-dimensional reconstruction based on these inconsistent data. In this embodiment, a five-dimensional function is used to normalize different sensor data, and subsequent steps are combined to achieve three-dimensional reconstruction of data collected by different multi-view sensors, thereby improving the universality of the method.

[0059] As an optional implementation manner, the step of generating a five-dimensional function according to the image array includes:

[0060] Performing epipolar correction on the image array to obtain a corrected image array;

[0061] A five-dimensional function is generated according to the rectified image array.

[0062] In the specific implementation process, in order to reduce the amount of calculation in the three-dimensional reconstruction process, the surface matching is converted into line matching, the image array can be subjected to epipolar correction, and then a five-dimensional function is generated according to the corrected image array. Epipolar correction is a prior art and will not be described in detail in this embodiment.

[0063] In an optional implementation, the multi-view sensor may be a light field camera. The original image decoding of the light field camera is mainly performed by calibrating the center point of the white image, determining the position of each microlens, and sampling the microlens array from a hexagonal arrangement to a quadrilateral arrangement, thereby obtaining a five-dimensional function. Due to the special properties of the microlens array light field camera itself, epipolar correction is not required.

[0064] S60, performing a pixel shift operation on the five-dimensional function to obtain a matching cost volume;

[0065] In the specific implementation process, based on the above five-dimensional function, it is assumed that the range of the parallax d is d∈{d min , ..., d max}, the matching cost volume can be constructed as follows:

[0066]

[0067] In the formula, Represents the offset characteristics of the viewing angle (u, v) under parallax d; represents the cost value at disparity d; (u c , v c ) represents the central view image; c represents the image color channel. The cost volume is obtained by stacking the cost values ​​under all parallaxes, that is, Therefore, the matching cost volume is Wherein, num_d represents the number of elements in the disparity set.

[0068] S80: Extract matching cost features of the matching cost volume.

[0069] In a specific implementation process, based on the above, a three-dimensional convolution operation can be used to extract the matching features of the matching cost volume. Specifically, the matching cost features can be extracted using the following matching feature formula:

[0070] f=ω2*(σ(ω1*C+b1))+b2

[0071] In the formula, Represents matching features; * represents a three-dimensional convolution operation; (ω1, b1) and (ω2, b2) represent the weights and biases of the three-dimensional convolution with a convolution kernel of 1*1 and a convolution kernel of 2*2, respectively; σ(·) represents the activation function, i.e., ReLU().

[0072] At the same time, as an optional implementation, a multi-layer attention mechanism can also be used to extract cross-scale matching features of the matching cost volume.

[0073] In the specific implementation process, the channel attention mechanism and the spatial attention mechanism can be used to focus on the cross-scale matching features, enhance the accuracy and robustness of 3D reconstruction, and reduce the mismatch rate in weak texture and strong noise conditions. In addition, compared with the existing method that does not pay attention to cross-scale matching features, the method can adapt to different multi-view sensor data.

[0074] Specifically, it can be achieved by the following methods:

[0075] A C =softmax(AvgPool(σ(ω3*f+b3))),

[0076] In the formula, (ω3, b3) represents the weight and bias of the three-dimensional convolution with a convolution kernel of 3*3; AvgPool(·) represents the average pooling operation; softmax(·) represents the activation function; A C It is the channel attention mechanism.

[0077] F=A C ☉f

[0078] In the formula, ⊙ represents feature-by-feature multiplication; Represents the weighted features of the channel attention mechanism. At the same time, the spatial attention mechanism also helps to improve the robustness of matching cost feature extraction.

[0079] A S =Sigmod(ω4*F+b4),

[0080] In the formula, A Sis a weight matrix used to focus on matching features; Sigmod(·) represents a nonlinear activation function; (ω4, b4) represents the weight and bias of a three-dimensional convolution with a convolution kernel of 4*4.

[0081]

[0082] Where · represents pixel-by-pixel multiplication; D(...,...) represents the cascade operation; (ω5, b5) represents the weight and bias of the three-dimensional convolution with a convolution kernel of 5*5; Represents the weighted features of the spatial attention mechanism.

[0083] S100, completing three-dimensional reconstruction of the target object based on the matching cost feature;

[0084] In a specific implementation process, the step of completing the three-dimensional reconstruction of the target object based on the matching cost feature includes:

[0085] Perform weighted summation on the matching cost features extracted by the multi-layer attention mechanism to obtain the weighted matching cost features;

[0086] Based on the weighted matching cost feature, the three-dimensional reconstruction of the target object is completed.

[0087] Furthermore, the step of completing the three-dimensional reconstruction of the target object based on the weighted matching cost feature includes:

[0088] Based on the weighted matching cost feature, obtaining a disparity result;

[0089] Based on the parallax result, three-dimensional reconstruction of the target object is completed.

[0090] Specifically, the matching features extracted by multiple attention mechanisms are again converted to 1 by using a three-dimensional convolution operation, as shown below:

[0091]

[0092] In the formula, It represents the final matching cost volume, and the cost volume aggregation operation is used to obtain the disparity result, thereby realizing 3D reconstruction. (ω6, b6) represents the weight and bias of the 3D convolution with a convolution kernel of 6*6.

[0093]

[0094] In the formula, Represents the disparity estimation result.

[0095] Specifically, the existing deep learning framework PyTorch and programming language Python can be used to build the network, and Adam can be used to update the network parameters to make it converge, thereby realizing three-dimensional convolution and attention mechanism feature extraction.

[0096] In addition, simulation data can also be used as the actual 3D reconstruction result for supervising the network.

[0097] It should be understood that the above is only an example and does not constitute any limitation to the technical solution of the present application. Those skilled in the art can make settings based on needs in actual applications and no limitation is made here.

[0098] It is not difficult to find from the above description that the three-dimensional reconstruction method based on multi-view geometry of this embodiment includes: decoding the target data collected by the multi-view sensor at different viewpoints for the target object to obtain an image array; generating a five-dimensional function according to the image array; wherein the five-dimensional function includes two-dimensional angle coordinates, two-dimensional image coordinates and the number of image channels; performing pixel offset operation on the five-dimensional function to obtain a matching cost volume; extracting the matching cost features of the matching cost volume; and completing the three-dimensional reconstruction of the target object based on the matching cost features. That is, the method realizes three-dimensional reconstruction by converting different sensor data into five-dimensional functions, constructing a matching cost volume using the principle of pixel offset, and extracting features. Among them, by generating a five-dimensional function including two-dimensional angle coordinates, two-dimensional image coordinates and the number of image channels from the image array, different sensor data can be standardized, and then the method can be applied to data collected by different multi-view sensors to realize three-dimensional reconstruction, thereby improving the universality of the method.

[0099] Reference Figure 4 Based on the same inventive concept, an embodiment of the present application further provides a three-dimensional reconstruction device based on multi-view geometry, comprising:

[0100] An image decoding module is used to decode the target data collected by the multi-view sensor at different viewpoints of the target object to obtain an image array;

[0101] A data specification module, used to generate a five-dimensional function according to the image array; wherein the five-dimensional function includes a two-dimensional angle coordinate, a two-dimensional image coordinate and the number of image channels;

[0102] An offset operation module, used for performing a pixel offset operation on the five-dimensional function to obtain a matching cost volume;

[0103] A feature extraction module, used to extract matching cost features of the matching cost volume;

[0104] The three-dimensional reconstruction module is used to complete the three-dimensional reconstruction of the target object based on the matching cost feature.

[0105] It should be noted that each module in the three-dimensional reconstruction device based on multi-perspective geometry in this embodiment corresponds one-to-one to each step in the three-dimensional reconstruction method based on multi-perspective geometry in the aforementioned embodiment. Therefore, the specific implementation manner and technical effects achieved by this embodiment can refer to the implementation manner of the aforementioned three-dimensional reconstruction method based on multi-perspective geometry, and will not be repeated here.

[0106] In addition, in one embodiment, the present application further provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method in the aforementioned embodiment are implemented.

[0107] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0108] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0109] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0110] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0111] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0112] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0114] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A three-dimensional reconstruction method based on multi-view geometry, characterized in that: include: Performing image decoding on target data collected by the multi-view sensor at different viewpoints of the target object to obtain an image array; According to the image array, a five-dimensional function is generated; wherein the five-dimensional function includes a two-dimensional angle coordinate, a two-dimensional image coordinate and the number of image channels; the five-dimensional function is expressed as f(T)=L(u, v, x, y, c), wherein T represents the data space of different multi-view imaging sensors; L(u, v, x, y, c) represents the five-dimensional function, (u, v) represents the two-dimensional angle coordinate, (x, y) represents the two-dimensional image coordinate, and c represents the number of image channels; Perform pixel shift operation on the five-dimensional function to obtain a matching cost volume; based on the five-dimensional function, assume that the range of the disparity d is d∈{d min , ..., d max }, the matching cost volume can be constructed as follows: In the formula, Represents the offset characteristics of the viewing angle (u, v) under parallax d; represents the cost value at disparity d; (u c , v c ) represents the central view image; c represents the image color channel; the cost volume is obtained by stacking the cost values ​​under all parallaxes, that is, The matching cost volume is Among them, num_d represents the number of elements in the disparity set; extracting a matching cost feature of the matching cost volume; Based on the matching cost feature, the three-dimensional reconstruction of the target object is completed.

2. The method according to claim 1, characterized in that The step of generating a five-dimensional function according to the image array comprises: Performing epipolar correction on the image array to obtain a corrected image array; A five-dimensional function is generated according to the rectified image array.

3. The method according to claim 1, characterized in that The step of extracting the matching cost feature of the matching cost volume comprises: A multi-layer attention mechanism is used to extract cross-scale matching features of the matching cost volume.

4. The method according to claim 3, characterized in that The step of extracting cross-scale matching features of the matching cost volume using a multi-layer attention mechanism comprises: A channel attention mechanism and a spatial attention mechanism are used to extract cross-scale matching features of the matching cost volume.

5. The method according to claim 3, characterized in that The step of completing the three-dimensional reconstruction of the target object based on the matching cost feature includes: Perform weighted summation on the matching cost features extracted by the multi-layer attention mechanism to obtain the weighted matching cost features; Based on the weighted matching cost feature, the three-dimensional reconstruction of the target object is completed.

6. The method according to claim 5, characterized in that The step of completing the three-dimensional reconstruction of the target object based on the weighted matching cost feature includes: Based on the weighted matching cost feature, obtaining a disparity result; Based on the parallax result, three-dimensional reconstruction of the target object is completed.

7. The method according to claim 3, characterized in that Before the step of extracting the matching cost features of the matching cost volume using the multi-layer attention mechanism, the step further includes: A three-dimensional convolution operation is used to extract matching features of the matching cost volume.

8. A three-dimensional reconstruction device based on multi-view geometry, characterized in that: include: An image decoding module is used to decode the target data collected by the multi-view sensor at different viewpoints of the target object to obtain an image array; A data specification module is used to generate a five-dimensional function according to the image array; wherein the five-dimensional function includes a two-dimensional angle coordinate, a two-dimensional image coordinate and the number of image channels; the five-dimensional function is expressed as f(T)=L(u, v, x, y, c), wherein T represents the data space of different multi-view imaging sensors; L(u, v, x, y, c) represents the five-dimensional function, (u, v) represents the two-dimensional angle coordinate, (x, y) represents the two-dimensional image coordinate, and c represents the number of image channels; The offset operation module is used to perform pixel offset operation on the five-dimensional function to obtain a matching cost volume; based on the five-dimensional function, assuming that the range of the disparity d is d∈{d min , ..., d max }, the matching cost volume can be constructed as follows: In the formula, Represents the offset characteristics of the viewing angle (u, v) under parallax d; represents the cost value at disparity d; (u c , v c ) represents the central view image; c represents the image color channel; the cost volume is obtained by stacking the cost values ​​under all parallaxes, that is, The matching cost volume is Among them, num_d represents the number of elements in the disparity set; A feature extraction module, used to extract matching cost features of the matching cost volume; The three-dimensional reconstruction module is used to complete the three-dimensional reconstruction of the target object based on the matching cost feature.

9. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the processor executes the computer program to implement the method according to any one of claims 1 to 7.

Images