Method for three-dimensional reconstruction of object, electronic device, system for three-dimensional reconstruction of object, and storage medium

Abstract

Embodiments of the present application relate to the technical field of three-dimensional reconstruction, and disclose a method for three-dimensional reconstruction of an object, an electronic device, a computer-readable storage medium, and a system. The method comprises: acquiring an upright image set, processing the upright image set on the basis of a preset three-dimensional reconstruction algorithm, obtaining a reference image model, acquiring a flipped image set, registering the flipped image set to the reference image model, and fusing upright three-dimensional description information and flipped three-dimensional description information on the basis of the preset three-dimensional imaging algorithm, and obtaining a three-dimensional model. According to the embodiments, it is not necessary to use professional hardware equipment to collect three-dimensional information of an object, and it is only necessary to collect an upright view and a flipped view of a target object by means of a mobile terminal, so that an electronic device can perform three-dimensional reconstruction of the object, thereby improving the application scope of the embodiments of the present application.

Description

3D object reconstruction methods, electronic devices, 3D object reconstruction systems and storage media

[0001] This application claims priority to Chinese Patent Application No. 2024111989942, filed on August 29, 2024, entitled "Method for Three-Dimensional Reconstruction of Objects, Electronic Device and Computer-Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of three-dimensional reconstruction technology, and in particular to a method for three-dimensional reconstruction of an object, an electronic device, a system for three-dimensional reconstruction of an object, and a storage medium. Background Technology

[0003] The 3D object reconstruction methods provided by related technologies require specialized hardware to acquire the object's 3D information, such as using LiDAR, structured light components, or multi-view vision cameras. The object is then reconstructed based on this information. Many application scenarios lack the necessary specialized hardware, which limits the application of these technologies. Summary of the Invention

[0004] One objective of this application is to provide a method for three-dimensional reconstruction of an object, an electronic device, a system for three-dimensional reconstruction of an object, and a storage medium, in order to solve the technical problem that the application scope of related technologies is easily limited.

[0005] In a first aspect, embodiments of this application provide a method for three-dimensional reconstruction of an object, applied to an electronic device for communication connection with a mobile terminal. The method includes: acquiring an upright image set, the upright image set including at least one upright view of a target object in an upright state obtained by the mobile terminal from different shooting angles; processing the upright image set according to a preset three-dimensional reconstruction algorithm to obtain a reference image model, the reference image model carrying upright three-dimensional description information of each upright view; acquiring a flipped image set, the flipped image set including at least one target flipped view of a target object in a flipped state obtained by the mobile terminal from different shooting angles; registering the flipped image set to the reference image model so that the flipped three-dimensional description information of the target flipped view is mapped to the reference image model; and fusing the upright three-dimensional description information and the flipped three-dimensional description information of the reference image model according to a preset three-dimensional imaging algorithm to obtain a three-dimensional model of the target object.

[0006] This application embodiment does not require specialized hardware equipment to collect the object's 3D information. It only requires acquiring the target object's frontal and flipped views via a mobile terminal, enabling electronic devices to perform 3D reconstruction. This expands the application scope of this application embodiment. Furthermore, this application embodiment can accurately reconstruct the object's 3D model, and the 3D reconstruction process is relatively simple, eliminating the need to construct multiple sets of 3D point clouds, thus improving the efficiency of 3D reconstruction.

[0007] Optionally, the upright 3D description information includes upright posture information and upright feature information. The step of processing the upright image set according to a preset 3D reconstruction algorithm to obtain a reference image model includes: processing multiple upright views according to a preset 3D reconstruction algorithm to obtain a reference image model, the reference image model carrying the upright posture information of each upright view, determining the upright feature information of each upright view, and updating the reference image model according to each upright feature information to obtain a reference image model.

[0008] In this embodiment, a reference image model is constructed with the frontal view as the center. Since the frontal view contains a lot of shape information about the target object, the reference image model can also be configured with the frontal three-dimensional description information of the frontal view to the maximum extent.

[0009] Optionally, determining the elevation feature information of each of the elevation views includes: extracting an elevation foreground image corresponding to the target object in each elevation view, performing feature extraction processing on the elevation foreground image, and obtaining the elevation feature information of each of the elevation views.

[0010] In this embodiment, before constructing the reference image model, the foreground image and background image of the upright view are decoupled, invalid background images are removed, and the foreground image is retained to construct the reference image model. This helps to improve the accuracy and reliability of the reference image model.

[0011] Optionally, obtaining the flipped image set includes: obtaining a reference flipped view, wherein the reference flipped view is an image of a target object in a flipped state captured by the mobile terminal from the target shooting perspective; detecting whether the reference flipped view meets preset image registration conditions; if it does, then the reference flipped view is placed into the flipped image set as the target flipped view; if it does not, then reshooting information is generated, wherein the reshooting information is used to prompt the user to reshoot the target object in a flipped state.

[0012] In this embodiment, each reference flipped view needs to be checked to see if it meets the preset image registration conditions. This avoids registering low-quality or invalid reference flipped views as target flipped views to the base image model, thereby improving the accuracy of the three-dimensional model of the target object.

[0013] Optionally, the upright 3D description information includes upright feature information, and the step of detecting whether the reference flipped view meets the preset image registration conditions includes: determining the flipped feature information of the reference flipped view; among the upright feature information of multiple upright views, detecting whether there is upright feature information with a matching degree greater than or equal to a preset matching threshold; if so, determining that the reference flipped view meets the preset image registration conditions; if not, determining that the reference flipped view does not meet the preset image registration conditions. This embodiment of the application, by combining the upright feature information of the upright view, can quickly and effectively determine whether the reference flipped view is similar to the upright view, ensuring that the found target flipped view can be effectively registered to the reference image model and avoiding registration failure.

[0014] Optionally, determining the flip feature information of the reference flip view includes: extracting a flip foreground image corresponding to the target object from the reference flip view, performing feature extraction processing on the flip foreground image, and obtaining the flip feature information of each reference flip view.

[0015] In this embodiment, before updating the reference image model, the flipped foreground image and the flipped background image of the flipped view are decoupled, invalid flipped background images are removed, and the flipped foreground image is retained to update the reference image model. This helps to improve the accuracy and reliability of the 3D model.

[0016] Optionally, obtaining the reference flipped view includes: controlling the mobile terminal to display flipped shooting guidance information, the flipped shooting guidance information being used to prompt the user to operate the mobile terminal to shoot the target object in a flipped state from the target shooting perspective; responding to the user's shooting operation input to the mobile terminal; and controlling the camera of the mobile terminal to shoot the target object in a flipped state from the target shooting perspective to obtain a reference flipped view. Under the guidance of the flipped shooting guidance information, this embodiment of the application can systematically acquire high-quality reference flipped views, avoiding invalid reference flipped views acquired due to user confusion.

[0017] Optionally, the reference flip view includes an image of the invisible area of ​​the target object in an upright position.

[0018] This application embodiment can capture the invisible areas of a target object in its upright position into a reference flipped view, thus ensuring that the image of the invisible area can also be registered in the base image model. The final output 3D model of the target object in this application embodiment can not only restore the front and side information of the target object, but also restore the invisible areas of the target object in its upright position, thereby providing users with a more complete 3D model of the target object.

[0019] Optionally, after placing the reference flipped view as the target flipped view into the flipped image set, the method further includes: determining the number of target flipped views contained in the flipped image set; if the number is less than a preset threshold, generating continue shooting information, which prompts the user to continue shooting the target object in a flipped state from different shooting angles. By employing this method, this embodiment of the application can obtain a sufficient number of high-quality target flipped views, greatly enriching the view information about various flipping angles of the target object contained in the flipped image set, increasing the proportion of high-quality target flipped views registered to the reference image model, and facilitating the generation of a more complete and high-quality 3D model.

[0020] Optionally, the upright 3D description information includes upright posture information, and the flipped 3D description information includes flipped posture information and flipped feature information. The step of registering the flipped image set to the reference image model so that the flipped 3D description information of the target flipped view is mapped to the reference image model includes: finding an upright view that matches the target flipped view in at least one of the upright views as the target upright view; determining the flipped posture information of the target flipped view based on the upright posture information of the target upright view; obtaining the flipped feature information of the target flipped view; and registering the flipped posture information and flipped feature information of the target flipped view to the reference image model.

[0021] This application embodiment uses image matching to find a target upright view similar to the target flipped view. The upright posture information of the target upright view has been obtained in the construction stage of the reference image model. This application embodiment derives the flipped posture information of the target flipped view based on the known upright posture information. It is not necessary to obtain the flipped posture information of each target flipped view based on multiple target flipped views, which helps to improve the efficiency of registering the target flipped view to the reference image model.

[0022] Optionally, acquiring the upright image set includes: controlling the mobile terminal to display upright shooting guidance information, the upright shooting guidance information being used to prompt the user to operate the mobile terminal to surround the target object in an upright state for shooting; responding to the user's shooting operation input to the mobile terminal, controlling the camera of the mobile terminal to surround the target object in an upright state for shooting, thereby obtaining an upright image set. This embodiment of the application guides the user to correctly shoot the target object through upright shooting guidance information in order to obtain a high-quality upright view, which helps to improve the accuracy of constructing the three-dimensional model of the target object and enables high-quality reconstruction of the three-dimensional model of the target object.

[0023] In a second aspect, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory is connected to the processor, and the processor is configured to execute one or more computer programs stored in the memory, wherein when the processor executes the one or more computer programs, the electronic device enables the above-described object three-dimensional reconstruction method.

[0024] In a third aspect, embodiments of this application provide a three-dimensional object reconstruction system, including a mobile terminal and the aforementioned electronic device, wherein the electronic device is communicatively connected to the mobile terminal.

[0025] Optionally, the mobile terminal is equipped with a data acquisition application, which is configured to locate the target object, generate a view frame at the location of the target object to encompass the target object, and display upright shooting guidance information on the application interface of the data acquisition application. The upright shooting guidance information is used to prompt the user to operate the mobile terminal to shoot around the target object in an upright state.

[0026] Optionally, the electronic device is configured to, after receiving multiple upright views sent by the mobile terminal, construct a reference image model based on the upright views, and send view response information to the data acquisition application of the mobile terminal. The view response information is used to instruct the mobile terminal to end the acquisition of upright views and proceed to the operation of acquiring flipped views.

[0027] Optionally, the data acquisition application is also configured to respond to view reply information, presenting object flipping operation information on the application interface. The object flipping operation information is used to prompt the user to flip the target object in order to display the bottom of the target object.

[0028] Optionally, the data acquisition application is further configured to present flip shooting guidance information on the application interface, the flip shooting guidance information being used to prompt the user to operate the mobile terminal to shoot the target object in a flipped state from the target shooting perspective.

[0029] Optionally, the electronic device is further configured to, upon receiving a plurality of the flipped views sent by the mobile terminal, update the reference image model according to the flipped views, and generate a three-dimensional model of the target object based on the updated reference image model.

[0030] Optionally, the data acquisition application is configured to display an upright shooting end message or a flipping end message on the application interface when it detects that the number of upright views or flipped views has reached a preset number.

[0031] In a fourth aspect, embodiments of this application provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the above-described object three-dimensional reconstruction method. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 is a schematic diagram of the system architecture of a three-dimensional object reconstruction system provided in an embodiment of this application;

[0034] Figure 2 is a schematic diagram of generating a view frame to enclose the target object at the position of the target object according to an embodiment of this application;

[0035] Figure 3 shows multiple upright views of the target object provided in the embodiments of this application in an upright state;

[0036] Figure 4 shows multiple flipped views of the target object in a flipped state provided in the embodiments of this application;

[0037] Figure 5 is a flowchart illustrating a three-dimensional object reconstruction method provided in an embodiment of this application;

[0038] Figure 6 is a schematic diagram of the reference image model provided in the embodiment of this application, wherein the orientation information of each frontal view is mapped onto the coordinate system of the reference image model;

[0039] Figure 7 is a schematic diagram of extracting multiple frontal foreground images from multiple frontal views according to an embodiment of this application;

[0040] Figure 8 is a schematic diagram of the reference image model provided in the embodiment of this application, wherein the upright posture information and upright feature information of each upright view are mapped onto the coordinate system of the reference image model;

[0041] Figure 9 is a schematic diagram of extracting multiple flipped foreground images from multiple flipped views according to an embodiment of this application;

[0042] Figure 10 is a schematic diagram of mapping the flipping posture information of the target flipped view onto the coordinate system of the reference image model according to an embodiment of this application;

[0043] Figure 11 is a schematic diagram of mapping the flipping posture information and flipping feature information of the target flipped view onto the coordinate system of the reference image model according to an embodiment of this application;

[0044] Figure 12 is a schematic diagram of the three-dimensional model of the reconstructed target object provided in an embodiment of this application;

[0045] Figure 13 is a schematic diagram of the structure of a three-dimensional object reconstruction device provided in an embodiment of this application;

[0046] Figure 14 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Embodiments of the present invention

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0048] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this application do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0049] 3D object reconstruction refers to the restoration and reconstruction of an object in three dimensions. Object 3D reconstruction technology has been widely applied in fields such as games, movies, surveying and mapping, positioning, navigation, autonomous driving, VR / AR, industrial manufacturing, and consumer goods. The object 3D reconstruction methods provided by related technologies can be divided into the following two types:

[0050] 1) Object 3D Reconstruction Method Based on Multi-View Geometry

[0051] Based on multi-view geometry, 3D object reconstruction methods can be divided into active and passive methods according to whether the sensor actively illuminates the object. Active 3D reconstruction methods involve the sensor actively emitting signals towards the object and then retrieving the returned signals to obtain the object's 3D information. Common active 3D reconstruction methods include structured light-based methods, Time-of-Flight (TOF) laser-based methods, and triangulation-based methods. Passive 3D reconstruction methods rely directly on ambient light sources to acquire RGB images. These images are then analyzed using multi-view geometry principles (monocular vision or binocular / multi-view vision) to obtain the object's 3D information.

[0052] Such technologies require specialized hardware to acquire three-dimensional information about objects, which can limit their application scope.

[0053] This application embodiment does not require the use of professional hardware equipment to collect the three-dimensional information of the object. It only requires installing a data acquisition application (APP) on a commonly used mobile phone, collecting the frontal and flipped views of the object through the data acquisition application, and transmitting the frontal and flipped views of the object to an electronic device for three-dimensional reconstruction of the object. This helps to improve the application scope of this application embodiment.

[0054] 2) The Neural Radiance Fields (NeRF) method, a machine learning-based method for 3D object reconstruction, is a type of machine learning algorithm that trains neural radiance fields to learn and infer the color and depth information of each point in the scene, thereby achieving high-precision new view synthesis and 3D object reconstruction.

[0055] The 3D Gaussian Splatting (3D-GS) method utilizes a set of Gaussian ellipsoids to construct a scene, thereby achieving efficient rendering by rasterizing the Gaussian ellipsoids into an image. In addition to its fast rendering speed, the explicit representation of 3D-GS can facilitate the advancement of downstream tasks such as dynamic reconstruction, geometry editing, and physical simulation.

[0056] When related technologies use machine learning to perform comprehensive 3D reconstruction of objects, the 3D model of the reconstruction is required to include not only the front and side views of the object when it is upright, but also the top view of the object when it is upright, that is, the bottom area of ​​the object when it is upright. Therefore, related technologies usually need to collect multiple upright views of the object in an upright state and multiple flipped views in a flipped state.

[0057] The relevant technology generates a first 3D point cloud based on multiple upright views and a second 3D point cloud based on multiple flipped views. The first and second 3D point clouds are then matched and aligned in the same coordinate system to obtain the omnidirectional visual information of the object in that coordinate system. Finally, the NeRF algorithm or 3D-GS algorithm is combined to perform 3D reconstruction, thereby obtaining the 3D model of the object.

[0058] In some applications, bottom information of an object is just as important as front or side information. For example, in online e-commerce product displays, the bottom information of a product directly affects consumers' purchasing intentions. Therefore, the aforementioned technologies can clearly display the bottom information of an object. However, these technologies require collecting a large number of frontal and flipped views to accurately reconstruct the bottom information of the object. This also means that the technologies need to generate a large amount of first and second 3D point clouds based on numerous frontal and flipped views, and then integrate the first and second 3D point clouds. This process is cumbersome, consumes a lot of computing power and time, and has low real-time performance.

[0059] This application embodiment first constructs a reference image model based on the upright view, then registers the target flipped view into the reference image model, and finally constructs the object's 3D model based on the upright and flipped 3D description information of the reference image model. This application embodiment can accurately restore the object's 3D model, and the 3D reconstruction process is relatively simple, requiring no construction of two sets of 3D point clouds and no addition of too many flipped views, thus improving the efficiency of 3D reconstruction. Furthermore, the object 3D reconstruction method provided by this application embodiment is well-suited for application scenarios where the requirements for bottom information of the object are not high. For example, in educational scenarios where a bottle needs to be shown to students, whether the bottom information of the bottle can be clearly presented is not important; rather, showing students the front or side information of the bottle is more important. Using the object 3D reconstruction method provided by this application embodiment, the 3D model of the bottle can be restored efficiently and quickly.

[0060] The following embodiments of this application provide a three-dimensional object reconstruction system. Referring to Figure 1, the three-dimensional object reconstruction system 100 includes a mobile terminal 200 and an electronic device 300, and the electronic device 300 and the mobile terminal 200 are communicatively connected.

[0061] The mobile terminal 200 is equipped with a data acquisition application. When a user needs to perform 3D reconstruction of an object, the user operates the mobile terminal 200 to open the data acquisition application. After the data acquisition application starts, it controls the mobile terminal 200 to establish a communication connection with the electronic device 300.

[0062] Please refer to Figure 2. The data acquisition application locates the target object and generates a view frame 21 at the position of the target object 20 to surround the target object 20. The application interface of the data acquisition application displays upright shooting guidance information, which prompts the user to operate the mobile terminal 200 to shoot around the target object 20 which is in an upright state.

[0063] Please refer to Figure 3. With the help of the perspective frame 21 and the guidance of the upright shooting guidance information, the user can operate the mobile terminal to take pictures of the target object 20 in sequence from different shooting angles, except that the bottom of the target object 20 cannot be captured, thus obtaining multiple upright views 22.

[0064] After receiving multiple upright views, the electronic device 300 constructs a reference image model based on these views and sends a view response message to the data acquisition application of the mobile terminal 200. This view response message instructs the mobile terminal 200 to end the acquisition of the upright views and proceed to acquire a flipped view. The data acquisition application responds to the view response message by displaying object flipping operation information on the application interface. This object flipping operation information prompts the user to flip the target object 20 to reveal its bottom. After viewing the object flipping operation information, the user actively flips the target object 20, exposing its bottom to the mobile terminal's camera.

[0065] The data acquisition application displays flip shooting guidance information on the application interface. The flip shooting guidance information is used to prompt the user to operate the mobile terminal to shoot the target object 20 in a flipped state from the target shooting perspective.

[0066] Please refer to Figure 4. Under the guidance of the flip shooting guide information, the user operates the mobile terminal to shoot not only the side of the target object, but also the bottom of the target object from different shooting angles, so as to obtain multiple flip views 23.

[0067] After receiving multiple flipped views 23, the electronic device updates the reference image model based on the flipped views 23. Finally, the electronic device generates a three-dimensional model of the target object 20 based on the updated reference image model.

[0068] In some embodiments, when the data acquisition application detects that the number of upright views or flipped views has reached a preset number, it displays upright shooting end information or flipping end information on the application interface. When the user sees the upright shooting end information or flipping end information, he or she operates the mobile terminal to end the operation of the data acquisition application in acquiring upright views or acquiring flipped views.

[0069] In some embodiments, users can operate the mobile terminal at any time to end the data acquisition application's operation of acquiring an upright view or acquiring a flipped view, according to their own needs.

[0070] In some embodiments, the data acquisition application transmits the upright view or the flipped view to the electronic device 300 each time it captures an upright view or a flipped view.

[0071] In some embodiments, after the data acquisition application acquires a preset number of upright views or a preset number of flip views, it transmits the preset number of upright views or the preset number of flip views to the electronic device 300.

[0072] It is understood that in some embodiments, the mobile terminal and the electronic device can be an integrated device, that is, the data acquisition function of the mobile terminal and the object three-dimensional reconstruction function of the electronic device can be integrated on the same device. Therefore, the embodiments of this application do not impose any limitations on the product shape of the mobile terminal and the electronic device. The mobile terminal and the electronic device can be two components in an integrated device, or they can be two devices that are physically completely separate.

[0073] The following embodiment of this application provides another method for three-dimensional reconstruction of an object, applied to an electronic device used for communication connection with a mobile terminal. Referring to Figure 5, the method for three-dimensional reconstruction of an object includes the following steps:

[0074] S51: Obtain the set of upright images.

[0075] In this step, the upright image set includes at least one upright view of the target object taken by the mobile terminal from different shooting angles.

[0076] The shooting angle is the position and angle of the mobile terminal's camera relative to the target object. For example, if the mobile terminal is 20 centimeters away from the target object, the target object is photographed from shooting angles of 0 degrees, 30 degrees, 60 degrees, 90 degrees, 120 degrees, ..., 360 degrees, resulting in 12 frontal views.

[0077] The upright state refers to the state in which the target object exposes its main surface information to the mobile terminal. It's understandable that the understanding of an object's main surface information is influenced by different usage scenarios or the perceptions of different groups of people. Therefore, the upright state of different types of objects can manifest in many different ways. For example, the bottle shown in Figure 2 is in an upright state, where it exposes both its front and side information to the mobile terminal for photographing. Similarly, the upright state of a book exposes its cover.

[0078] An orthogonal view is an image obtained by a mobile terminal capturing a target object in an upright position. Different orthogonal views may have different shooting angles. As shown in Figure 3, the mobile terminal captures the target object from different shooting angles, resulting in multiple orthogonal views.

[0079] In some embodiments, obtaining an upright image set includes the following steps: controlling a mobile terminal to display upright shooting guidance information, which prompts the user to operate the mobile terminal to take pictures around the upright target object; responding to the user's shooting operation input to the mobile terminal, controlling the mobile terminal's camera to take pictures around the upright target object, thereby obtaining an upright image set. This application embodiment guides the user to correctly photograph the target object through upright shooting guidance information, in order to obtain a high-quality upright view. This helps improve the accuracy of constructing the 3D model of the target object and enables high-quality reconstruction of the 3D model of the target object.

[0080] In some embodiments, obtaining an upright image set includes the following steps: reading an upright image set pre-stored in the target device, wherein the target device reads the upright image set from the mobile terminal and saves it locally.

[0081] S52: Process the upright image set according to the preset three-dimensional reconstruction algorithm to obtain the reference image model.

[0082] In this step, the reference image model carries the 3D description information of each upright view. This 3D description information describes the pose and shape of the target object when photographed by the mobile terminal in an upright state. The 3D description information includes upright pose information and upright feature information. The upright pose information represents the pose of the target object when photographed by the mobile terminal in an upright state, while the upright feature information represents the characteristics of the target object in an upright state, including shape features or color features, etc.

[0083] The reference image model is configured with a coordinate system, and the three-dimensional description information of each frontal view is mapped onto the same coordinate system. In this way, the reference image model carries the three-dimensional description information of each frontal view.

[0084] The preset 3D reconstruction algorithms include SFM (Structure from Motion) and others. SFM can reconstruct a 3D reference image model from multiple 2D orthogonal views. In this embodiment, feature points of each orthogonal view are extracted using the SFM algorithm. Feature matching processing is then performed on these feature points using the SFM algorithm to obtain two mutually matching orthogonal views. Camera pose estimation is then performed on each pair of mutually matching orthogonal views using the SFM algorithm to obtain the orthogonal pose information for each orthogonal view. Finally, based on the SFM algorithm and the principle of triangulation, the 3D position of each feature point in each orthogonal view in the coordinate system is estimated, thereby obtaining the reference image model.

[0085] S53: Obtain the flipped image set.

[0086] In this step, the flipped image set includes at least one flipped view of the target object obtained by the mobile terminal shooting the target object in a flipped state from different shooting angles.

[0087] A flipped state is a state that exposes surface information of a target object that is not visible when it is upright. It's understandable that, similarly, the flipped state can manifest in many different ways for different types of objects. For example, the bottle shown in Figure 4 is in a flipped state, in which the bottle's bottom information is exposed to the mobile terminal for photographing. Similarly, the flipped state of a book exposes its back side.

[0088] A flip view is an image obtained by a mobile terminal capturing a target object in a flipped state. Different flip views can have different shooting angles. As shown in Figure 4, the mobile terminal captures the target object from different shooting angles, resulting in multiple flip views.

[0089] Obtaining a flipped image set includes the following steps: reading a pre-stored flipped image set on the target device, wherein the target device reads the flipped image set from the mobile terminal and saves it locally. It is understood that other methods may also be used to obtain the flipped image set in this embodiment.

[0090] S54: Register the flipped image set to the reference image model so that the flipped 3D description information of the target flipped view is mapped to the reference image model.

[0091] In this step, the reference image model also carries flipped 3D description information for each flipped view. That is, the reference image model is configured not only with upright 3D description information for each frontal view, but also with flipped 3D description information for each flipped view. The flipped 3D description information is used to describe the posture and shape of the target object in the flipped state when captured by the mobile terminal. The flipped 3D description information includes flipped posture information and flipped feature information. The flipped posture information represents the posture of the target object in the flipped state when captured by the mobile terminal, and the flipped feature information represents the characteristics of the target object in the flipped state, including shape features or color features, etc.

[0092] Registration refers to mapping the flipped 3D description information of the flipped image set to the coordinate system of the reference image model. Before registration, the coordinate system of the reference image model has been mapped with the upright 3D description information of the upright view. After the flipped image set is registered to the reference image model, the coordinate system of the reference image model will contain the upright 3D description information of the mapped upright view and the flipped 3D description information of the mapped target flipped view.

[0093] S55: Based on the preset 3D imaging algorithm, the upright 3D description information and the flipped 3D description information of the reference image model are fused to obtain the 3D model of the target object.

[0094] In this step, the preset three-dimensional imaging algorithm includes the NeRF algorithm or the 3D-GS algorithm. In this embodiment of the application, the NeRF algorithm or the 3D-GS algorithm can be used to fuse the upright three-dimensional description information and the flipped three-dimensional description information of the reference image model to obtain the three-dimensional model of the target object.

[0095] This application embodiment does not require specialized hardware equipment to collect the object's 3D information. It only requires acquiring the target object's frontal and flipped views via a mobile terminal, enabling electronic devices to perform 3D reconstruction. This expands the application scope of this application embodiment. Furthermore, this application embodiment can accurately reconstruct the object's 3D model, and the 3D reconstruction process is relatively simple, eliminating the need to construct multiple sets of 3D point clouds, thus improving the efficiency of 3D reconstruction.

[0096] The embodiments of this application can not only map the orientation information of each orientation view to the reference image module, but also map the orientation feature information of each orientation view to the reference image module.

[0097] Specifically, the process of processing the upright image set according to the preset 3D reconstruction algorithm to obtain the reference image model includes the following steps:

[0098] S521: Process multiple upright views according to a preset 3D reconstruction algorithm to obtain a reference image model, which carries the upright posture information of each upright view.

[0099] S522: Determine the elevation feature information for each elevation view.

[0100] S523: Update the reference image model based on each upright feature information to obtain the baseline image model.

[0101] In S521, this embodiment of the application extracts the upright posture information of each upright view captured by the mobile terminal according to the SFM algorithm, and maps the upright posture information of each upright view onto the same coordinate system to obtain a reference image model. Please refer to Figure 6, in the reference image model 60, each triangle 61 is used to represent the upright posture information of the corresponding upright view captured by the mobile terminal.

[0102] In S522, in some embodiments, determining the elevation feature information of each elevation view may include the following steps: performing comprehensive feature extraction on the elevation view to obtain the elevation feature information of each elevation view.

[0103] In other embodiments, determining the elevation feature information of each elevation view may include the following steps: extracting the elevation foreground image corresponding to the target object in each elevation view, performing feature extraction processing on the elevation foreground image, and obtaining the elevation feature information of each elevation view.

[0104] Please refer to Figure 7. The upright foreground image 71 is a partial area image corresponding to the target object in the upright view. When the mobile terminal captures the target object, it can capture not only the target object but also its background. Therefore, the upright view includes not only the upright foreground image but also the upright background image.

[0105] During the process of a user circling and photographing a target object, the same feature point of the target object corresponds to different background feature points. If the background feature points are included in the construction of the reference image model, it can easily interfere with the 3D reconstruction of the target object. In this embodiment, before constructing the reference image model, the foreground image and background image of the upright view are decoupled. Invalid background images are removed, and the foreground image is retained for constructing the reference image model. This avoids interference from the shape or color of other objects in the background image, thus improving the accuracy and reliability of the reference image model.

[0106] In S523, this embodiment of the application binds the orthogonal feature information and orthogonal posture information of each orthogonal view to update the reference image model, thereby obtaining an updated reference image model, wherein the updated reference image model is a base image model. Referring to FIG8, this embodiment of the application updates the orthogonal feature information of each orthogonal view to the reference image model 60 based on the reference image model 60, thereby obtaining the base image model 80 as shown in FIG8, wherein each triangle 81 in the base image model 80 is used to represent the orthogonal posture information of the corresponding orthogonal view captured by the mobile terminal and the orthogonal feature information of the target object under the shooting angle. This embodiment of the application constructs a base image model about the orthogonal view with the orthogonal view as the center. Since the orthogonal view contains a lot of shape information about the target object, the base image model can also maximize the configuration of the orthogonal three-dimensional description information of the orthogonal view.

[0107] This application embodiment can obtain not only high-quality upright views, but also high-quality flipped views. Specifically, obtaining a set of flipped images may include the following steps:

[0108] S531: Obtain a reference flip view, which is an image of the target object in a flipped state captured by the mobile terminal from the target shooting perspective.

[0109] S532: Detect whether the reference flipped view meets the preset image registration conditions.

[0110] S533: If satisfied, the reference flipped view is placed into the flipped image set as the target flipped view.

[0111] S534: If not satisfied, generate reshoot information. The reshoot information is used to prompt the user to reshoot the target object in the flipped state.

[0112] In S531, obtaining the reference flipped view includes the following steps: controlling the mobile terminal to display flipped shooting guidance information, which prompts the user to operate the mobile terminal to shoot the target object in a flipped state from the target shooting perspective; responding to the user's shooting operation input to the mobile terminal, controlling the mobile terminal's camera to shoot the target object in a flipped state from the target shooting perspective, thereby obtaining the reference flipped view. Under the guidance of the flipped shooting guidance information, this embodiment of the application can systematically acquire high-quality reference flipped views, avoiding invalid reference flipped views acquired due to user confusion.

[0113] Please refer to Figure 4. The flipped view provided in Figure 4 is an image of a target object in a flipped state captured by a mobile terminal from different shooting angles. Therefore, the flipped view provided in Figure 4 is a reference flipped view.

[0114] In some embodiments, the reference flip view includes an image of the invisible area of ​​the target object in its upright position. The invisible area image is an image of a portion of the target object that the mobile terminal cannot capture when the target object is in its upright position. As shown in Figure 2, the bottom of the bottle is invisible in its upright position; therefore, the image corresponding to the bottom is an image of the invisible area.

[0115] This application embodiment can capture the invisible areas of a target object in its upright position into a reference flipped view, thus ensuring that the image of the invisible area can also be registered in the base image model. The final output 3D model of the target object in this application embodiment can not only restore the front and side information of the target object, but also restore the invisible areas of the target object in its upright position, thereby providing users with a more complete 3D model of the target object.

[0116] In S532, the preset image registration conditions are used to detect whether a reference flipped view meets the requirements for registration with the baseline image model. Detecting whether a reference flipped view meets the preset image registration conditions may include the following steps: determining the flipped feature information of the reference flipped view; among the upright feature information of multiple upright views, detecting whether there is upright feature information with a matching degree greater than or equal to a preset matching threshold; if so, determining that the reference flipped view meets the preset image registration conditions; otherwise, determining that the reference flipped view does not meet the preset image registration conditions. The preset matching threshold is customized by the designer based on engineering experience, for example, a preset matching threshold of 80%. This embodiment of the application, by combining the upright feature information of the upright view, can quickly and effectively determine whether a reference flipped view is similar to an upright view, ensuring that the found target flipped view can be effectively registered with the baseline image model and avoiding registration failure.

[0117] In some embodiments, determining the flip feature information of a reference flipped view may include the following steps: performing comprehensive feature extraction on the reference flipped view to obtain the flip feature information of each reference flipped view.

[0118] In other embodiments, determining the flip feature information of a reference flip view may include the following steps: extracting a flip foreground image corresponding to the target object in each reference flip view, performing feature extraction processing on the flip foreground image, and obtaining the flip feature information of each reference flip view.

[0119] In S533, the reference flipped view meets the preset image registration conditions. In this embodiment of the application, the reference flipped view is placed into the flipped image set as the target flipped view.

[0120] In S534, if the reference flipped view does not meet the preset image registration conditions, this embodiment generates reshooting information. After the user views the reshooting information, they reshoot the target object in the flipped state to ensure that a high-quality target flipped view can be obtained.

[0121] In this embodiment, each reference flipped view needs to be checked to see if it meets the preset image registration conditions. This avoids registering low-quality or invalid reference flipped views as target flipped views to the base image model, thereby improving the accuracy of the three-dimensional model of the target object.

[0122] Determining the flip feature information of a reference flip view may include the following steps: extracting a flip foreground image corresponding to the target object from the reference flip view, performing feature extraction processing on the flip foreground image, and obtaining the flip feature information of each reference flip view.

[0123] Please refer to Figure 9. The flipped foreground image 91 is a local region image corresponding to the target object in the flipped view. As mentioned above, the flipped view includes not only the flipped foreground image but also the flipped background image. In this embodiment, before updating the reference image model, the flipped foreground image and the flipped background image of the flipped view are decoupled. Invalid flipped background images are discarded, and the flipped foreground image is retained to update the reference image model. This avoids interference from the shape or color of other objects in the flipped background image to the registration of the flipped view to the reference image model, which is beneficial to improving the accuracy and reliability of the 3D model.

[0124] In some embodiments, after placing the reference flipped view as the target flipped view into the flipped image set, the object 3D reconstruction method further includes the following steps: determining the number of target flipped views contained in the flipped image set; if the number is less than a preset number threshold, generating a continue shooting message to prompt the user to continue shooting the target object in the flipped state from different shooting angles; if the number is equal to the preset number threshold, generating a flipping shooting end message. The preset number threshold is customized by the designer based on engineering experience, for example, a preset number threshold of 5.

[0125] For example, in this embodiment, after each target flipped view is added to the flipped image set, the number of target flipped views in the flipped image set is counted to see if it is 5. If it is, this embodiment notifies the mobile terminal to stop shooting and registers all target flipped views in the flipped image set to the reference image model. Then, according to a preset 3D imaging algorithm, the upright 3D description information and the flipped 3D description information of the reference image model are fused to obtain the 3D model of the target object. If not, this embodiment notifies the mobile terminal to display a continue shooting message so that the user can continue shooting the target object in the flipped state from different shooting angles. By adopting this method, this embodiment can obtain a sufficient number of high-quality target flipped views, greatly enriching the view information about various flipping angles of the target object contained in the flipped image set, increasing the proportion of high-quality target flipped views registered to the reference image model, and facilitating the generation of a more complete and high-quality 3D model.

[0126] Registering a set of flipped images to a reference image model so that the flipped 3D description information of the target flipped view is mapped to the reference image model may include the following steps: finding a matching upright view among at least one upright view as the target upright view; determining the flipped posture information of the target flipped view based on the upright posture information of the target upright view; obtaining the flipped feature information of the target flipped view; and registering the flipped posture information and flipped feature information of the target flipped view to the reference image model. In this embodiment, an image matching operation is used to find a target upright view similar to the target flipped view. The upright posture information of the target upright view has already been obtained during the construction phase of the reference image model. This embodiment derives the flipped posture information of the target flipped view based on the known upright posture information, eliminating the need to calculate the flipped posture information of each target flipped view from multiple target flipped views. This improves the efficiency of registering the target flipped view to the reference image model.

[0127] For example, upright image set Flip the image set ,in, For the i-th frontal view, For the j-th target, flip the view; if the (i+1)-th upright view... The matching degree with the j-th target flipped view is 90%, and the (i+2)-th upright view is... The matching degree with the j-th target flipped view is 95%, and the (i+3)-th upright view is... If the matching degree with the j-th target flipped view is 75%, then in this embodiment of the application, the (i+2)-th upright view is selected. This is the frontal view of the target. Since the flipping motion of the target object is a rigid body flipping motion, this embodiment uses the (i+2)th frontal view. By combining the upright posture information with the SFM algorithm, the flip posture information of the target flipped view can be determined.

[0128] Please refer to Figure 10. In this embodiment of the application, after knowing the upright posture information of the target's upright view, the flip posture information of the target's flipped view can be determined by combining the SFM algorithm. As shown in Figure 10, each triangle 101 is used to represent the flip posture information of the target object in a flipped state captured by the mobile terminal.

[0129] Finally, the embodiment of this application will include the (i+2)th frontal view. The flipping posture information and flipping feature information are registered into the baseline image model.

[0130] Please refer to Figure 11. After registering the flipped image set to the reference image model, an updated reference image model is obtained. As shown in Figure 11, each triangle 112 in the updated reference image model 111 is used to represent the flipping posture information of the corresponding flipped view captured by the mobile terminal and the flipping feature information of the target object under the shooting angle. Thus, in this embodiment, it is not necessary to convert multiple target flipped views into 3D point clouds. It is only necessary to register the two-dimensional target flipped views into the three-dimensional reference image model. In this embodiment, the upright three-dimensional description information and the flipped three-dimensional description information of the reference image model are fused according to a preset three-dimensional imaging algorithm to obtain the three-dimensional model of the target object.

[0131] Please refer to Figure 12. In this embodiment of the application, a preset three-dimensional imaging algorithm is used to fuse the upright three-dimensional description information and the flipped three-dimensional description information of the reference image model to obtain the three-dimensional model 120 of the target object 20.

[0132] It should be noted that in the above embodiments, there is no necessarily a certain order between the steps. Those skilled in the art can understand from the description of the embodiments of this application that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0133] As another aspect of the embodiments of this application, this application provides a three-dimensional object reconstruction device. The three-dimensional object reconstruction device can be a software module, which includes several instructions stored in a memory. A processor can access the memory, call the instructions, and execute them to complete the three-dimensional object reconstruction methods described in the various embodiments above.

[0134] In some embodiments, the object 3D reconstruction device can also be constructed from hardware devices. For example, the object 3D reconstruction device can be constructed from one or more chips, and the chips can work together to complete the object 3D reconstruction method described in the various embodiments above. As another example, the object 3D reconstruction device can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0135] Please refer to Figure 13. The object 3D reconstruction device 130 includes an upright image acquisition module 131, an upright image processing module 132, a flipped image acquisition module 133, a flipped image registration module 134, and a 3D model reconstruction module 135.

[0136] The upright image acquisition module 131 is used to acquire an upright image set, which includes at least one upright view of the target object captured by the mobile terminal from different shooting angles. The upright image processing module 132 is used to process the upright image set according to a preset 3D reconstruction algorithm to obtain a reference image model, which carries the upright 3D description information of each upright view. The flipped image acquisition module 133 is used to acquire a flipped image set, which includes at least one flipped view of the target object captured by the mobile terminal from different shooting angles. The flipped image registration module 134 is used to register the flipped image set to the reference image model so that the flipped 3D description information of the target flipped view is mapped to the reference image model. The 3D model reconstruction module 135 is used to fuse the upright 3D description information and the flipped 3D description information of the reference image model according to a preset 3D imaging algorithm to obtain a 3D model of the target object.

[0137] This application embodiment does not require specialized hardware equipment to collect the object's 3D information. It only requires acquiring the target object's frontal and flipped views via a mobile terminal, enabling electronic devices to perform 3D reconstruction. This expands the application scope of this application embodiment. Furthermore, this application embodiment can accurately reconstruct the object's 3D model, and the 3D reconstruction process is relatively simple, eliminating the need to construct multiple sets of 3D point clouds, thus improving the efficiency of 3D reconstruction.

[0138] In some embodiments, the upright 3D description information includes upright posture information and upright feature information. The upright image processing module 132 is specifically used to: process multiple upright views according to a preset 3D reconstruction algorithm to obtain a reference image model, the reference image model carrying the upright posture information of each upright view, determine the upright feature information of each upright view, update the reference image model according to each upright feature information, and obtain a baseline image model.

[0139] In some embodiments, the upright image processing module 132 is further configured to: extract the upright foreground image corresponding to the target object in each upright view, perform feature extraction processing on the upright foreground image, and obtain the upright feature information of each upright view.

[0140] In some embodiments, the flip image acquisition module 133 is specifically used to: acquire a reference flip view, which is an image of a target object in a flipped state captured by the mobile terminal from the target shooting perspective; detect whether the reference flip view meets the preset image registration conditions; if it does, the reference flip view is placed into the flip image set as the target flip view; if it does not meet the conditions, reshooting information is generated to prompt the user to reshoot the target object in a flipped state.

[0141] In some embodiments, the upright three-dimensional description information includes upright feature information, and the flipped image acquisition module 133 is further specifically used to: determine the flipped feature information of the reference flipped view, and among the upright feature information of multiple upright views, detect whether there is upright feature information with a matching degree greater than or equal to a preset matching threshold. If there is, determine that the reference flipped view meets the preset image registration conditions; if there is no such feature information, determine that the reference flipped view does not meet the preset image registration conditions.

[0142] In some embodiments, the flip image acquisition module 133 is further specifically used to: extract the flip foreground image corresponding to the target object in the reference flip view, perform feature extraction processing on the flip foreground image, and obtain the flip feature information of each reference flip view.

[0143] In some embodiments, the flip image acquisition module 133 is further specifically used to: control the mobile terminal to present flip shooting guidance information, the flip shooting guidance information being used to prompt the user to operate the mobile terminal to shoot the target object in the flipped state from the target shooting perspective, respond to the user's shooting operation input to the mobile terminal, control the camera of the mobile terminal to shoot the target object in the flipped state from the target shooting perspective, and obtain a reference flipped view.

[0144] In some embodiments, the reference flip view includes an image of the invisible area of ​​the target object in an upright position.

[0145] In some embodiments, after the reference flipped view is placed into the flipped image set as the target flipped view, the flipped image acquisition module 133 is further specifically used to: determine the number of target flipped views contained in the flipped image set; if the number is less than a preset number threshold, generate continue shooting information; the continue shooting information is used to prompt the user to continue shooting the target object in the flipped state from different shooting angles.

[0146] In some embodiments, the upright 3D description information includes upright posture information, and the flipped 3D description information includes flipped posture information and flipped feature information. The flipped image registration module 134 is specifically used to: find an upright view that matches the target flipped view in at least one upright view as the target upright view, determine the flipped posture information of the target flipped view based on the upright posture information of the target upright view, obtain the flipped feature information of the target flipped view, and register the flipped posture information and flipped feature information of the target flipped view to the reference image model.

[0147] In some embodiments, the upright image acquisition module 131 is specifically used to: control the mobile terminal to present upright shooting guidance information, the upright shooting guidance information being used to prompt the user to operate the mobile terminal to shoot around the target object in an upright state, respond to the user's shooting operation input to the mobile terminal, control the camera of the mobile terminal to shoot around the target object in an upright state, and obtain an upright image set.

[0148] It should be noted that the above-described object 3D reconstruction device can execute the object 3D reconstruction method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the embodiments of the object 3D reconstruction device can be found in the object 3D reconstruction method provided in the embodiments of this application.

[0149] Please refer to Figure 14, which is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 140 includes one or more processors 141 and a memory 142. The memory 142 is connected to one or more processors 141, for example, via a bus.

[0150] Processor 141 is configured to support the electronic device in performing the corresponding functions in the methods described in the above method embodiments. The processor may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0151] Memory 142 is used to store program code, etc. Memory may include volatile memory (VM), such as random access memory (RAM); memory may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory may also include combinations of the above types of memory.

[0152] The memory 142 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the object 3D reconstruction method in the embodiments of this application. The processor executes the various functional applications and data processing of the object 3D reconstruction method and object 3D reconstruction device by running the non-volatile software programs, instructions, and modules stored in the memory, that is, it realizes the functions of each module or unit of the object 3D reconstruction method and object 3D reconstruction device provided in the above method embodiments.

[0153] The memory may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the object 3D reconstruction device. In some embodiments, the memory may include memory remotely located relative to the processor, which can be connected to the object 3D reconstruction device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0154] The one or more modules are stored in the memory. When executed by the one or more processors, they perform the object 3D reconstruction method in any of the above method embodiments. For example, they perform the method steps described in the above method embodiments to realize the functions of the modules described in the above device embodiments.

[0155] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by an electronic device, cause the electronic device to perform the method described in the foregoing embodiments.

[0156] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0157] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for three-dimensional reconstruction of an object, applied to an electronic device, the electronic device being used for communication connection with a mobile terminal, characterized in that, The method includes: Acquire an upright image set, the upright image set including at least one upright view of the target object captured by the mobile terminal from different shooting angles; The upright image set is processed according to a preset three-dimensional reconstruction algorithm to obtain a reference image model, which carries the upright three-dimensional description information of each of the upright views; Acquire a set of flipped images, the set of flipped images including at least one flipped view of the target object obtained by the mobile terminal from different shooting angles; Register the flipped image set to the reference image model so that the flipped 3D description information of the target flipped view is mapped to the reference image model; The three-dimensional model of the target object is obtained by fusing the upright three-dimensional description information and the flipped three-dimensional description information of the reference image model according to the preset three-dimensional imaging algorithm.

2. The method according to claim 1, characterized in that, The upright 3D description information includes upright posture information and upright feature information. The step of processing the upright image set according to a preset 3D reconstruction algorithm to obtain a baseline image model includes: Multiple upright views are processed according to a preset 3D reconstruction algorithm to obtain a reference image model, which carries the upright posture information of each upright view; Determine the elevation feature information for each of the elevation views; The reference image model is updated based on each of the upright feature information to obtain the baseline image model.

3. The method according to claim 2, characterized in that, The determination of the elevation feature information for each of the elevation views includes: Extract the foreground image corresponding to the target object from each of the upright views; The foreground image is subjected to feature extraction processing to obtain the elevation feature information of each elevation view.

4. The method according to claim 1, characterized in that, The acquisition of the flipped image set includes: Obtain a reference flipped view, which is an image of a target object in a flipped state captured by the mobile terminal from the target shooting perspective; Detect whether the reference flipped view meets the preset image registration conditions; If the conditions are met, the reference flipped view is placed into the flipped image set as the target flipped view; If the conditions are not met, a reshooting message is generated, which prompts the user to reshoot the target object in the flipped state.

5. The method according to claim 4, characterized in that, The upright 3D description information includes upright feature information, and the detection of whether the reference flipped view meets the preset image registration conditions includes: Determine the flip feature information of the reference flipped view; Among the upright feature information of multiple upright views, detect whether there is upright feature information whose matching degree with the flipped feature information is greater than or equal to a preset matching threshold; If it exists, then the reference flipped view is determined to meet the preset image registration conditions; If it does not exist, then it is determined that the reference flip view does not meet the preset image registration conditions.

6. The method according to claim 5, characterized in that, The determination of the flip feature information of the reference flipped view includes: Extract the flipped foreground image corresponding to the target object from the reference flipped view; Feature extraction processing is performed on the flipped foreground image to obtain the flipped feature information of each reference flipped view.

7. The method according to claim 4, characterized in that, The process of obtaining the reference flipped view includes: The mobile terminal is controlled to display flip shooting guidance information, which is used to prompt the user to operate the mobile terminal to shoot the target object in a flipped state from the target shooting perspective. In response to a user's shooting operation input on the mobile terminal, the camera of the mobile terminal is controlled to shoot the target object in a flipped state from the target shooting angle to obtain a reference flipped view.

8. The method according to claim 4, characterized in that, The reference flip view includes an image of the invisible area of ​​the target object in an upright position.

9. The method according to claim 4, characterized in that, After placing the reference flipped view as the target flipped view into the flipped image set, the method further includes: Determine the number of target flipped views contained in the flipped image set; If the number is less than a preset threshold, a "continue shooting" message is generated to prompt the user to continue shooting the target object in a flipped state from different shooting angles.

10. The method according to any one of claims 1 to 9, characterized in that, The upright 3D description information includes upright posture information, and the flipped 3D description information includes flipped posture information and flipped feature information. Registering the flipped image set to the reference image model, so that the flipped 3D description information of the target flipped view is mapped to the reference image model, includes: In at least one of the upright views, the upright view that matches the target flipped view is identified as the target upright view; The flipping posture information of the target flipped view is determined based on the upright posture information of the target upright view; Obtain the flipping feature information of the target flipped view; The flipping posture information and flipping feature information of the target flipped view are registered to the reference image model.

11. The method according to any one of claims 1 to 9, characterized in that, The acquisition of the upright image set includes: The mobile terminal is controlled to display upright shooting guidance information, which prompts the user to operate the mobile terminal to shoot around a target object that is upright. In response to a user's input of a shooting operation on the mobile terminal, the camera of the mobile terminal is controlled to surround the target object in an upright position to take pictures and obtain an upright image set.

12. An electronic device, characterized in that, The device includes a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, the processor causing the electronic device to implement the object three-dimensional reconstruction method as described in any one of claims 1-11 when executing the one or more computer programs.

13. A three-dimensional object reconstruction system, characterized in that, It includes a mobile terminal and an electronic device as described in claim 12, wherein the electronic device is communicatively connected to the mobile terminal.

14. The three-dimensional object reconstruction system according to claim 13, characterized in that, The mobile terminal is equipped with a data acquisition application, which is configured to locate a target object, generate a view frame at the location of the target object to encompass the target object, and display upright shooting guidance information on the application interface of the data acquisition application. The upright shooting guidance information is used to prompt the user to operate the mobile terminal to shoot around the target object which is in an upright state.

15. The three-dimensional object reconstruction system according to claim 14, characterized in that, The electronic device is configured to, upon receiving multiple upright views sent by the mobile terminal, construct a reference image model based on the upright views and send view response information to the data acquisition application of the mobile terminal. The view response information is used to instruct the mobile terminal to end the acquisition of upright views and proceed to the operation of acquiring flipped views.

16. The three-dimensional object reconstruction system according to claim 15, characterized in that, The data acquisition application is also configured to respond to view reply information, presenting object flipping operation information on the application interface. The object flipping operation information is used to prompt the user to flip the target object in order to show the bottom of the target object.

17. The three-dimensional object reconstruction system according to claim 16, characterized in that, The data acquisition application is also configured to present flip shooting guidance information on the application interface, which is used to prompt the user to operate the mobile terminal to shoot the target object in a flipped state from the target shooting perspective.

18. The three-dimensional object reconstruction system according to claim 17, characterized in that, The electronic device is also configured to, upon receiving multiple flipped views sent by the mobile terminal, update the reference image model according to the flipped views, and generate a three-dimensional model of the target object based on the updated reference image model.

19. The three-dimensional object reconstruction system according to claim 18, characterized in that, The data acquisition application is configured to display an end-of-shoot message or an end-of-flip message on the application interface when it detects that the number of upright views or flipped views has reached a preset number.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the object three-dimensional reconstruction method as described in any one of claims 1-11.

Images