Three-dimensional reconstruction method and device of target and electronic equipment

Abstract

The invention discloses a three-dimensional reconstruction method and device of a target and electronic equipment, and belongs to the field of simulation. The method comprises the following steps: in response to a received loading instruction, obtaining parameter information of a target three-dimensional ellipsoid in an original three-dimensional Gaussian model corresponding to a specified target; rendering the original three-dimensional Gaussian model based on the parameter information to obtain an intermediate three-dimensional Gaussian model, and displaying the intermediate three-dimensional Gaussian model; in response to an editing operation of a user on a specified three-dimensional ellipsoid in the middle three-dimensional Gaussian model, displaying an edited three-dimensional Gaussian model, and obtaining a three-dimensional reconstruction result of the specified target based on the edited three-dimensional Gaussian model; wherein the editing operation is used for editing at least one of the following parameters of the specified three-dimensional ellipsoid: position, shape, size, proportion and transparency. According to the invention, the refinement degree of the target three-dimensional reconstruction result can be effectively improved.

Description

Technical Field

[0001] The present application relates to the fields of simulation technology and unmanned driving technology, and in particular, to a three-dimensional reconstruction method, device and electronic equipment for a target. Background Art

[0002] With the widespread adoption of autonomous driving technology, people are paying more and more attention to its safety. To ensure the safety of autonomous driving, extensive testing is required. However, if all tests are conducted in real-world conditions, not only will the diversity of tests not be guaranteed, but the testing costs will also increase significantly. Therefore, simulation testing is an essential part of autonomous driving. To ensure the accuracy of simulation testing, the three-dimensional models of target objects, such as vehicles, must be accurate. However, the 3D model construction methods used in related technologies, especially those for highly complex targets, often suffer from varying degrees of "burrs" around the perimeter, resulting in poor precision. Summary of the Invention

[0003] In order to overcome the deficiencies of the related art, the present application provides a method, device and electronic device for three-dimensional reconstruction of a target to solve the problems in the related art.

[0004] The technical solution adopted by this application to solve its technical problems is:

[0005] In a first aspect, a method for three-dimensional reconstruction of an object is provided, comprising:

[0006] In response to receiving the loading instruction, obtaining parameter information of the target three-dimensional ellipsoid in the original three-dimensional Gaussian model corresponding to the specified target;

[0007] Rendering the original three-dimensional Gaussian model based on the parameter information to obtain an intermediate three-dimensional Gaussian model, and displaying the intermediate three-dimensional Gaussian model;

[0008] In response to a user's editing operation on a designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model, and obtaining a three-dimensional reconstruction result of the designated target based on the edited three-dimensional Gaussian model;

[0009] The editing operation is used to edit at least one of the following parameters of the specified three-dimensional ellipsoid: position, shape, size, scale, and transparency.

[0010] Furthermore, the method further comprises:

[0011] Displaying an initial three-dimensional ellipsoid of a designated area in the intermediate three-dimensional Gaussian model in a first preset manner, wherein the designated area includes at least one of the following: a gap space, airflow light and shadow;

[0012] The editing operation is received to determine the designated three-dimensional ellipsoid from the initial three-dimensional ellipsoid.

[0013] Furthermore, the method further comprises:

[0014] The three-dimensional ellipsoid of other areas in the intermediate three-dimensional Gaussian model except the designated area is displayed in a second preset manner, wherein the first preset manner is a more eye-catching display manner than the second preset manner.

[0015] Furthermore, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model includes:

[0016] In response to the user's initial screening operation, the three-dimensional ellipsoid of the first area is eliminated; wherein the first area is the area determined in the intermediate three-dimensional Gaussian model based on the initial screening operation.

[0017] Furthermore, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model includes:

[0018] In response to a user's selection operation, determining candidate three-dimensional ellipsoids in the second area, wherein the candidate three-dimensional ellipsoids are all three-dimensional ellipsoids or part of the three-dimensional ellipsoids in the second area;

[0019] The candidate three-dimensional ellipsoid is displayed with specified attributes, where the specified attributes include at least one of the following: a specified color, a specified transparency, and a specified scaling ratio. The second area is an area determined in the intermediate three-dimensional Gaussian model based on the box selection operation.

[0020] Furthermore, determining the candidate three-dimensional ellipsoid in the second area includes:

[0021] Determine a candidate three-dimensional ellipsoid in the second region based on at least one of the following rules:

[0022] The size of the three-dimensional ellipsoid is smaller than a preset size;

[0023] The transparency of the three-dimensional ellipsoid is less than a preset transparency;

[0024] The color difference between adjacent three-dimensional ellipsoids is greater than a preset difference.

[0025] Furthermore, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model includes:

[0026] In response to a reverse selection operation by the user, display of the first three-dimensional ellipsoid is canceled, where the first three-dimensional ellipsoid is the three-dimensional ellipsoid determined in the intermediate three-dimensional Gaussian model based on the reverse selection operation.

[0027] Furthermore, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model includes:

[0028] In response to the user's modification selection operation, the attributes of the second three-dimensional ellipsoid are modified to the preset attributes corresponding to the modification selection operation and displayed; the second three-dimensional ellipsoid is a three-dimensional ellipsoid determined in the intermediate three-dimensional Gaussian model based on the modification selection operation.

[0029] In a second aspect, a device for three-dimensional reconstruction of a target is provided, comprising:

[0030] A model loading module, configured to obtain parameter information of a target three-dimensional ellipsoid in an original three-dimensional Gaussian model corresponding to a specified target in response to receiving a loading instruction;

[0031] a model rendering module, configured to render the original three-dimensional Gaussian model based on the parameter information to obtain an intermediate three-dimensional Gaussian model, and display the intermediate three-dimensional Gaussian model;

[0032] a model editing module, configured to, in response to a user's editing operation on a designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, display the edited three-dimensional Gaussian model, and obtain a three-dimensional reconstruction result of the designated target based on the edited three-dimensional Gaussian model;

[0033] The editing operation is used to edit at least one of the following parameters of the specified three-dimensional ellipsoid: position, shape, size, scale, and transparency.

[0034] In a third aspect, a computer-readable storage medium is provided, on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the steps of the three-dimensional reconstruction method of the target provided by the technical solution of the first aspect are implemented.

[0035] In a fourth aspect, an electronic device is provided, including:

[0036] at least one processor and at least one memory;

[0037] The memory stores executable instructions of the processor;

[0038] The processor is configured to execute the three-dimensional reconstruction method of the target provided by the technical solution of the first aspect.

[0039] Beneficial effects:

[0040] The technical solution of the present application provides a method, device and electronic device for three-dimensional reconstruction of a target. The three-dimensional reconstruction method includes obtaining parameter information of a target three-dimensional ellipsoid in an original three-dimensional Gaussian model corresponding to a specified target in response to receiving a loading instruction; rendering the original three-dimensional Gaussian model based on the parameter information to obtain an intermediate three-dimensional Gaussian model, and displaying the intermediate three-dimensional Gaussian model; in response to a user's editing operation on a specified three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model, and obtaining a three-dimensional reconstruction result of the specified target based on the edited three-dimensional Gaussian model, wherein the editing operation is used to edit at least one of the following parameters of the specified three-dimensional ellipsoid: position, shape, size, scale, and transparency. The solution of the present application constructs a three-dimensional Gaussian model of an index target, and then, based on the user's editing operation on a specific minimum visual unit, realizes precise operation on the minimum visual unit in the three-dimensional Gaussian model. This not only makes the processing process more convenient, but also effectively improves the degree of refinement of the target three-dimensional reconstruction result. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 This is a flow chart of a method for three-dimensional reconstruction of a target provided in an embodiment of the present application;

[0043] Figure 2 This is a flow chart of a method for reconstructing an original three-dimensional Gaussian model provided in an embodiment of the present application;

[0044] Figure 3 This is a flow chart of an optional target three-dimensional reconstruction method provided in an embodiment of the present application;

[0045] Figure 4 This is a schematic diagram of the results of a preliminary screening operation provided in an embodiment of the present application;

[0046] Figure 5 This is a schematic diagram of the result of performing a frame selection operation provided in an embodiment of the present application;

[0047] Figure 6 This is a schematic diagram of the results of performing a frame selection operation, a reverse selection operation, and a modification selection operation provided in an embodiment of the present application;

[0048] Figure 7 is a schematic diagram of a final three-dimensional reconstruction result provided by an embodiment of the present application;

[0049] Figure 8 It is a structural schematic diagram of a target three-dimensional reconstruction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application are described in detail below with reference to the accompanying drawings and examples. Obviously, the described embodiments are only some of the embodiments of this application, rather than all of them. Based on the embodiments in this application, all other implementation methods obtained by ordinary technicians in this field without making any creative work are within the scope of protection of this application.

[0051] Reference Figure 1 , an embodiment of the present application provides a method for three-dimensional reconstruction of a target, comprising:

[0052] S11: In response to receiving the loading instruction, obtaining parameter information of the target three-dimensional ellipsoid in the original three-dimensional Gaussian model corresponding to the specified target; optionally, the original three-dimensional Gaussian model can be composed of multiple three-dimensional ellipsoids of arbitrary shape, transparency and color.

[0053] It should be noted that the original 3D Gaussian model obtained by conventional 3D reconstruction includes not only the target 3D ellipsoid but also irrelevant 3D ellipsoids that need to be eliminated. The parameter information of these 3D ellipsoids is obtained when the original 3D Gaussian model is loaded. Therefore, upon receiving the load instruction, the parameter information of the 3D ellipsoids within the original 3D Gaussian model is obtained simultaneously with the loading of the original 3D Gaussian model. These parameters include position, shape, size, scale, transparency, and color.

[0054] In one embodiment, the loading instruction is triggered manually by the user, that is, the user clicks a load button or option on the interface, or selects a load command on the file of the original 3D Gaussian model.

[0055] In another embodiment, the loading instruction is triggered automatically after a specified event occurs. For example, the loading instruction is automatically triggered after the user determines that the original three-dimensional Gaussian model is obtained during the three-dimensional reconstruction.

[0056] As a preferred implementation method of the embodiment of this application, Figure 2 As shown, a reconstruction method of the original three-dimensional Gaussian model is provided, and the steps are as follows:

[0057] S21: Acquire a two-dimensional image and image information containing a specified target, wherein the image information includes a shooting position and a shooting angle when the two-dimensional image is taken; illustratively, obtain photos of the specified target (such as a vehicle) from multiple perspectives or obtain a 360° real-time video of the vehicle.

[0058] S22: Generate a three-dimensional point cloud based on the two-dimensional image and image information, and obtain the position and color information of each point in the three-dimensional point cloud; that is, according to the shooting position and shooting angle, the position and color of the points constituting the specified target under one perspective can be determined. In this way, by combining the points determined from multiple perspectives, the three-dimensional point cloud of the specified target and the position and color information of each point can be obtained.

[0059] S23: Place a three-dimensional ellipsoid at the position of each point, calculate the spherical harmonic coefficients of the three-dimensional ellipsoid based on the color information of the point, and set initial parameters to obtain an initial three-dimensional Gaussian model. The initial parameters include transparency and covariance matrix; wherein the spherical harmonic coefficients are a set of numerical values used to represent color, specifically representing different colors and textures at different positions on the surface of an object. Transparency indicates the degree of light transmittance, with a value of 0-1, where 1 means that light cannot pass through the object at all. The covariance matrix is used to represent the shape, size, and direction of each three-dimensional Gaussian, and is a 3*3 matrix.

[0060] S24: Rendering the initial three-dimensional Gaussian model based on the spherical harmonic coefficients and the initial parameters, and calculating the difference between the first pixel at each position in each two-dimensional image and the second pixel at the corresponding position in the initial three-dimensional Gaussian model; after rendering the initial three-dimensional Gaussian model, a three-dimensional Gaussian model is obtained, but it is generally significantly different from the actual designated target, so training is required. This embodiment uses a method of comparing the pixels at the same position of the rendered initial three-dimensional Gaussian model with the actual two-dimensional image, and then adjusting the transparency and covariance matrix based on the comparison results.

[0061] S25: Calculate the average difference of all two-dimensional images. In the actual training process, even if the transparency and covariance matrix are adjusted multiple times, it cannot be guaranteed that the pixel difference at each position meets the requirements. Therefore, the average difference of all two-dimensional images is generally used.

[0062] S26: When the average difference is greater than a threshold, adjusting the initial parameters of the initial three-dimensional Gaussian model until the average difference is less than or equal to the threshold; when the average difference is less than or equal to the threshold, determining that the current three-dimensional Gaussian model is the original three-dimensional Gaussian model.

[0063] As a preferred implementation method of the embodiment of this application:

[0064] When the average difference is greater than a threshold, obtaining a single-view average difference at each viewing angle, wherein the single-view average difference refers to: an average value of pixel differences between all positions of the two-dimensional image and all positions of the initial three-dimensional Gaussian model at each viewing angle;

[0065] Determine the viewing angle with the largest single-view average difference as the target viewing angle;

[0066] After adjusting the initial parameters of the initial three-dimensional Gaussian model, firstly calculating the target single-view average difference between the adjusted initial three-dimensional Gaussian model and the corresponding two-dimensional image at the target viewing angle;

[0067] After the target single-view average difference is less than the preset difference, the average difference under all viewing angles is calculated. When the target single-view average difference is greater than or equal to the preset difference, the initial parameters are directly adjusted again without calculating the average difference under all viewing angles, which greatly reduces the amount of calculation.

[0068] S12: Rendering the original three-dimensional Gaussian model based on the parameter information to obtain an intermediate three-dimensional Gaussian model, and displaying the intermediate three-dimensional Gaussian model.

[0069] As a preferred implementation of this embodiment of the present application, an initial three-dimensional ellipsoid representing a designated area within the intermediate three-dimensional Gaussian model is displayed in a first preset manner, wherein the designated area includes at least one of the following: interstitial space, airflow, and light and shadow; wherein the designated area is used to represent areas that are difficult for a user to directly observe. Even after rendering, it is difficult to directly distinguish whether the three-dimensional ellipsoid belongs to the designated target.

[0070] The editing operation is received to determine the designated three-dimensional ellipsoid from the initial three-dimensional ellipsoid.

[0071] The rendered intermediate three-dimensional Gaussian model can be intuitively displayed to the user, making it convenient for the user to perform editing operations and accurately remove irrelevant three-dimensional ellipsoids.

[0072] In some other embodiments, the following is further included:

[0073] The three-dimensional ellipsoids of the areas other than the designated area in the intermediate three-dimensional Gaussian model are displayed in a second preset manner, wherein the first preset manner is a more prominent display manner than the second preset manner. This allows the user to more intuitively distinguish different areas when eliminating irrelevant three-dimensional ellipsoids, and to more closely examine designated areas that are difficult to visually observe, or to perform specific editing operations to more accurately eliminate the three-dimensional ellipsoids.

[0074] S13: In response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model, and obtaining a three-dimensional reconstruction result of the designated target based on the edited three-dimensional Gaussian model;

[0075] The editing operation is used to edit at least one of the following parameters of the specified three-dimensional ellipsoid: position, shape, size, scale, and transparency.

[0076] In one embodiment, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model includes:

[0077] In response to the user's initial screening operation, the three-dimensional ellipsoid of the first area is eliminated; wherein the first area is the area determined in the intermediate three-dimensional Gaussian model based on the initial screening operation.

[0078] The application scenario of this embodiment is: the three-dimensional ellipsoids in a certain area of the intermediate three-dimensional Gaussian model do not belong to the specified target. In this case, the user does not need to accurately select the three-dimensional ellipsoids for removal one by one, but can remove the entire area. Figure 4 , the designated target is a vehicle, and the black area on the left is not a vehicle. Therefore, a straight line or curve can be drawn in the middle three-dimensional Gaussian model according to the coordinate axis to directly eliminate the area on the left. It should be noted that in this application, the first area can be formed by demarcating a straight line or curve and a boundary, or a closed curve or a closed area composed of multiple straight lines can be directly used as the first area.

[0079] In the above embodiment, the initial screening operation can achieve good accuracy for a simple intermediate three-dimensional Gaussian model (such as a designated target is a cube or a sphere), but for Figure 4 The complex three-dimensional Gaussian model shown takes a long time to eliminate according to the initial screening operation, and the final effect is not good.

[0080] Therefore, in another embodiment, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model includes:

[0081] In response to a user's selection operation, determining candidate three-dimensional ellipsoids in the second area, wherein the candidate three-dimensional ellipsoids are all three-dimensional ellipsoids or part of the three-dimensional ellipsoids in the second area;

[0082] The candidate three-dimensional ellipsoid is displayed with specified attributes, where the specified attributes include at least one of the following: a specified color, a specified transparency, and a specified scaling ratio. The second area is an area determined in the intermediate three-dimensional Gaussian model based on the box selection operation.

[0083] The application scenario of this embodiment is: there is a 3D ellipsoid of a specified target that needs to be retained in a certain area of the intermediate 3D Gaussian model, but it may also contain 3D ellipsoids that need to be removed. The user cannot judge whether it needs to be deleted based on the current rendering result. Therefore, in order to make it easier for the user to intuitively distinguish the 3D ellipsoids in the area, after the box selection operation, Figure 5 As shown, the candidate three-dimensional ellipsoid in the second region has the specified attributes ( Figure 5The three-dimensional ellipsoids in the middle are changed from their original colors to the specified colors (red / green) for display, so that users can intuitively see whether these three-dimensional ellipsoids need to be eliminated.

[0084] It should be noted that, as an optional implementation of this embodiment, the candidate three-dimensional ellipsoids are all three-dimensional ellipsoids in the second region. That is, after the user performs a selection operation, all three-dimensional ellipsoids in the second region are displayed with the specified attributes. The reason for this design is that the original attributes of different three-dimensional ellipsoids in the second region may be different, which affects the user's judgment. After the display is displayed with specified attributes, the user can make a more intuitive judgment. For example, the second region is displayed in a specified color, which makes it easier to compare with the three-dimensional ellipsoids outside the second region to determine whether certain three-dimensional ellipsoids need to be eliminated. For example, some three-dimensional ellipsoids in the second region are too small for the user to see directly. After the selection operation is performed, the second region is enlarged at the specified magnification ratio, and the user can see the smaller three-dimensional ellipsoids in the second region and determine whether they need to be eliminated.

[0085] In the above implementation, since the candidate 3D ellipsoids are all 3D ellipsoids in the second region, after they are displayed with the specified attributes, the user still needs to compare each 3D ellipsoid to determine which 3D ellipsoids need to be removed, which is relatively time-consuming and laborious. Therefore, as an alternative implementation of this embodiment, the candidate 3D ellipsoids are selected from a subset of the 3D ellipsoids in the second region. After the user selects them, the 3D ellipsoids in the second region that may be removed are directly displayed with the specified attributes. The user only needs to determine whether these candidate 3D ellipsoids need to be deleted. In this way, the user does not need to review the remaining 3D ellipsoids, greatly reducing the user's workload.

[0086] As a preferred implementation of the embodiment of the present application, determining the candidate three-dimensional ellipsoid in the second area includes:

[0087] Determine a candidate three-dimensional ellipsoid in the second region based on at least one of the following rules:

[0088] The size of the three-dimensional ellipsoid is smaller than the preset size because the three-dimensional ellipsoid smaller than the preset size is not easily visually seen by the user when displayed.

[0089] The transparency of the three-dimensional ellipsoid is less than the preset transparency; because the smaller the transparency of the three-dimensional ellipsoid is, the more likely the user is to be affected by the three-dimensional ellipsoid behind it when observing.

[0090] The color difference between adjacent three-dimensional ellipsoids is greater than the preset difference. In practice, the target three-dimensional ellipsoid and the non-target three-dimensional ellipsoid have large color differences, so adjacent three-dimensional ellipsoids with large color differences are also used as candidate three-dimensional ellipsoids.

[0091] In another embodiment, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model comprises:

[0092] In response to a reverse selection operation by the user, display of the first three-dimensional ellipsoid is canceled, where the first three-dimensional ellipsoid is the three-dimensional ellipsoid determined in the intermediate three-dimensional Gaussian model based on the reverse selection operation.

[0093] It should be noted that the reference Figure 6 Canceling the display of the first 3D ellipsoid means that the first 3D ellipsoid is not displayed on the interface. However, the first 3D ellipsoid will still appear in the final model, which means that after canceling the display, the display must be restored.

[0094] This embodiment is applicable to scenarios where a certain 3D ellipsoid (i.e., the first 3D ellipsoid) is determined to be the target 3D ellipsoid and therefore cannot be removed. However, due to angle issues, there may be other 3D ellipsoids behind the first 3D ellipsoid. These 3D ellipsoids cannot be removed due to occlusion by the first 3D ellipsoid (e.g., the first 3D ellipsoid is too large or too transparent). Therefore, the display of the first 3D ellipsoid needs to be canceled to allow the user to determine whether the 3D ellipsoids behind the first 3D ellipsoid are the target 3D ellipsoid.

[0095] As one implementation of this embodiment, the display of the first three-dimensional ellipsoid is canceled by setting its transparency to 0. This prevents the first three-dimensional ellipsoid from visually disappearing, allowing the three-dimensional ellipsoid behind it to be visible. After determining whether the three-dimensional ellipsoid behind it should be removed, the display of the first three-dimensional ellipsoid is restored. It should be noted that since this implementation only sets the transparency to 0, the three-dimensional ellipsoid still exists. Therefore, when restoring the transparency, only the transparency of the first three-dimensional ellipsoid needs to be restored.

[0096] As another implementation of this embodiment, the display of the first three-dimensional ellipsoid is canceled by setting its size to 0. After determining whether the rear three-dimensional ellipsoid needs to be removed, the display of the first three-dimensional ellipsoid is restored. In this implementation, since setting the size to 0 is equivalent to removal, restoration requires not only its size attribute information but also its transparency, color, position, and other information.

[0097] In some embodiments, the operation of restoring the display of the first three-dimensional ellipsoid is: when performing a reverse selection operation, the attribute information of the first three-dimensional ellipsoid is stored locally, and when restoration is needed, the attribute information of the first three-dimensional ellipsoid stored locally is called, and the attribute information of the first three-dimensional ellipsoid is changed to the locally stored attribute information.

[0098] In other embodiments, the operation for restoring the display of the first three-dimensional ellipsoid includes, when restoration is required, obtaining original attribute information of the first three-dimensional ellipsoid from the original three-dimensional Gaussian model, and re-rendering the first three-dimensional ellipsoid based on the original attribute information. The method of this embodiment does not require additional local storage space, thus saving storage costs, and does not require changing the attribute information of the first three-dimensional ellipsoid, as the rendering is directly re-rendered.

[0099] As a preferred implementation of the present application, when performing a reverse selection operation, the size of the remaining space in the local storage space and the size of the occupied space of the first three-dimensional ellipsoid are obtained. When the remaining space is greater than the size of the occupied space, the attribute information of the first three-dimensional ellipsoid is stored locally. When restoration is required, the locally stored attribute information of the first three-dimensional ellipsoid is retrieved and replaced with the locally stored attribute information. When the remaining space is less than or equal to the size of the occupied space, the attribute information of the first three-dimensional ellipsoid is not stored locally. When restoration is required, the original attribute information of the first three-dimensional ellipsoid is retrieved from the original three-dimensional Gaussian model and the first three-dimensional ellipsoid is re-rendered based on the original attribute information. Because local storage increases storage costs, it is faster when restoring the display of the first three-dimensional ellipsoid. While obtaining attribute information from the original three-dimensional ellipsoid model does not increase storage costs, it requires loading and is slower. In practice, a certain amount of storage space is generally set locally. Therefore, this embodiment combines the remaining local storage space and the occupied space of the first three-dimensional ellipsoid to further determine which method to use to cancel or restore display.

[0100] In another embodiment, in response to the user editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model comprises:

[0101] In response to the user's modification selection operation, the attributes of the second three-dimensional ellipsoid are modified to the preset attributes corresponding to the modification selection operation and displayed; the second three-dimensional ellipsoid is a three-dimensional ellipsoid determined in the intermediate three-dimensional Gaussian model based on the modification selection operation.

[0102] The application scenario of this embodiment is that when the second three-dimensional ellipsoid is displayed with its original attributes, it is difficult to determine whether it is the three-dimensional ellipsoid belonging to the specified target. Therefore, by changing the attributes of the second three-dimensional ellipsoid, it is convenient for the user to determine whether it is the three-dimensional ellipsoid belonging to the specified target and whether it needs to be deleted.

[0103] It should be noted that the editing operation in this application can adopt one of the above embodiments and combine multiple operations to obtain the final reconstruction result. Figure 7 .

[0104] After obtaining the final reconstruction result, the method further includes outputting the target three-dimensional Gaussian model in response to the received export instruction.

[0105] In one embodiment, when outputting the target three-dimensional Gaussian model, the original format of the original three-dimensional Gaussian model is obtained, and the target three-dimensional Gaussian model is output in the original format.

[0106] In another embodiment, when an export instruction is received, an output format determination interface is displayed, wherein the format determination interface is used for the user to input or select an output format;

[0107] In response to the output format determined by the user in the output format determination interface, the target three-dimensional Gaussian model is output in the determined output format.

[0108] As an optional implementation of the present application, this embodiment provides an optional three-dimensional reconstruction method of a target, such as Figure 3 As shown, this embodiment is based on the original three-dimensional Gaussian model and is divided into three stages: loading and parsing, editing, and result exporting.

[0109] During the loading and parsing phase, all 3D ellipsoid information contained in the original 3D Gaussian model is read. Each ellipsoid primarily includes its position, scale, transparency, and a set of spherical harmonic function parameters. After obtaining this information, it is managed by category and its current state is maintained. This state information is then used to render the 3D scene, providing a visual representation of the current state of the 3D Gaussian model.

[0110] The editing phase is completed through a combination of 3D scene interaction and display effects. In terms of 3D scene interaction, it supports browsing, picking, inverting and removing 3D ellipsoids; in terms of display effects, after picking a group of 3D ellipsoids in a local area, the color, scale, transparency, etc. of the selected 3D ellipsoids can be modified as needed, and the final result is as follows: Figure 7 The model shown is reflected in the rendering results in real time through the parameter management module, which can help users intuitively and accurately identify which ellipsoids need to be removed from a complex set of ellipsoids, achieving a controllable editing result. In addition, the editing process involves repeated screening and identification, and this embodiment also supports convenient repetition of this operation.

[0111] For example, Figure 4 As shown, the user performs a preliminary screening operation, selects the first area to be eliminated (the black area on the left), and eliminates the three-dimensional ellipsoid of the first area.

[0112] like Figure 5 As shown, the user performs a box selection operation, and the three-dimensional ellipsoid in the second area is displayed with the specified attributes. The user can manually delete all ellipsoids in the area. The specified attributes can include a specified color, color, and default size.

[0113] like Figure 6 As shown, the user performs a reverse selection operation to cancel the first three-dimensional ellipsoid that does not need to be deleted (i.e., not delete it); and then restores it. It should be noted that Figure 6 The red 3D ellipsoid in the middle is enlarged or reshaped. This effect is the result of a user modifying the selection.

[0114] The result export, based on the elimination results in the editing stage, organizes and outputs the retained three-dimensional ellipsoid in the standard format of the original data to ensure its continued universal use.

[0115] Tools based on this solution have been applied to modeling various heavy trucks and operating equipment used in unmanned mining scenarios. These large-scale machines are complex and detailed, requiring meticulous editing of their Gaussian models. This tool accurately identifies and removes common details in these models, such as various 3D ellipsoids corresponding to gaps or airflow shadows, especially those located in confined spaces or partially embedded in enclosed areas. This ultimately yields a complete and detailed 3D Gaussian model.

[0116] The above embodiment constructs a three-dimensional Gaussian model of the indicator target, and then realizes precise operation of the minimum visualization unit in the three-dimensional Gaussian model based on the user's editing operation on the specific minimum visualization unit. This not only makes the processing process more convenient, but also effectively improves the degree of refinement of the target three-dimensional reconstruction result.

[0117] Based on the same inventive concept, an embodiment of the present application provides a 3D reconstruction device 80 for a target, comprising:

[0118] The model loading module 81 is configured to, in response to receiving a loading instruction, obtain parameter information of a target three-dimensional ellipsoid in an original three-dimensional Gaussian model corresponding to a designated target.

[0119] The model rendering module 82 is configured to render the original three-dimensional Gaussian model based on the parameter information to obtain an intermediate three-dimensional Gaussian model, and display the intermediate three-dimensional Gaussian model.

[0120] Also includes:

[0121] Displaying an initial three-dimensional ellipsoid of a designated area in the intermediate three-dimensional Gaussian model in a first preset manner, wherein the designated area includes at least one of the following: a gap space, airflow light and shadow;

[0122] The editing operation is received to determine the designated three-dimensional ellipsoid from the initial three-dimensional ellipsoid.

[0123] Also includes:

[0124] The three-dimensional ellipsoid of other areas in the intermediate three-dimensional Gaussian model except the designated area is displayed in a second preset manner, wherein the first preset manner is a more eye-catching display manner than the second preset manner.

[0125] a model editing module 83 for displaying the edited three-dimensional Gaussian model in response to a user's editing operation on a designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, and obtaining a three-dimensional reconstruction result of the designated target based on the edited three-dimensional Gaussian model;

[0126] The editing operation is used to edit at least one of the following parameters of the specified three-dimensional ellipsoid: position, shape, size, scale, and transparency.

[0127] In one embodiment, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model includes:

[0128] In response to the user's initial screening operation, the three-dimensional ellipsoid of the first area is eliminated; wherein the first area is the area determined in the intermediate three-dimensional Gaussian model based on the initial screening operation.

[0129] In another embodiment, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model comprises:

[0130] In response to a user's selection operation, determining candidate three-dimensional ellipsoids in the second area, wherein the candidate three-dimensional ellipsoids are all three-dimensional ellipsoids or part of the three-dimensional ellipsoids in the second area;

[0131] The candidate three-dimensional ellipsoid is displayed with specified attributes, where the specified attributes include at least one of the following: a specified color, a specified transparency, and a specified scaling ratio. The second area is an area determined in the intermediate three-dimensional Gaussian model based on the box selection operation.

[0132] The determining of the candidate three-dimensional ellipsoid in the second area includes:

[0133] Determine a candidate three-dimensional ellipsoid in the second region based on at least one of the following rules:

[0134] The size of the three-dimensional ellipsoid is smaller than a preset size;

[0135] The transparency of the three-dimensional ellipsoid is less than a preset transparency;

[0136] The color difference between adjacent three-dimensional ellipsoids is greater than a preset difference.

[0137] In another embodiment, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model comprises:

[0138] In response to a reverse selection operation by the user, display of the first three-dimensional ellipsoid is canceled, where the first three-dimensional ellipsoid is the three-dimensional ellipsoid determined in the intermediate three-dimensional Gaussian model based on the reverse selection operation.

[0139] In another embodiment, in response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model comprises:

[0140] In response to the user's modification selection operation, the attributes of the second three-dimensional ellipsoid are modified to the preset attributes corresponding to the modification selection operation and displayed; the second three-dimensional ellipsoid is a three-dimensional ellipsoid determined in the intermediate three-dimensional Gaussian model based on the modification selection operation.

[0141] Based on the same inventive concept, the present application provides a computer-readable storage medium having a computer program or instructions stored thereon. When the computer program or instructions are executed by a processor, the steps of the three-dimensional reconstruction method of the target provided by any of the above embodiments are implemented.

[0142] Based on the same inventive concept, the present application provides an electronic device, including:

[0143] at least one processor and at least one memory;

[0144] The memory stores executable instructions of the processor;

[0145] The processor is configured to execute the three-dimensional reconstruction method of the target provided by any of the above embodiments.

[0146] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0147] It should be noted that, in the description of this application, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0148] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0149] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0150] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0151] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0152] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0153] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0154] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for three-dimensional reconstruction of an object, characterized in that: include: In response to receiving the loading instruction, obtaining parameter information of the target three-dimensional ellipsoid in the original three-dimensional Gaussian model corresponding to the specified target; Rendering the original three-dimensional Gaussian model based on the parameter information to obtain an intermediate three-dimensional Gaussian model, and displaying the intermediate three-dimensional Gaussian model; In response to a user's editing operation on a designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model, and obtaining a three-dimensional reconstruction result of the designated target based on the edited three-dimensional Gaussian model; The editing operation is used to edit at least one of the following parameters of the specified three-dimensional ellipsoid: position, shape, size, scale, and transparency.

2. The method according to claim 1, characterized in that The method further comprises: Displaying an initial three-dimensional ellipsoid of a designated area in the intermediate three-dimensional Gaussian model in a first preset manner, wherein the designated area includes at least one of the following: a gap space, airflow light and shadow; The editing operation is received to determine the designated three-dimensional ellipsoid from the initial three-dimensional ellipsoid.

3. The method according to claim 2, characterized in that The method further comprises: The three-dimensional ellipsoid of other areas in the intermediate three-dimensional Gaussian model except the designated area is displayed in a second preset manner, wherein the first preset manner is a more eye-catching display manner than the second preset manner.

4. The method according to claim 1, wherein In response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model comprises: In response to the user's initial screening operation, the three-dimensional ellipsoid of the first area is eliminated; wherein the first area is the area determined in the intermediate three-dimensional Gaussian model based on the initial screening operation.

5. The method according to claim 1, wherein In response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model comprises: In response to a user's selection operation, determining candidate three-dimensional ellipsoids in the second area, wherein the candidate three-dimensional ellipsoids are all three-dimensional ellipsoids or part of the three-dimensional ellipsoids in the second area; The candidate three-dimensional ellipsoid is displayed with specified attributes, where the specified attributes include at least one of the following: a specified color, a specified transparency, and a specified scaling ratio. The second area is an area determined in the intermediate three-dimensional Gaussian model based on the box selection operation.

6. The method according to claim 5, characterized in that Determining the candidate three-dimensional ellipsoid in the second area includes: Determine a candidate three-dimensional ellipsoid in the second region based on at least one of the following rules: The size of the three-dimensional ellipsoid is smaller than a preset size; The transparency of the three-dimensional ellipsoid is less than a preset transparency; The color difference between adjacent three-dimensional ellipsoids is greater than a preset difference.

7. The method according to claim 1, characterized in that In response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model comprises: In response to a reverse selection operation by the user, display of the first three-dimensional ellipsoid is canceled, where the first three-dimensional ellipsoid is the three-dimensional ellipsoid determined in the intermediate three-dimensional Gaussian model based on the reverse selection operation.

8. The method according to claim 1, characterized in that In response to the user's editing operation on the designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, displaying the edited three-dimensional Gaussian model comprises: In response to the user's modification selection operation, the attributes of the second three-dimensional ellipsoid are modified to the preset attributes corresponding to the modification selection operation and displayed; the second three-dimensional ellipsoid is a three-dimensional ellipsoid determined in the intermediate three-dimensional Gaussian model based on the modification selection operation.

9. A three-dimensional reconstruction device for an object, characterized in that: include: A model loading module, configured to obtain parameter information of a target three-dimensional ellipsoid in an original three-dimensional Gaussian model corresponding to a specified target in response to receiving a loading instruction; a model rendering module, configured to render the original three-dimensional Gaussian model based on the parameter information to obtain an intermediate three-dimensional Gaussian model, and display the intermediate three-dimensional Gaussian model; a model editing module, configured to, in response to a user's editing operation on a designated three-dimensional ellipsoid in the intermediate three-dimensional Gaussian model, display the edited three-dimensional Gaussian model, and obtain a three-dimensional reconstruction result of the designated target based on the edited three-dimensional Gaussian model; The editing operation is used to edit at least one of the following parameters of the specified three-dimensional ellipsoid: position, shape, size, scale, and transparency.

10. An electronic device, characterized in that: include: at least one processor and at least one memory; The memory stores executable instructions of the processor; The processor is configured to execute the method according to any one of claims 1 to 8.

Images