Product mounting and dismounting state detection method, device and equipment based on three-dimensional Gaussian sputtering

Abstract

The application provides a product assembly and disassembly state detection method, device and equipment based on three-dimensional Gaussian sputtering, which comprises the following steps: converting product part models in a detection scene into triangular facet models; splitting the triangular facet models in units of each part to obtain split facet models with the length of each triangular facet being less than a preset length; outputting the vertexes of each triangular facet in the split facet models as initial point clouds for three-dimensional Gaussian sputtering; arranging virtual-real fusion targets at corresponding positions in virtual models and real scenes; scanning around a product area to be detected to obtain image information and pose information containing the virtual-real fusion targets; performing three-dimensional scene reconstruction by using three-dimensional Gaussian sputtering based on the image information, the pose information and the initial point clouds to obtain a product scene model to be detected; and determining the assembly and disassembly state of the product to be detected as an installed state or an uninstalled state, so that the accuracy of product assembly and disassembly state detection is effectively ensured.

Description

Technical Field

[0001] This invention relates to the field of manufacturing and assembly technology, and in particular to a method, apparatus and equipment for detecting the assembly and disassembly status of products based on three-dimensional Gaussian sputtering. Background Technology

[0002] In the development of complex products such as spacecraft, it is necessary to repeatedly confirm the assembly and disassembly status of components. In addition to manual inspection methods, machine vision methods are now widely used to achieve pattern recognition of product assembly and disassembly status through training with a large number of samples.

[0003] However, when the training samples are few and the surrounding conditions of the target object are uncertain, the accuracy of machine vision is relatively low. Summary of the Invention

[0004] This invention provides a product assembly / disassembly status detection method, apparatus, and equipment based on three-dimensional Gaussian sputtering, to address the shortcomings of existing machine vision technologies that rely heavily on training samples.

[0005] In a first aspect, the present invention provides a product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering, comprising:

[0006] Convert the product component model in the inspection scenario into a triangular patch model;

[0007] The triangular facet model is split into individual parts to obtain a split facet model in which the side length of each triangular facet is smaller than the preset side length.

[0008] Output the vertices of each triangular facet in the split facet model as an initial point cloud for 3D Gaussian sputtering;

[0009] Targets for virtual-real fusion are placed at corresponding locations in the virtual model and the real scene;

[0010] The area around the product under test is scanned to obtain image information and pose information containing the virtual-real fusion target.

[0011] Based on the image information, pose information, and initial point cloud, a three-dimensional scene is reconstructed using three-dimensional Gaussian sputtering to obtain a scene model of the product under test.

[0012] Based on the scenario model of the product under test, the installation and disassembly status of the product under test is determined to be either installed or not installed.

[0013] According to the present invention, a product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering is provided, wherein determining the assembly / disassembly status of the product under test as an installed or uninstalled state based on the scene model of the product under test includes:

[0014] Construct the envelope of the product under test in the real scene, and obtain all three-dimensional Gaussian sputtering bodies in the envelope of the corresponding position in the scene model of the product under test;

[0015] Determine the effective viewing direction for each of the three-dimensional Gaussian sputtering volumes;

[0016] Based on product transparency and each valid observation direction, the installation / disassembly status of the product under test is determined to be either installed or not installed.

[0017] According to the present invention, a product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering is provided, wherein determining the effective observation direction of each of the three-dimensional Gaussian sputtering bodies includes:

[0018] With the geometric center of the envelope as the center of the sphere, multiple observation points facing the center of the sphere are set on the virtual sphere;

[0019] The three-dimensional Gaussian sputtered volume is rendered to the direction of the observation viewpoint to determine the effective observation direction.

[0020] According to the present invention, a product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering is provided, wherein determining the effective observation direction includes:

[0021] Determine the percentage of the envelope covered by the three-dimensional Gaussian sputtering body;

[0022] When the percentage exceeds the preset percentage, the corresponding observation direction is determined as the valid observation direction.

[0023] According to the present invention, a method for detecting the assembly / disassembly status of a product based on three-dimensional Gaussian sputtering is provided, wherein determining the assembly / disassembly status of the product under test as an installed or uninstalled state based on product transparency and each effective observation direction includes:

[0024] Determine the average pixel transparency of the effective rendering region in each of the effective viewing directions;

[0025] Determine the average pixel transparency corresponding to all the effective viewing directions;

[0026] The installation or disassembly status of the product under test is determined based on the magnitude of the average value.

[0027] According to the present invention, a method for detecting the assembly / disassembly status of a product based on three-dimensional Gaussian sputtering is provided, wherein determining the assembly / disassembly status of the product under test as an installed or uninstalled state based on the magnitude of the average value includes:

[0028] When the average value is greater than the first preset threshold, the installation and disassembly status of the product under test is determined to be the installed state;

[0029] When the average value is less than the second preset threshold, the product under test is determined to be in an uninstalled state.

[0030] Wherein, the first preset threshold is greater than the second preset threshold.

[0031] The product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering provided by the present invention further includes:

[0032] When the average value is greater than the second preset threshold and less than the first preset threshold, the confidence level for determining that the product is in an installed state is as follows:

[0033]

[0034] Where D represents the confidence level, K represents the mean, and δ min Indicates the second preset threshold, δ max This indicates the first preset threshold.

[0035] According to the present invention, a product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering is provided, wherein the method involves splitting the triangular facet model on a per-component basis to obtain a split facet model in which the side lengths of the triangular facets are all smaller than a preset side length, comprising:

[0036] The shape of each component is determined to obtain regular rectangular surfaces and irregular rectangular surfaces;

[0037] The regular rectangular surface is split into two parts, with half the length of the rectangle's diagonal being less than a preset side length.

[0038] For the irregular rectangular face, the midpoint of the longest side of the triangular facet that is longer than the preset side length is connected to the opposite side, and the triangular facet is split until all side lengths are less than the preset side length.

[0039] Secondly, the present invention also provides a product assembly / disassembly status detection device based on three-dimensional Gaussian sputtering, comprising:

[0040] The conversion module is used to convert product component models in the inspection scene into triangular patch models;

[0041] The splitting module is used to split the triangular facet model into units of each component, resulting in a split facet model in which the side length of each triangular facet is smaller than a preset side length.

[0042] The output module is used to output the vertices of each triangular facet in the split facet model as an initial point cloud for three-dimensional Gaussian sputtering.

[0043] The deployment module is used to deploy targets for virtual-real fusion at corresponding positions in the virtual model and the real scene.

[0044] The scanning module is used to scan around the area of ​​the product to be tested to obtain image information and pose information containing the virtual-real fusion target;

[0045] The reconstruction module is used to reconstruct a three-dimensional scene based on the image information, the pose information and the initial point cloud, using three-dimensional Gaussian sputtering to obtain a scene model of the product under test.

[0046] The determination module is used to determine, based on the scenario model of the product under test, whether the product under test is in an installed state or an uninstalled state.

[0047] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering as described above.

[0048] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering as described above.

[0049] Fifthly, the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering as described above.

[0050] This invention provides a method, apparatus, and equipment for detecting the assembly / disassembly status of a product based on 3D Gaussian sputtering. The method includes converting a product component model in the detection scene into a triangular facet model; splitting the triangular facet model for each component to obtain a split facet model where the side length of each triangular facet is smaller than a preset side length; outputting the vertices of each triangular facet in the split facet model as an initial point cloud for 3D Gaussian sputtering; placing virtual-real fusion targets at corresponding positions in the virtual model and the real scene; scanning around the area of ​​the product under test to obtain image information and pose information containing the virtual-real fusion targets; reconstructing a 3D scene using 3D Gaussian sputtering based on the image information, pose information, and initial point cloud to obtain a scene model of the product under test; and determining the assembly / disassembly status of the product under test as either installed or not installed based on the scene model. Because the 3D Gaussian sputtering method is used to reconstruct the 3D scene model, it is not affected by training samples, effectively ensuring the accuracy of the product assembly / disassembly status detection. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating the product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering provided in an embodiment of the present invention.

[0053] Figure 2 These are product component model diagrams provided in embodiments of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure of the triangular facet model provided in an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the split facet model provided in an embodiment of the present invention;

[0056] Figure 5 This is a schematic diagram of the initial point cloud provided in an embodiment of the present invention;

[0057] Figure 6 This is a schematic diagram of the target arrangement provided in an embodiment of the present invention;

[0058] Figure 7 This is a schematic diagram illustrating the principle of assembly / disassembly status recognition provided in an embodiment of the present invention;

[0059] Figure 8 The envelope viewpoint is the first observation viewpoint for observing the scene.

[0060] Figure 9 The envelope viewpoint is the observation screen from the second observation viewpoint.

[0061] Figure 10 The envelope viewpoint is the observation screen from the third observation viewpoint.

[0062] Figure 11 This is a schematic diagram of the product assembly / disassembly status detection device based on three-dimensional Gaussian sputtering provided in an embodiment of the present invention;

[0063] Figure 12 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0065] The following is combined with Figures 1-12 This invention describes a product assembly / disassembly status detection method, apparatus, and equipment based on three-dimensional Gaussian sputtering.

[0066] 3D Gaussian Sputtering (3DGS) is a 3D scene reconstruction and rendering technique that captures the geometric and visual details of a scene by decomposing it into a set of 3D Gaussian functions, each with attributes such as position, opacity, color, and shape. The 3DGS workflow typically begins with a sparse 3D point cloud, with each point cloud location subsequently modeled as a 3DGS volume. Through continuous optimization of the parameters of these 3DGS volumes during training, the 3DGS volumes can reconstruct scenes in a compact and efficient manner, supporting the synthesis of high-quality images from new perspectives, and are suitable for real-time rendering and large-scale scene reconstruction.

[0067] Figure 1 This is a flowchart illustrating the product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering provided in an embodiment of the present invention.

[0068] like Figure 1 As shown in the figure, the present invention provides a product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering, which mainly includes the following steps:

[0069] 101. Convert the product component model in the inspection scene into a triangular facet model.

[0070] In a specific implementation process, the product component model in the detection scenario is first converted into a triangular facet model. Figure 2 These are product component model diagrams provided in embodiments of the present invention. Figure 3 This is a structural schematic diagram of the triangular facet model provided in the embodiment of the present invention; the product component model is as follows: Figure 2 As shown, 21, 22, and 23 represent the product's components, namely the workbench, module, and fastener, respectively. The triangular facet model is shown below. Figure 3 As shown, the different faces of the components are divided into triangles.

[0071] 102. Split the triangular facet model into units of each component to obtain a split facet model in which the side length of each triangular facet is smaller than the preset side length.

[0072] Figure 4 This is a schematic diagram of the split facet model provided in an embodiment of the present invention. After converting to a triangular facet model, the model is split into units based on each component, that is, the triangular facet corresponding to each component is split. The splitting is based on ensuring that the side length of the split triangular facets is less than a preset side length, thereby ensuring that the density of the initial point cloud constructed in subsequent steps is uniform and that the distances between the point clouds are relatively close. The resulting split triangular facet model is shown below. Figure 4 As shown, through Figure 4 As can be seen, the function of splitting is to divide a large triangle into multiple smaller triangles with the largest side length being less than the preset side length.

[0073] 103. Output the vertices of each triangular facet in the split facet model as the initial point cloud for 3D Gaussian sputtering.

[0074] Figure 5 This is a schematic diagram of the initial point cloud provided in an embodiment of the present invention. After obtaining the split facet model, the vertices of different triangular facets are relatively close to each other, thus using the vertices of the triangular facet corresponding to each component as the initial point cloud for three-dimensional Gaussian sputtering, as shown below. Figure 5 As shown, since the side length of each triangular facet in the split facet model is smaller than the preset side length, the resulting initial point cloud density is more uniform. Furthermore, the density of the initial point cloud can be adjusted by changing the preset side length.

[0075] 104. Place targets for virtual-real fusion at corresponding positions in the virtual model and the real scene.

[0076] Figure 6 This is a schematic diagram of the target arrangement provided in an embodiment of the present invention, as shown below. Figure 6 As shown, one or more virtual-real fusion targets 61 are arranged in the virtual model and the real scene. The targets can be in the form of two-dimensional image targets. The main function of the targets is to ensure the stability and accuracy of subsequent scene tracking. They can also be understood as targets used as reference objects to ensure that they exist during subsequent scanning and shooting.

[0077] 105. Scan around the area of ​​the product under test to obtain image information and pose information including the target for virtual-real fusion.

[0078] Augmented reality (AR) devices are used to perform a surround scan of the product area under test. The AR devices utilize image and depth cameras for thorough scanning, ensuring the completeness of the reconstructed scene. By calibrating the target's pose in the actual scene and continuously tracking it, the credibility of the final scene model is further enhanced.

[0079] 106. Based on image information, pose information and initial point cloud, a 3D scene reconstruction is performed using 3D Gaussian sputtering to obtain a scene model of the product under test.

[0080] By combining the obtained initial point cloud, image information containing depth information, and camera pose information, a 3D scene is reconstructed using 3D Gaussian sputtering. If the obtained product scene model shows obvious discontinuities, it indicates that the initial point cloud density is low. Therefore, the preset side length is redefined, and the triangular facets are split again to increase the density of the initial point cloud until a continuous and complete product scene model is obtained.

[0081] 107. Based on the scenario model of the product under test, determine whether the product under test is installed or not.

[0082] After obtaining the scenario model of the product under test, the product's transparency characteristics can be combined to determine whether the product under test is in an installed or uninstalled state.

[0083] This embodiment of the product assembly / disassembly status detection method based on 3D Gaussian sputtering enables automatic detection of the assembly / disassembly status of product objects in small-batch, uncertain environments. It does not require the preparation of batches of prior real-world samples and omits the data training step. It can realize product status detection in small-batch, high-random-disturbance scenarios. It uses the product's 3D model as the basis for initializing point cloud generation using the 3D Gaussian sputtering method, which simplifies the 3D Gaussian sputtering application process and optimizes the continuity of the 3D reconstruction scene.

[0084] Figure 7 This is a schematic diagram illustrating the principle of assembly / disassembly status recognition provided in an embodiment of the present invention.

[0085] Furthermore, based on the above embodiments, the determination of the installation / disassembly state of the product under test as an installed or uninstalled state based on the scene model of the product under test in this embodiment includes: constructing the envelope of the product under test in the real scene, obtaining all three-dimensional Gaussian sputtering bodies in the envelope of the corresponding position in the scene model of the product under test; determining the effective viewing direction of each three-dimensional Gaussian sputtering body; and determining the installation / disassembly state of the product under test as an installed or uninstalled state based on the product transparency and each effective viewing direction.

[0086] Specifically, firstly, an envelope of the product under test is constructed in the scene. All 3D Gaussian sputtered volumes (3DGS volumes) within the envelope at the corresponding location in the 3D reconstructed scene are obtained. Then, 3DGS volumes with high uncertainty due to occlusion or other reasons are removed. Using the geometric center of the envelope as the center of a sphere, multiple viewpoints facing the center are set on a virtual sphere to ensure that all viewpoints can observe the complete envelope in that direction. The 3D Gaussian sputtered volumes are then rendered to the direction of the viewpoints to determine the effective observation directions. Firstly, the percentage of the envelope covered by the 3D Gaussian sputtered volumes is determined; when this percentage exceeds a preset percentage (e.g., 50%), the corresponding observation direction is determined as the effective observation direction. Then, the effective observation direction v is statistically determined. i The average pixel transparency K of the effective rendering area i Determine the average pixel transparency K corresponding to all valid observation directions; then determine the installation or removal status of the product under test based on the magnitude of the average value.

[0087] The determination of the installation / disassembly status of the product under test as either installed or not based on the magnitude of the average value includes: when the average value is greater than a first preset threshold δ max (i.e., K>δ) max When the average value is less than the second preset threshold δ, the installation status of the product under test is determined to be the installed state; when the average value is less than the second preset threshold δ, the installation status of the product under test is determined to be the installed state. min (i.e., K < δ) min When the product to be tested is determined to be in an uninstalled state, the first preset threshold is greater than the second preset threshold.

[0088] Figure 8 The envelope viewpoint is the first observation viewpoint. Figure 9 This is the envelope viewpoint for observing the scene from the second observation viewpoint. Figure 10 The envelope view is the viewpoint observed from the third observation point.

[0089] For example, such as Figure 7 As shown, 71 is the envelope of the product under test, 72 is the center of the virtual sphere and the geometric center of the product envelope, and 73 is the surface of the virtual sphere. Taking 74, 75, and 76 in the figure as observation points, we obtain the following results: Figure 8 , Figure 9 and Figure 10 Three envelope volume observation images. In this embodiment, viewpoints 74 and 76 show no occlusion exceeding 50%, and are therefore considered valid observation directions; in viewpoint 75, the bottom of the product under test is completely occluded by other products, and is therefore considered an invalid observation direction. For viewpoints 74 and 76, the average pixel transparency of the valid rendering area is calculated. In this embodiment, the generated valid 3DGS volume is simplified to... Figures 8-10 10 to 15 in the range. For perspective 74, i.e., statistics. Figure 7The average transparency K1 of pixels rendered at a viewpoint of 74 for five 3DGS volumes (10, 11, 12, 13, and 14); for a viewpoint of 75, i.e., statistical... Figure 7 The average transparency K2 of the pixels rendered at a viewing angle of 75 for five 3DGS volumes: 10, 11, 12, 13, and 15.

[0090] Then, calculate the average value K = (K1 + K2) / 2 for all valid observation directions. Finally, set the judgment thresholds δmin = 20 and δmax = 230 (pixel transparency range is [0, 255]). When K < 10, the product status is determined to be not installed; when K > 230, the product status is determined to be installed.

[0091] When the average value is greater than the second preset threshold and less than the first preset threshold, i.e., 20≤K≤230, the confidence level for determining that the product is in an installed state is determined, as shown in formula (1):

[0092]

[0093] Where D represents the confidence level, K represents the mean, and δ min Indicates the second preset threshold, δ max This indicates the first preset threshold.

[0094] Furthermore, based on the above embodiments, this embodiment splits the triangular facet model on a per-component basis, resulting in a split facet model where the side length of each triangular facet is less than a preset side length. Specifically, this is used to determine the shape of each component, resulting in regular rectangular faces and irregular rectangular faces. For regular rectangular faces, a rectangular mesh division method is used, splitting the facets such that half the length of the rectangle's diagonal is less than the preset side length, until half the length of the rectangle's diagonal is less than the preset side length, thus ensuring that the longest side of the split triangular facet is less than the preset side length. For irregular rectangular faces, the longest side is selected from the triangular facets with a side length greater than the preset side length. The midpoint of the longest side of the triangular facet longer than the preset side length is connected to the opposite side, splitting the triangular facet to obtain two new triangles, until all side lengths are less than the preset side length, thus ensuring that the final generated initial point cloud has a higher density.

[0095] Based on the same general inventive concept, this invention also protects a product assembly / disassembly status detection device based on three-dimensional Gaussian sputtering. The product assembly / disassembly status detection device based on three-dimensional Gaussian sputtering provided by this invention will be described below. The product assembly / disassembly status detection device based on three-dimensional Gaussian sputtering described below can be referred to in correspondence with the product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering described above.

[0096] Figure 11This is a schematic diagram of the product assembly / disassembly status detection device based on three-dimensional Gaussian sputtering provided in an embodiment of the present invention.

[0097] like Figure 11 As shown in the figure, this embodiment provides a product assembly / disassembly status detection device based on three-dimensional Gaussian sputtering, comprising:

[0098] The conversion module 1101 is used to convert the product component model in the inspection scene into a triangular patch model;

[0099] The splitting module 1102 is used to split the triangular facet model on a unit basis, so as to obtain a split facet model in which the side length of each triangular facet is smaller than the preset side length.

[0100] Output module 1103 is used to output the vertices of each triangular facet in the split facet model as an initial point cloud for three-dimensional Gaussian sputtering;

[0101] The deployment module 1104 is used to deploy targets for virtual-real fusion at corresponding positions in the virtual model and the real scene.

[0102] The scanning module 1105 is used to scan around the area of ​​the product under test to obtain image information and pose information containing the target for virtual-real fusion.

[0103] The reconstruction module 1106 is used to reconstruct a three-dimensional scene based on image information, pose information and initial point cloud using three-dimensional Gaussian sputtering to obtain a scene model of the product under test.

[0104] The determination module 1107 is used to determine whether the installation or disassembly status of the product under test is installed or not, based on the scenario model of the product under test.

[0105] Furthermore, the determining module 1107 in this embodiment is specifically used for:

[0106] Construct the envelope of the product under test in the real scene, and obtain all three-dimensional Gaussian sputtering bodies in the envelope of the corresponding position in the scene model of the product under test;

[0107] Determine the effective viewing direction for each of the three-dimensional Gaussian sputtering volumes;

[0108] Based on product transparency and each valid observation direction, the installation / disassembly status of the product under test is determined to be either installed or not installed.

[0109] Furthermore, the determining module 1107 in this embodiment is specifically used for:

[0110] With the geometric center of the envelope as the center of the sphere, multiple observation points facing the center of the sphere are set on the virtual sphere;

[0111] The three-dimensional Gaussian sputtered volume is rendered to the direction of the observation viewpoint to determine the effective observation direction.

[0112] Furthermore, the determining module 1107 in this embodiment is specifically used for:

[0113] Determine the percentage of the envelope covered by the three-dimensional Gaussian sputtering body;

[0114] When the percentage exceeds the preset percentage, the corresponding observation direction is determined as the valid observation direction.

[0115] Furthermore, the determining module 1107 in this embodiment is specifically used for:

[0116] Determine the average pixel transparency of the effective rendering region in each of the effective viewing directions;

[0117] Determine the average pixel transparency corresponding to all the effective viewing directions;

[0118] The installation or disassembly status of the product under test is determined based on the magnitude of the average value.

[0119] Furthermore, the determining module 1107 in this embodiment is specifically used for:

[0120] When the average value is greater than the first preset threshold, the installation and disassembly status of the product under test is determined to be the installed state;

[0121] When the average value is less than the second preset threshold, the product under test is determined to be in an uninstalled state.

[0122] Wherein, the first preset threshold is greater than the second preset threshold.

[0123] Furthermore, the determining module 1107 in this embodiment is specifically used for:

[0124] When the average value is greater than the second preset threshold and less than the first preset threshold, the confidence level for determining that the product is in an installed state is as follows:

[0125]

[0126] Where D represents the confidence level, K represents the mean, and δ min Indicates the second preset threshold, δ max This indicates the first preset threshold.

[0127] Furthermore, the splitting module 1102 in this embodiment is specifically used for:

[0128] The shape of each component is determined to obtain regular rectangular surfaces and irregular rectangular surfaces;

[0129] The regular rectangular surface is split into two parts, with half the length of the rectangle's diagonal being less than a preset side length.

[0130] For the irregular rectangular face, the midpoint of the longest side of the triangular facet that is longer than the preset side length is connected to the opposite side, and the triangular facet is split until all side lengths are less than the preset side length.

[0131] Figure 12 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention.

[0132] like Figure 12 As shown, the electronic device may include: a processor 1210, a communications interface 1220, a memory 1230, and a communications bus 1240, wherein the processor 1210, the communications interface 1220, and the memory 1230 communicate with each other through the communications bus 1240. The processor 1210 can call logic instructions in the memory 1230 to execute a product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering. This method includes: converting a product component model in the detection scene into a triangular facet model; splitting the triangular facet model into a split facet model where each triangular facet has a side length smaller than a preset side length; outputting the vertices of each triangular facet in the split facet model as an initial point cloud for three-dimensional Gaussian sputtering; arranging virtual-real fusion targets at corresponding positions in the virtual model and the real scene; scanning around the area of ​​the product under test to obtain image information and pose information containing the virtual-real fusion targets; reconstructing a three-dimensional scene using three-dimensional Gaussian sputtering based on the image information, the pose information, and the initial point cloud to obtain a scene model of the product under test; and determining the assembly / disassembly status of the product under test as either an installed or uninstalled state based on the scene model of the product under test.

[0133] Furthermore, the logical instructions in the aforementioned memory 1230 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0134] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering provided by the above methods. The method includes: converting a product component model in the detection scene into a triangular facet model; splitting the triangular facet model on a unit basis for each component to obtain a split facet model in which the side length of each triangular facet is smaller than a preset side length; outputting the vertices of each triangular facet in the split facet model as an initial point cloud for three-dimensional Gaussian sputtering; arranging virtual-real fusion targets at corresponding positions in the virtual model and the real scene; scanning around the area of ​​the product to be tested to obtain image information and pose information containing the virtual-real fusion targets; reconstructing a three-dimensional scene using three-dimensional Gaussian sputtering based on the image information, the pose information, and the initial point cloud to obtain a scene model of the product to be tested; and determining the assembly / disassembly status of the product to be tested as either an installed state or an uninstalled state based on the scene model of the product to be tested.

[0135] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the product assembly / disassembly state detection method based on three-dimensional Gaussian sputtering provided by the above methods. The method includes: converting a product component model in the detection scene into a triangular facet model; splitting the triangular facet model on a unit basis for each component to obtain a split facet model in which the side length of each triangular facet is smaller than a preset side length; outputting the vertices of each triangular facet in the split facet model as an initial point cloud for three-dimensional Gaussian sputtering; arranging virtual-real fusion targets at corresponding positions in the virtual model and the real scene; scanning around the area of ​​the product to be tested to obtain image information and pose information containing the virtual-real fusion targets; reconstructing a three-dimensional scene using three-dimensional Gaussian sputtering based on the image information, the pose information, and the initial point cloud to obtain a scene model of the product to be tested; and determining the assembly / disassembly state of the product to be tested as either an installed state or an uninstalled state based on the scene model of the product to be tested.

[0136] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting the assembly / disassembly status of products based on three-dimensional Gaussian sputtering, characterized in that, include: Convert the product component model in the inspection scenario into a triangular patch model; The triangular facet model is split into individual components to obtain a split facet model in which the side length of each triangular facet is smaller than the preset side length. Output the vertices of each triangular facet in the split facet model as an initial point cloud for 3D Gaussian sputtering; Targets for virtual-real fusion are placed at corresponding locations in the virtual model and the real scene; The area around the product under test is scanned to obtain image information and pose information containing the virtual-real fusion target. Based on the image information, pose information, and initial point cloud, a 3D scene is reconstructed using 3D Gaussian sputtering to obtain a scene model of the product under test; the envelope of the product under test is constructed in the real scene, and all 3D Gaussian sputtered bodies in the envelope of the corresponding position in the scene model of the product under test are obtained. Determine the effective viewing direction for each of the three-dimensional Gaussian sputtering volumes; Based on product transparency and each valid observation direction, the installation / disassembly status of the product under test is determined to be either installed or not installed.

2. The product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering according to claim 1, characterized in that, Determining the effective viewing direction for each of the three-dimensional Gaussian sputters includes: With the geometric center of the envelope as the center of the sphere, multiple observation points facing the center of the sphere are set on the virtual sphere; The three-dimensional Gaussian sputtered volume is rendered to the direction of the observation viewpoint to determine the effective observation direction.

3. The product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering according to claim 2, characterized in that, Determining the effective observation direction includes: Determine the percentage of the envelope covered by the three-dimensional Gaussian sputtering body; When the percentage exceeds the preset percentage, the corresponding observation direction is determined as the valid observation direction.

4. The product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering according to claim 3, characterized in that, The process of determining whether the product under test is installed or not, based on product transparency and each valid observation direction, includes: Determine the average pixel transparency of the effective rendering region in each of the effective viewing directions; Determine the average pixel transparency corresponding to all the effective viewing directions; The installation or disassembly status of the product under test is determined based on the magnitude of the average value.

5. The product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering according to claim 4, characterized in that, The step of determining the installation / disassembly status of the product under test as either installed or not based on the magnitude of the average value includes: When the average value is greater than the first preset threshold, the installation and disassembly status of the product under test is determined to be the installed state; When the average value is less than the second preset threshold, the product under test is determined to be in an uninstalled state. Wherein, the first preset threshold is greater than the second preset threshold.

6. The product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering according to claim 5, characterized in that, Also includes: When the average value is greater than the second preset threshold and less than the first preset threshold, the confidence level for determining that the product is in an installed state is as follows: Where D represents the confidence level and K represents the mean. This indicates the second preset threshold. The first preset threshold is displayed.

7. A product assembly / disassembly status detection device based on three-dimensional Gaussian sputtering, characterized in that, include: The conversion module is used to convert product component models in the inspection scene into triangular patch models; The splitting module is used to split the triangular facet model into units of each component, resulting in a split facet model in which the side length of each triangular facet is smaller than a preset side length. The output module is used to output the vertices of each triangular facet in the split facet model as an initial point cloud for three-dimensional Gaussian sputtering. The deployment module is used to deploy targets for virtual-real fusion at corresponding positions in the virtual model and the real scene. The scanning module is used to scan around the area of ​​the product to be tested to obtain image information and pose information containing the virtual-real fusion target; The reconstruction module is used to reconstruct a three-dimensional scene based on the image information, the pose information and the initial point cloud, using three-dimensional Gaussian sputtering to obtain a scene model of the product under test. The determination module is used to construct the envelope of the product under test in the real scene, obtain all three-dimensional Gaussian sputtering bodies in the envelope of the corresponding position in the scene model of the product under test; determine the effective viewing direction of each three-dimensional Gaussian sputtering body; and determine the installation / disassembly state of the product under test as either installed or not installed based on the product transparency and each effective viewing direction.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the product assembly / disassembly status detection method based on three-dimensional Gaussian sputtering as described in any one of claims 1 to 6.

Images