A 3D reconstruction method and system for a truss 3D camera

Abstract

The embodiments of the present specification provide a truss 3D camera three-dimensional reconstruction method and system, the method comprising: obtaining optical scanning parameters based on the structural characteristics of the truss; optically imaging the target object based on the optical scanning parameters to determine the appearance image of the target object; determining the regional positions of multiple sub-regions of the target object and the shape complexity of the multiple sub-regions based on the appearance image; determining a first scanning parameter based on the regional positions of the multiple sub-regions and the shape complexity of the multiple sub-regions; sending the first scanning parameter to a robotic arm, which performs scanning and obtains first point cloud data of the target object.

Description

Technical Field

[0001] The present invention relates to the field of 3D reconstruction, and in particular to a 3D reconstruction method and system for a truss 3D camera. Background Art

[0002] With the continuous advancement of industrial automation and robotics, 3D reconstruction technology has been widely used in industrial inspection, robot grasping, augmented reality and other fields. The 3D reconstruction is completed by mounting the camera at the end of a robotic arm or truss to form a so-called "eye-in-hand" configuration and moving the camera to capture multiple perspective images of the object. However, although this method can effectively capture 3D information, when the complexity of the various parts of the target object varies, a single scanning method often cannot achieve the ideal reconstruction effect, and it is difficult to achieve high precision and high resolution requirements.

[0003] Therefore, it is hoped to propose a truss 3D camera three-dimensional reconstruction method and system, by adopting different scanning methods for different areas of the target object to obtain more accurate point cloud data, thereby improving the accuracy and efficiency of three-dimensional reconstruction. Summary of the invention

[0004] One of the embodiments of the present specification provides a truss 3D camera three-dimensional reconstruction method, which is executed by a truss 3D camera three-dimensional reconstruction system, including: acquiring optical scanning parameters based on structural characteristics of the truss; optically imaging a target object based on the optical scanning parameters to determine an appearance image of the target object; determining area positions of multiple sub-areas of the target object and shape complexity of the multiple sub-areas based on the appearance image; determining first scanning parameters based on the area positions of the multiple sub-areas and the shape complexity of the multiple sub-areas; and sending the first scanning parameters to a robotic arm, which performs scanning and acquires first point cloud data of the target object.

[0005] One of the embodiments of the present specification provides a truss 3D camera three-dimensional reconstruction system, the system comprising an acquisition module, a first determination module, a second determination module, a third determination module and an execution module: the acquisition module is configured to: acquire optical scanning parameters based on the structural characteristics of the truss; the first determination module is configured to: perform optical imaging of the target object based on the optical scanning parameters, and determine the appearance image of the target object; the second determination module is configured to: determine the area positions of multiple sub-areas of the target object and the shape complexity of the multiple sub-areas based on the appearance image; the third determination module is configured to: determine the first scanning parameter based on the area positions of the multiple sub-areas and the shape complexity of the multiple sub-areas; the execution module is configured to: send the first scanning parameter to the robotic arm, and the robotic arm performs scanning and acquires the first point cloud data of the target object.

[0006] One of the embodiments of the present specification provides a truss 3D camera three-dimensional reconstruction device, the device includes at least one processor and at least one memory; the at least one memory is used to store computer instructions; the at least one memory is used to store computer instructions; the at least one processor is used to execute at least part of the computer instructions to implement the aforementioned truss 3D camera three-dimensional reconstruction method.

[0007] One of the embodiments of the present specification provides a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the aforementioned truss 3D camera three-dimensional reconstruction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0009] Figure 1 is an exemplary module diagram of a truss 3D camera 3D reconstruction system according to some embodiments of this specification;

[0010] Figure 2 is an exemplary flow chart of a 3D reconstruction method of a truss 3D camera according to some embodiments of this specification;

[0011] Figure 3 is an exemplary schematic diagram of determining a first scanning parameter according to some embodiments of this specification;

[0012] Figure 4 is an exemplary flowchart of determining target point cloud data according to some embodiments of this specification. DETAILED DESCRIPTION

[0013] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0014] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0015] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0016] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0017] Figure 1 is an exemplary module diagram of a truss 3D camera 3D reconstruction system according to some embodiments of this specification. Figure 1 As shown, the module 100 of the truss 3D camera 3D reconstruction system includes an acquisition module 110 , a first determination module 120 , a second determination module 130 , a third determination module 140 , an execution module 150 and a fourth determination module 160 .

[0018] In some embodiments, the acquisition module 110 is configured to acquire optical scanning parameters based on structural characteristics of the truss.

[0019] The truss is a frame structure with fixed ends, which allows the robot to move freely on it. The movement includes three directions along the XYZ axis. The 3D camera is installed on the robot arm. As the robot arm moves, the 3D camera can flexibly change its position and angle in space, thereby realizing three-dimensional scanning of the target object from multiple perspectives.

[0020] A 3D camera is an imaging device used to obtain a two-dimensional image and three-dimensional spatial information of a target object. For example, a 3D camera may include, but is not limited to, an image acquisition device and a point cloud acquisition device.

[0021] The image acquisition device refers to a device used to capture a two-dimensional visual image (eg, color, texture, etc.) of a target object, such as a camera.

[0022] The point cloud acquisition device refers to a device used to collect three-dimensional coordinate data of a target object and form point cloud data, such as LIDAR, depth camera, etc.

[0023] In some embodiments, the image acquisition device and the point cloud acquisition device can be integrated into different modules on the 3D camera.

[0024] In some embodiments, the first determination module 120 is configured to perform optical imaging on the target object to determine an appearance image of the target object based on the optical scanning parameters.

[0025] In some embodiments, the second determination module 130 is configured to determine the region positions of the multiple sub-regions of the target object and the shape complexity of the multiple sub-regions based on the appearance image.

[0026] In some embodiments, the second determination module 130 is further configured to merge multiple sub-regions of the target object based on the appearance image, and determine the merged region, the region position of the merged region, and the shape complexity of the merged region.

[0027] In some embodiments, the third determination module 140 is configured to determine the first scanning parameter based on the region positions of the multiple sub-regions and the shape complexity of the multiple sub-regions.

[0028] In some embodiments, the third determination module 140 is further configured to determine the first scanning parameter based on the region position of the merged region and the shape complexity of the merged region.

[0029] In some embodiments, the fourth determination module 150 is configured to determine the defect area and the degree of defect based on the first point cloud data; determine the second scanning parameters based on the defect area and the degree of defect; perform a supplementary scan on the defect area based on the second scanning parameters to determine the second point cloud data; and determine the target point cloud data based on the first point cloud data and the second point cloud data.

[0030] In some embodiments, the fourth determination module 150 is further configured to determine the scanning angle and scanning path of the second scanning parameter based on the defect surface; and determine the scanning speed of the second scanning parameter based on the defect area and the defect degree.

[0031] In some embodiments, the execution module 160 is configured to send the first scanning parameter to the robotic arm, and the robotic arm performs scanning and acquires first point cloud data of the target object.

[0032] For more information about the aforementioned modules and their functions, see Figure 2-4 and its related description.

[0033] In some embodiments, part or all of the modules of the truss 3D camera 3D reconstruction system may be integrated into a processor to implement one or more functions described in this embodiment.

[0034] The processor may process information and / or data related to the truss 3D camera 3D reconstruction system. In some embodiments, the processor may include one or more processing engines (e.g., a single-chip processing engine or a multi-chip processing engine). As an example only, the processor may include a central processing unit (CPU), an application specific integrated circuit (ASIC), etc., or any combination thereof.

[0035] It should be noted that the above description of the truss 3D camera 3D reconstruction system and its modules is only for convenience of description and cannot limit this specification to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the modules or form a subsystem to connect with other modules without deviating from this principle.

[0036] Figure 2 FIG. 1 is an exemplary flow chart of a 3D reconstruction method of a truss 3D camera according to some embodiments of this specification. Figure 2 As shown, the process 200 includes the following steps. In some embodiments, the process 200 may be executed by a processor.

[0037] In some embodiments, the processor obtains optical scanning parameters based on the structural characteristics of the truss; based on the optical scanning parameters, optically images the target object to determine an appearance image of the target object; based on the appearance image, determines the regional positions of multiple sub-regions of the target object and the shape complexity of the multiple sub-regions; determines first scanning parameters based on the regional positions of the multiple sub-regions and the shape complexity of the multiple sub-regions; and sends the first scanning parameters to the robotic arm, which performs scanning and obtains first point cloud data of the target object.

[0038] Step 210: Obtain optical scanning parameters based on the structural characteristics of the truss.

[0039] Structural features refer to parameters used to describe the structural properties of a truss. For example, the structural features of a truss may include, but are not limited to, the number of dead spots in the truss that cannot be reached by a robot arm and the maximum external force that the truss can withstand.

[0040] The optical scanning parameters refer to the photographing parameters of the image acquisition device in the 3D camera. For example, the optical scanning parameters may include but are not limited to the photographing point, the photographing angle, etc.

[0041] In some embodiments, the processor determines the optical scanning parameters of the image acquisition device by querying a first preset relationship table based on the structural characteristics of the truss.

[0042] In some embodiments, the first preset relationship table may include a correspondence between the structural features of the truss and the optical scanning parameters. In some embodiments, the first preset relationship table may be determined based on the structural features of the truss in the historical data and the actual optical scanning parameters. For example, in response to the number of blind spots that the robot arm cannot reach and the maximum external force that the truss can withstand, the corresponding optical scanning parameters have more photographic points and more photographic angles.

[0043] Step 220 , optically imaging the target object based on the optical scanning parameters to determine an appearance image of the target object.

[0044] The target object is the object that needs to be reconstructed and scanned. For example, the target object includes an industrial robot or an automation system (such as an automatic welding production system).

[0045] Appearance images refer to images used to display two-dimensional information of a target object at different angles. For example, appearance images can display information such as the concave and convex features, sharp corner features, color features, texture features, and brightness features of the target object.

[0046] The concave-convex feature refers to information used to reflect whether the target object is flat. In some embodiments, the processor can represent the concave-convex feature by the number of concave-convex points in the appearance image of the target object.

[0047] The sharp corner feature refers to information used to reflect whether the target object has a sharp turning point. In some embodiments, the processor can characterize the sharp corner feature by the number of sharp corners in the appearance image of the target object.

[0048] The color feature refers to the color information of the target object. In some embodiments, the processor can calculate the grayscale distribution or color distribution of the appearance image, and use the grayscale distribution value or color distribution value as the color feature of the appearance image of the target image.

[0049] Texture features refer to information such as patterns, textures, and structures on the surface of a target object. In some embodiments, the processor uses a local binary pattern (LBP) to extract texture features of the target object. That is, local texture information is encoded by comparing the grayscale values ​​of each pixel with its surrounding pixels.

[0050] The brightness feature refers to the brightness information of the target object. In some embodiments, the processor can use the brightness distribution range as the brightness feature.

[0051] In some embodiments, the processor configures the image acquisition device based on the optical scanning parameters, and the image acquisition device performs optical imaging on the target object according to the set optical scanning parameters, thereby acquiring an appearance image of the target object.

[0052] Step 230 : determining the region positions of the multiple sub-regions of the target object and the shape complexity of the multiple sub-regions based on the appearance image.

[0053] A subregion refers to a small area with specific size and shape characteristics divided from the appearance image of the target object.

[0054] In some embodiments, the processor may divide the appearance image into a plurality of sub-areas according to a preset area size and a preset area shape.

[0055] The preset area size refers to the pre-set size of each sub-area, which can be set by professional technicians or by system default.

[0056] The preset area shape refers to the shape for dividing the appearance image, such as a rectangle, a circle, etc., and can be set by professional technicians or by system default.

[0057] The region position of the sub-region refers to the specific position of the sub-region in the appearance image. For example, the region position of the sub-region can be represented in the form of coordinates.

[0058] In some embodiments, the processor may use the coordinates of the center point of the sub-region and the coordinates of its corner points to indicate its position in the appearance image.

[0059] The center point coordinates refer to the coordinate position of the center of the sub-area. For example, in response to the preset area shape being a rectangle, the center point coordinates are the coordinates of the intersection of the diagonal lines. For another example, in response to the preset area shape being a circle, the center point coordinates are the coordinates of the center of the circle.

[0060] Corner point coordinates refer to the coordinates of the vertices of the sub-region. For example, in response to the preset region shape being a rectangle, the corner point coordinates are the coordinates of the vertices. For another example, in response to the preset region shape being a circle, the corner point coordinates are the coordinates of any point on the circle.

[0061] The shape complexity of the sub-region refers to the shape complexity of the sub-region. For example, the shape complexity of the sub-region can be represented in the form of a numerical value, and the larger the numerical value, the higher the complexity of the sub-region.

[0062] In some embodiments, the processor may calculate the shape complexity of the sub-region based on the concave-convex features, sharp corner features, color features, texture features, and brightness features of the appearance image of the target object through a first preset algorithm.

[0063] The first preset algorithm is to perform weighted summation on the concave-convex features, sharp corner features, color features, texture features and brightness features of the appearance image. The weighted coefficient can be set by professional technicians or by system default.

[0064] As an example only, the processor responds to the target object's appearance image's appearance image having a greater number of concave and convex points in the concave and convex features, a greater number of sharp corners in the sharp corner features, a wider grayscale distribution or color distribution of the appearance image in the color features, a more complex LBP feature in the texture features, and a wider brightness distribution range in the brightness features, then the sub-region has a higher shape complexity.

[0065] Step 240: Determine a first scanning parameter based on the region positions of the multiple sub-regions and the shape complexity of the multiple sub-regions.

[0066] The first scanning parameter refers to a parameter used to obtain the initial point cloud data of the target object. For example, the first scanning parameter includes but is not limited to a scanning path, a scanning angle, and a scanning speed.

[0067] In some embodiments, the first scanning parameter may be represented in the form of a sequence. For example, the first scanning parameter may be {[scanning path is from A1 to A2, scanning angle is 90°, scanning speed is S1], [scanning path is from A2 to A3, scanning angle is 90°, scanning speed is S2], ...}.

[0068] The scanning path refers to the trajectory of the point cloud acquisition device when performing a scanning task, that is, the route that the 3D camera moves on the truss.

[0069] The scanning angle refers to the angle set by the point cloud acquisition device relative to the scanning target object at each position in the scanning path.

[0070] Scanning speed refers to the average speed at which the point cloud acquisition device moves on the scanning path. When the length of the scanning path is the same, the slower the scanning speed, the more point cloud data the 3D camera can capture at each position, but the overall scanning time will be longer.

[0071] In some embodiments, the point cloud acquisition device can stop at each scanning point and collect point cloud data according to a preset scanning frequency. When moving between different scanning points, it is necessary to move according to a preset step length. The preset scanning frequency and the preset step length can be set by professional technicians or by system default.

[0072] In some embodiments, the processor may determine the scanning path in the first scanning parameter based on the regional positions of the plurality of sub-regions. For example, the processor may specify that the point cloud acquisition device of the 3D camera scans successively along the X-axis, Y-axis, and Z-axis directions of the truss to ensure that every position of the target object is included, thereby achieving all-round point cloud data collection.

[0073] In some embodiments, the processor may also determine the scanning angle and the scanning speed in the first scanning parameter by querying the second preset relationship table based on the shape complexity of the plurality of sub-regions.

[0074] In some embodiments, the second preset relationship table may include the shape complexity of different sub-areas, and the corresponding relationship between the scanning angle and the scanning speed in the first scanning parameter. In some embodiments, the second preset relationship table may be determined based on the shape complexity of different sub-areas in the historical data, and the actual scanning angle and scanning speed. For example, the actual scanning angle and scanning speed may be determined based on prior experience.

[0075] In some embodiments, the processor may also merge multiple sub-regions of the target object based on the appearance image to determine a merged region, a region position of the merged region, and a shape complexity of the merged region. The size of the merged region is greater than the size of the sub-region. Based on the region position of the merged region and the shape complexity of the merged region, the first scanning parameter is determined by a preset rule.

[0076] Merging regions refers to combining multiple adjacent sub-regions of a target object or sub-regions with similar characteristics to form a larger region through a preset rule or algorithm, wherein the preset rule or algorithm may be set by a professional technician or by system default.

[0077] The region position of the merged region refers to the specific spatial position of the merged region on the target object.

[0078] The regional complexity of the merged region refers to data used to characterize the complexity of the merged region.

[0079] In some embodiments, before the sub-regions are merged, the processor pre-processes the appearance image through image fusion technology, such as pixel-level or feature-level image fusion algorithm, to improve the accuracy of subsequent contour recognition. Then, based on the contour recognition algorithm, such as Canny edge detection, the sub-region contour in the appearance image is recognized. Subsequently, based on the recognized contour, multiple sub-regions within the same contour are merged into a larger merged region, thereby completing the determination of the merged region.

[0080] The same contour refers to a set of continuous edge points detected by the contour recognition algorithm. Edge points can be connected to form a closed path in the appearance image, representing the same surface or edge on the target object.

[0081] In some embodiments, the method of determining the regional position of the merged area and the regional complexity of the merged area is similar to the method of determining the regional position of the sub-area and the regional complexity of the sub-area described above, respectively, and will not be repeated here.

[0082] In some embodiments, the processor may also determine the merged region, the region position of the merged region, and the shape complexity of the merged region through a merge model based on the appearance image (ie, the region images of all sub-regions, the region positions of the sub-regions).

[0083] The sub-region region image refers to an image corresponding to each sub-region after the appearance image is divided into multiple sub-regions.

[0084] In some embodiments, the merged model may be a Convolutional Neural Network (CNN) model.

[0085] In some embodiments, the merged model can be obtained by training based on a large number of first training samples with first labels. The processor can input multiple first training samples with first labels into the initial merged model, construct a loss function through the first label and the result of the initial merged model, and iteratively update the initial merged model based on the loss function. When the preset conditions are met, the model training is completed and a trained merged model is obtained. Among them, the preset conditions can be that the loss function converges, the number of iterations reaches a threshold, etc.

[0086] In some embodiments, the first training sample for training the merging model may be a sample appearance image corresponding to a sample target object in the historical sample data. The first label may be the actual merged region, the region position of the merged region, and the shape complexity of the merged region corresponding to the merged multiple sub-regions of the target object based on the sample appearance image in the historical sample data.

[0087] The actual merged region can be obtained by manual annotation, for example, the sub-regions belonging to the same contour in the appearance image are annotated as merged regions. The region position of the actual merged region is the coordinates of the center point and the corner point of the merged region. The shape complexity of the actual merged region can be calculated and obtained by the aforementioned first preset algorithm.

[0088] In some embodiments of the present specification, the processor can more accurately identify the contour of the appearance image through the merging model based on the appearance image, and then effectively determine the merged area, the area position of the merged area, and the shape complexity of the merged area. This helps to efficiently and intelligently determine the merged area, the area position of the merged area, and the shape complexity of the merged area, and improve accuracy.

[0089] In some embodiments, the processor may also determine whether each sub-region is merged based on the region position of the merged region, and then determine the first scanning parameter through a preset rule. For example, for the sub-regions that are not merged, the preset rule is to increase the scanning speed of the sub-regions that are not merged based on the current first scanning parameter. For another example, for the merged sub-regions (i.e., the merged region), the preset rule may also be that the scanning speed in the first scanning parameter is negatively correlated with the shape complexity of the merged region; for example, the greater the shape complexity of the merged region, the slower the scanning speed.

[0090] In some embodiments of the present specification, the processor divides the target object into different areas by accurately dividing its outline, ensuring that each area basically contains the same type of objects, such as raw materials, production equipment or finished products. Personalized configuration is performed for each area, thereby significantly improving the scanning efficiency. For example, compared with setting the first scanning parameters for each sub-area one by one for scanning, using different first scanning parameters only for the merged area and the non-merged sub-area can effectively reduce the configuration time and improve the overall scanning efficiency.

[0091] For more information on determining the first scan parameters, see Figure 3 and its related description.

[0092] Step 250: Send the first scanning parameter to the robotic arm, and the robotic arm performs scanning and obtains first point cloud data of the target object.

[0093] The first point cloud data refers to a point cloud data set obtained by rapid scanning during the preliminary scanning. For example, the first point cloud data includes but is not limited to the spatial coordinate points on the surface of the target object.

[0094] In some embodiments, the processor may send the first scanning parameter to the robotic arm, and the robotic arm starts to move and scan the target object according to a predetermined scanning path, scanning angle, and scanning speed. During the movement, the point cloud acquisition device in the 3D camera will capture the surface information of the target object continuously or at specific intervals. Under the guidance of the robotic arm, the point cloud acquisition device quickly scans all positions of the target object, and generates the first point cloud data by scanning the collected data.

[0095] In some embodiments of the present specification, the processor obtains optical scanning parameters based on the structural features of the truss and determines the appearance image, which helps to improve the accuracy and uniformity of the appearance image. The processor determines the regional position and shape complexity of multiple sub-regions based on the appearance image, which can provide an important basis for subsequent scanning. In addition, the processor determines the first scanning parameters based on the regional position and shape complexity of the sub-region, and the robotic arm performs scanning to obtain the first point cloud data, which can quickly capture the overall situation of the target object and improve the accuracy and efficiency of three-dimensional reconstruction.

[0096] It should be noted that the above description of process 200 is only for illustration and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the process breath holding training under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0097] Figure 3 is an exemplary schematic diagram of determining the first scanning parameter according to some embodiments of the present specification.

[0098] In some embodiments, Figure 3 As shown, the processor may construct a scan structure map 330 based on the region position 310 of the merged region and the shape complexity 320 of the merged region.

[0099] The scanning structure graph refers to a knowledge graph used to characterize the distribution and attributes of the region positions of multiple merged regions. In some embodiments, the scanning structure graph is composed of at least one node and at least one edge, the edge connects the nodes, and the nodes and the edges have attributes.

[0100] In some embodiments, the nodes of the scanned structure map correspond to the regional positions of the merged regions. In some embodiments, the attributes of the nodes of the scanned structure map can reflect the characteristics of the nodes. For example, the attributes of the nodes of the scanned structure map include but are not limited to the shape complexity of the merged regions.

[0101] In some embodiments, in response to the size of the merged region after merging being greater than a preset size threshold, the node of the scanned structure map may also include the center point coordinates and corner point coordinates of the region position of the merged region (i.e., the region whose size is greater than the preset size threshold). The preset size threshold may be obtained by preset.

[0102] In some embodiments, the edge of the scan structure map refers to a scanning path that the 3D camera can move between the merged areas corresponding to two adjacent nodes. The scanning path is the shortest scanning path between two adjacent nodes, that is, the connection between the two adjacent nodes constitutes an edge.

[0103] In some embodiments, the 3D camera can scan back and forth between two adjacent nodes, and the edge of the scanning structure graph is an undirected edge, and the attributes of the edge include the scanning speed.

[0104] In some embodiments, the properties of the edges may also include the stability of the truss.

[0105] In some embodiments, the processor determines the coordinates of different positions on the edge where the node is located through a coordinate conversion rule, and then calculates the stability of the truss corresponding to the position through a second preset algorithm based on the coordinates on the truss. The coordinate conversion rule can be determined by professional technicians based on prior experience.

[0106] The stability of a truss refers to the ability of the truss structure to maintain its shape and function in different positions. Since the structure, material, size and support conditions of the truss may vary in different positions, the stability of the truss in different positions will also vary, so it is necessary to determine the stability of the truss corresponding to different positions.

[0107] In some embodiments, the processor may normalize the structure, material strength, weight, and size of the trusses at different positions (for example, logarithmic normalization, etc.), and then calculate the stability of the trusses corresponding to the positions through a second preset algorithm.

[0108] The second preset algorithm is to obtain a weighted sum of the normalized values ​​of the structure, material strength, weight, and size of the trusses at different positions. The weight coefficient can be set by professional technicians or by system default.

[0109] The structure of the truss refers to the structural features of the truss corresponding to different positions. For example, the structure of the truss can be characterized by the number of supporting legs of the truss corresponding to the position, which can be determined by input from a technician.

[0110] The strength of a material refers to the mechanical properties of the material at different positions, such as yield strength, tensile strength, etc., which can be determined by technical personnel input.

[0111] The weight refers to the deadweight of the truss corresponding to different positions, which can be determined by technical personnel.

[0112] The size refers to the overall size of the truss corresponding to different positions, which can be determined by technical personnel input.

[0113] In some embodiments, the stability of the truss is positively correlated with the structure, material strength, weight and size of the truss at that location; that is, the more supporting legs the structural features of the truss at that location have, the greater the material strength, the heavier the weight and the larger the size, the higher the stability of the truss corresponding to that location.

[0114] In some embodiments, the processor constructs a scan structure graph based on the above-mentioned nodes, edges, attributes of the nodes and attributes of the edges.

[0115] In some embodiments, the processor generates candidate scanning parameters 340 based on the scanning structure map 330 .

[0116] In some embodiments, the scanning paths and / or scanning speeds in different candidate scanning parameters may be different.

[0117] In some embodiments, the processor may determine a plurality of candidate scanning parameters by selecting a scanning strategy in conjunction with the scanning structure map.

[0118] In some embodiments, the scanning strategy may include but is not limited to strategy one and strategy two.

[0119] Strategy 1 is a strategy for randomly generating repeated scanning paths. For example, when processing complex areas, multiple round-trip scans are performed while keeping the scanning speed constant.

[0120] Strategy 2 is to generate a non-repetitive scanning path and randomly adjust the scanning speed. For example, for complex areas, the scanning speed is appropriately reduced to optimize the scanning effect.

[0121] In some embodiments, the processor may generate a random number in the interval [0, 1] based on a pseudo-random number generator, and compare the random number with a set probability value to determine a scanning strategy. For example, in response to the random number being less than or equal to the set probability value, strategy one is executed; otherwise, strategy two is executed.

[0122] In some embodiments, the size of the set probability value is positively correlated with the number of edges in the scanned structure graph.

[0123] In some embodiments, in response to the smaller number of edges in the scanned structure map, the smaller number of regions that represent the merged area, the smaller the selectable scan paths, and the lower the set probability value, the lower the probability that the random number is less than or equal to the set probability value. Therefore, at this time, it is more appropriate to adopt a strategy of not repeating the scan path and reducing the scan speed (strategy 2), which helps to avoid data redundancy and improve the scan quality.

[0124] In some embodiments, the adjustment range of the candidate scanning parameters between two nodes is positively correlated to the shape complexity of the merged area corresponding to the two nodes. The adjustment range includes the number of repetitions of the scanning path and the reduction range of the scanning speed. The adjustment range of the candidate scanning parameters refers to the difference between the next candidate scanning parameter and the candidate scanning parameter after determining a candidate scanning parameter.

[0125] In some embodiments of this specification, the 3D camera has three movement modes in the X, Y, and Z directions on the truss, which means that the scanning path has diversity. By constructing a scanning structure map, not only the movement path of the 3D camera can be considered, but also the relationship between the regional positions of the merged areas can be considered, which helps to determine the better candidate scanning parameters.

[0126] In some embodiments, the processor determines the scanning efficiency 360 of the candidate scanning parameters through the prediction model 350 based on the scanning structure map 330 and the candidate scanning parameters 340. The scanning efficiency is related to the total length of the scanning path and the scanning effect.

[0127] In some embodiments, the prediction model may be a Graph Neural Network (GNN) model.

[0128] In some embodiments, the prediction model can be obtained by training based on a large number of second training samples with second labels. The training method of the prediction model is similar to the training method of the merged model, which will not be repeated here.

[0129] In some embodiments, the second training sample for training the prediction model may be a sample scanning structure map and sample scanning parameters of a sample target object in the historical sample data. The second label may be the actual scanning efficiency under the sample scanning parameters corresponding to the sample scanning structure map.

[0130] Scanning efficiency refers to a metric used to measure the performance of the scanning process.

[0131] In some embodiments, the processor may determine the scanning efficiency by using the following formula (1) based on the relationship that the scanning efficiency is positively correlated with the scanning effect and negatively correlated with the total length of the scanning path.

[0132] (1)

[0133] in, For scanning efficiency, For scanning effect, is the total length of the scan path, , is the weight coefficient, which can be obtained by default.

[0134] The total length of the scan path refers to the total length of the scan path in the candidate scan parameters.

[0135] The scanning effect refers to the similarity between the first point cloud data and the appearance image of the target object. The higher the similarity, the better the scanning effect.

[0136] In some embodiments, the processor determines the first scanning parameter 370 based on the scanning efficiency 360. For example, the processor may determine the candidate scanning parameter with the highest scanning efficiency as the first scanning parameter.

[0137] For more information about the first scanning method, see Figure 2 and its related description.

[0138] In some embodiments, the processor determines the scanning path of the first scanning parameter based on the field of view of the 3D camera. The processor may divide the scanning path according to the field of view of the 3D camera. For example, the length of each scanning path is equal to the camera range; or the length of each scanning path is equal to half of the camera range, etc.

[0139] The field of view refers to the spatial area that a 3D camera can capture. For example, the field of view may be the camera range of a 3D camera.

[0140] In some embodiments, the processor may also determine a scanning speed of the first scanning parameter based on the stability of the truss. For example, the scanning speed of the first scanning parameter may be positively correlated with the stability of the truss. The higher the stability of the truss, the lower the possibility of position error of the 3D camera during movement, thereby allowing the 3D camera to scan at a faster speed. On the contrary, the lower the stability of the truss, the faster the 3D camera moves, the more likely it is that the truss will vibrate, thereby affecting the accuracy of the scan.

[0141] In some embodiments of the present specification, the processor determines the scanning path based on the field of view of the 3D camera, which can ensure that the scanning covers all necessary areas while avoiding repeated scanning or omissions, thereby improving the accuracy of the data. The scanning speed is determined based on the stability of the truss, so that the scanning can be performed within the tolerance range of the truss, reducing the vibration of the truss caused by excessive speed, thereby reducing errors in the scanning process.

[0142] In some embodiments of the present specification, by constructing a scanning structure map and determining candidate scanning parameters, it is helpful to optimize the scanning path and reduce overlaps and omissions during the scanning process. Based on the scanning structure map and scanning parameters, the scanning efficiency is determined through a prediction model, and the candidate scanning parameter with the highest scanning efficiency is determined as the first scanning parameter, which can significantly improve the accuracy and efficiency of the scanning operation.

[0143] Figure 4 is an exemplary flow chart for determining target point cloud data according to some embodiments of this specification. Figure 4 As shown, the process 400 includes the following steps. In some embodiments, the process 400 may be executed by a processor.

[0144] In some embodiments, the processor determines defect data based on the first point cloud data; determines second scanning parameters based on the defect data; performs a supplementary scan on the defect area based on the second scanning parameters to determine second point cloud data; and determines target point cloud data based on the first point cloud data and the second point cloud data.

[0145] Step 410: Determine defect data based on the first point cloud data.

[0146] In some embodiments, the processor performs three-dimensional reconstruction of the target object based on the first point cloud data to determine defect data.

[0147] Combined with the above, the 3D camera moves to the starting point of the truss workbench, and the entire target object is segmented and confirmed based on the camera range of the 3D camera. After reaching the predetermined photo point, the photo operation is performed. By using the calibration data of the 3D camera, the conversion relationship between the 3D camera coordinate system and the truss coordinate system is determined, and the point cloud data in the current 3D camera coordinate system is converted to the truss coordinate system.

[0148] The specific conversion process is as follows: First, obtain the position of the truss coordinate system when the 3D camera takes pictures and the position data of each axis of the current truss; then, by comparing the data of the current shooting point with the calibration data after the 3D camera is installed, calculate the corresponding position of the first point cloud data in the truss coordinate system, and calculate the difference between the current 3D camera shooting point and the 3D camera shooting point during calibration on each axis in the truss coordinate system; finally, interpolate these differences to each point of the first point cloud data. After completing the interpolation of the data of each axis, the point cloud data obtained by multiple shootings will be reconstructed in the same coordinate system, thereby realizing the three-dimensional reconstruction of the target object.

[0149] Defect data refers to data used to characterize poor 3D reconstruction effect. In some embodiments, the defect data includes defect area and / or defect degree.

[0150] The defective area refers to an area where the first point cloud data is insufficient and / or the first point cloud data is of poor quality.

[0151] In some embodiments, the defective area may also include a defective surface. During the scanning process, if the scanning data of any surface of the target object is incomplete, inaccurate, or fails to meet the predetermined quality standard due to the position, angle, or other scanning conditions of the 3D camera, the surface is considered a defective surface. For example, when the 3D camera scans the target object from directly above, if the side of the target object is not scanned well, the side is considered a defective surface.

[0152] The defect degree refers to the severity of the data quality problem in the defect area. For example, the defect degree can be represented by the registration accuracy.

[0153] Registration accuracy refers to the data used to evaluate the registration quality of the first point cloud data.

[0154] In some embodiments, the processor may determine defect data based on a variety of methods. For example, the processor determines the scan overlap area by scanning at different viewing angles; then evaluates the registration accuracy (i.e., the degree of defect) of the first point cloud data by calculating the registration error (e.g., root mean square error) of the overlap area; and finally, determines the area where the registration accuracy is lower than the accuracy threshold as the defect area. The accuracy threshold can be preset and obtained.

[0155] Scan overlap refers to the area where two or more scans partially or completely overlap in space due to multiple scans of the target object from different angles or positions during the 3D scanning process. If the quality of the scan overlap is poor, it indicates that there are flaws in the scan, for example, the first point cloud data does not match the target object accurately, or the first point cloud data is missing.

[0156] In some embodiments, the processor can evaluate whether there are differences or errors in the scan overlap area based on the registration accuracy. In addition, the processor can also reduce errors that occur during the scanning process by optimizing the registration algorithm (eg, ICP algorithm), thereby improving the accuracy of the scan data.

[0157] In some embodiments, the processor may also determine defect data based on the first point cloud data through a defect area model; wherein the defect area model is a machine learning model.

[0158] In some embodiments, the defect region model is a Convolutional Neural Network (CNN) model.

[0159] In some embodiments, the defect area model can be obtained by training based on a large number of third training samples with third labels. The training method of the defect area model is similar to the training method of the merged model, which will not be repeated here.

[0160] In some embodiments, the third training sample for training the defect area model may be the sample first point cloud data corresponding to the sample first scanning parameter in the historical sample data. The third label may be the actual defect data corresponding to the three-dimensional reconstruction based on the sample first point cloud data in the historical sample data. The processor may determine the degree of defect in the actual defect data based on the degree of defect in the defect data, which is positively correlated with the noise characteristics, data loss value, and number of outliers of the first point cloud data. The processor may determine the area where the degree of defect is greater than the accuracy threshold as the defect area in the actual defect data.

[0161] The noise feature refers to a random error value in the first point cloud data. In some embodiments, the processor may use the standard deviation of the first point cloud data as the noise feature of the first point cloud data.

[0162] The data loss value refers to the data that is blank or missing in the first point cloud data. In some embodiments, the processor may calculate the ratio of the missing data points to the total data points in the first point cloud data, and use the ratio value as the data loss value.

[0163] The number of outliers refers to the number of data points in the first point cloud data that are significantly different from the majority of surrounding data points in position, characteristics or behavior. In some embodiments, the processor can statistically identify the number of outliers based on a density-based clustering algorithm, such as RANSAC, DBSCAN, etc.

[0164] In some embodiments of the present specification, the processor can more accurately identify defect data through a defect area model based on the first point cloud data, thereby further improving the accuracy and reliability of determining the defect data.

[0165] Step 420 , determining a second scanning parameter based on the defect data.

[0166] The second scanning parameter refers to a parameter for scanning the defect area. For example, the second scanning parameter includes but is not limited to a scanning path, a scanning speed, and a scanning angle.

[0167] In some embodiments, the processor may determine the second scanning parameter in a variety of ways. For example, the processor may use the path of sequentially scanning the defective area as the scanning path in the second scanning parameter. For another example, the processor may determine the scanning speed in the second scanning parameter based on the negative correlation between the scanning speed and the degree of defect. For another example, the processor may determine the scanning angle in the second scanning parameter based on the positive correlation between the number of scanning angles and the degree of defect.

[0168] In some embodiments, the processor may further determine a second scanning parameter based on the defect region model.

[0169] In some embodiments, the defect area model may further include a second scanning parameter layer; the input of the second scanning parameter layer includes defect data and a scanning structure map; and the output includes second scanning parameters.

[0170] In some embodiments, the second scanning parameter layer may be a graph neural network (GNN) model.

[0171] In some embodiments, the second scanning parameter layer can be obtained by training based on a large number of fourth training samples with fourth labels. The training method of the second scanning parameter layer is similar to the training method of the merged model, which will not be repeated here.

[0172] In some embodiments, the fourth training sample for training the second scanning parameter layer may be a sample scanning structure map of a sample target object in the historical sample data, or sample defect data corresponding to the sample first point cloud data. The fourth label may be a preferred scanning parameter corresponding to the historical sample data, which may be obtained by manual annotation. For example, a professional technician may determine a scanning parameter with a defect degree lower than a preset threshold as a preferred scanning parameter (i.e., a second scanning parameter) based on historical experience or the defect degree of historical data. The preset threshold may be set by a professional technician or by the system by default.

[0173] In some embodiments of the present specification, based on the defect data and the scanned structure map, the second scanning parameters are determined through the second scanning parameter layer, which can ensure that the selected scanning parameters are closer to the defect area, and can help improve the accuracy of subsequent supplementary scanning of the defect area, thereby enhancing the overall scanning effect.

[0174] In some embodiments, the processor may also determine the scanning angle and scanning path of the second scanning parameter based on the defective surface. For example, based on the position of the defective surface, the scanning angle and scanning path in the second scanning parameter are further determined by a third preset algorithm. The third preset algorithm is that the processor may determine the scanning angle and scanning path when the field of view of the 3D camera completely covers the defective surface as the scanning angle and scanning path in the second scanning parameter.

[0175] In some embodiments, the scanning angle and scanning path when the field of view of the 3D camera completely covers the defective surface can be determined based on prior experience.

[0176] In some embodiments, the processor may further determine a scanning speed of the second scanning parameter based on the defect data. For example, the scanning speed may be negatively correlated with the degree of defect.

[0177] In some embodiments of this specification, the processor determines the scanning angle and scanning path of the second scanning parameter based on the defective surface, which can ensure that the 3D camera can completely cover the defective surface during scanning, thereby improving the coverage and quality of the scan. In areas with a high degree of defects, by appropriately reducing the scanning speed, more dense point cloud data can be obtained, thereby improving the ability to capture point cloud data in defective areas.

[0178] Step 430: Based on the second scanning parameters, perform a supplementary scan on the defective area to determine second point cloud data.

[0179] The second point cloud data refers to point cloud data obtained after scanning the defective area.

[0180] The method for acquiring the second point cloud data is similar to the method for acquiring the first point cloud data, except that the second point cloud data is acquired by performing a supplementary scan on the defective area.

[0181] Step 440: Determine target point cloud data based on the first point cloud data and the second point cloud data.

[0182] The target point cloud data refers to the point cloud data set that actually corresponds to the target object after processing and optimization.

[0183] In some embodiments, the processor may replace the first point cloud data of the defective area with the second point cloud data corresponding to the defective area, and then use the point cloud data of all areas of the target object as the target point cloud data.

[0184] In some embodiments, the processor may also pre-process the first point cloud data and the second point cloud data based on the shape complexity of the defective area and the merged area, and determine the pre-processed point cloud data as the target point cloud data.

[0185] The preprocessing includes retaining the second point cloud data of the defective area and compressing the point cloud data of the area whose shape complexity is less than the complexity threshold in the merged area.

[0186] The complexity threshold can be set by professional technicians or by system default.

[0187] In some embodiments, compression refers to performing a compression operation on point cloud data with the same or similar coordinates. For example, multiple similar point cloud data are compressed into one point cloud data, and the point cloud data is the average value of the coordinate values ​​of the aforementioned multiple point cloud data.

[0188] In some embodiments of the present specification, retaining the second point cloud data of the defective area can ensure the quality of the point cloud data of the defective area. In addition, compressing the point cloud data in the merged area helps to improve the efficiency of subsequent analysis and reconstruction by reducing the amount of point cloud data.

[0189] In some embodiments, preprocessing may further include merging point cloud data, and the degree of merging of the merged point cloud data is negatively correlated to the shape complexity of the merged area. For example, in response to the lower the shape complexity of the merged area, the greater the degree of merging. In this case, the point cloud data can be greatly reduced because simple shapes do not require a large number of points to accurately describe. As an example only, only the data of corner points or inflection points can be retained, and other intermediate points can be removed. Conversely, in response to the higher the shape complexity of the merged area, the lower the degree of merging, which helps to reflect the geometric features and details of the target object.

[0190] Merging point cloud data refers to the operation of integrating points in multiple point cloud data to reduce the amount of data and improve processing efficiency.

[0191] The merging degree refers to the degree to which point cloud data is removed when merging point cloud data.

[0192] In some embodiments of the present specification, the processor merges the point cloud data based on the degree of merging, which can ensure that when merging the point cloud data, redundant point cloud data in simple areas is reduced while details in complex areas are retained, thereby optimizing the quality and size of the target point cloud data.

[0193] In some embodiments of this specification, the processor determines the second scanning parameters based on the defect data and performs a supplementary scan to improve the point cloud data of the defect area and ensure the quality of point cloud data acquisition. By combining the two scan data, it helps to improve the accuracy and integrity of the target point cloud data, providing a more reliable basis for subsequent 3D reconstruction and analysis.

[0194] It should be noted that the above description of process 400 is only for illustration and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the process breath holding training under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0195] One of one or more embodiments of the present specification provides a 3D camera 3D reconstruction device. The device includes at least one storage medium and at least one processor, the at least one storage medium is used to store computer instructions; the at least one processor is used to execute the computer instructions to implement a truss 3D camera 3D reconstruction method.

[0196] One of one or more embodiments of the present specification provides a computer-readable storage medium, the storage medium storing computer instructions, and when a computer reads the computer instructions in the storage medium, the computer executes a truss 3D camera three-dimensional reconstruction method.

[0197] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0198] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0199] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0200] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0201] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining the digits. Although the numerical domains and parameters used to confirm the breadth of the range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0202] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification is hereby incorporated by reference in its entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0203] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A 3D reconstruction method of a truss 3D camera, characterized in that: The method is performed by a truss 3D camera three-dimensional reconstruction system, and includes: Based on the structural characteristics of the truss, optical scanning parameters are obtained; Based on the optical scanning parameters, optically image the target object to determine the appearance image of the target object; Determining, based on the appearance image, region positions of a plurality of sub-regions of the target object and shape complexity of the plurality of sub-regions includes: Based on the appearance image, merging the multiple sub-regions of the target object through a merging model to determine a merging region, a region position of the merging region, and a shape complexity of the merging region; Determining a first scanning parameter based on the region positions of the multiple sub-regions and the shape complexity of the multiple sub-regions includes: Determining a scanning path of the first scanning parameter based on the area positions of the multiple sub-areas or based on a field of view of a 3D camera; Determine a scanning speed of the first scanning parameter based on the shape complexity of the multiple sub-areas, or based on the stability of the truss; the scanning speed is an average speed of the point cloud acquisition device when it moves on the scanning path, the shape complexity is obtained based on the concave-convex features, sharp corner features, color features, texture features and brightness features of the appearance image of the target object, and the stability is obtained based on the structure, material strength, weight and size of the truss at different positions; and / or Determining the first scanning parameter based on the region position of the merged region and the shape complexity of the merged region includes: Constructing a scanning structure map based on the region position of the merged region and the shape complexity of the merged region; Based on the scanned structure map, generating candidate scanning parameters; Based on the scanning structure map and the candidate scanning parameters, determining the scanning efficiency of the candidate scanning parameters through a prediction model, wherein the scanning efficiency is related to the total length of the scanning path and the scanning effect; Based on the scanning efficiency, determining the first scanning parameter; The first scanning parameter is sent to a robotic arm, and the robotic arm performs scanning and acquires first point cloud data of the target object.

2. The method according to claim 1, characterized in that The method further comprises: Determine defect data based on the first point cloud data, wherein the defect data includes a defect area and / or a defect degree; Determining a second scanning parameter based on the defect data; Based on the second scanning parameters, performing a supplementary scan on the defective area to determine second point cloud data; Preprocessing is performed based on the first point cloud data and the second point cloud data to determine target point cloud data; wherein the preprocessing includes merging point cloud data, and the merging degree of the merged point cloud data is negatively correlated to the shape complexity of the merged area.

3. The method according to claim 2, characterized in that The defect area includes a defect surface, and determining the second scanning parameter based on the defect data includes: Based on the defective surface, determining a scanning angle and a scanning path of the second scanning parameter; A scanning speed of the second scanning parameter is determined based on the defect data.

4. A 3D reconstruction system of a truss 3D camera, characterized in that: The system includes an acquisition module, a first determination module, a second determination module, a third determination module and an execution module: The acquisition module is configured to: acquire optical scanning parameters based on structural characteristics of the truss; The first determination module is configured to: perform optical imaging on the target object based on the optical scanning parameters to determine the appearance image of the target object; The second determination module is configured to: determine the region positions of the multiple sub-regions of the target object and the shape complexity of the multiple sub-regions based on the appearance image; the second determination module is further configured to: Based on the appearance image, merging the multiple sub-regions of the target object through a merging model to determine a merging region, a region position of the merging region, and a shape complexity of the merging region; The third determination module is configured to: determine the first scanning parameter based on the region positions of the multiple sub-regions and the shape complexity of the multiple sub-regions; the third determination module is further configured to: Determining a scanning path of the first scanning parameter based on the area positions of the multiple sub-areas or based on a field of view of a 3D camera; Determine a scanning speed of the first scanning parameter based on the shape complexity of the multiple sub-areas, or based on the stability of the truss; the scanning speed is an average speed of the point cloud acquisition device when it moves on the scanning path, the shape complexity is obtained based on the concave-convex features, sharp corner features, color features, texture features and brightness features of the appearance image of the target object, and the stability is obtained based on the structure, material strength, weight and size of the truss at different positions; and / or Based on the region position of the merged region and the shape complexity of the merged region, the first scanning parameter is determined; and the third determination module is further configured to: Constructing a scanning structure map based on the region position of the merged region and the shape complexity of the merged region; Based on the scanned structure map, generating candidate scanning parameters; Based on the scanning structure map and the candidate scanning parameters, determining the scanning efficiency of the candidate scanning parameters through a prediction model, wherein the scanning efficiency is related to the total length of the scanning path and the scanning effect; Based on the scanning efficiency, determining the first scanning parameter; The execution module is configured to: send the first scanning parameter to a robotic arm, and the robotic arm performs scanning and acquires first point cloud data of the target object.

5. The system according to claim 4, characterized in that The system further includes a fourth determining module, wherein the fourth determining module is configured to: Determine defect data based on the first point cloud data, wherein the defect data includes a defect area and / or a defect degree; Determining a second scanning parameter based on the defect data; Based on the second scanning parameters, performing a supplementary scan on the defective area to determine second point cloud data; Preprocessing is performed based on the first point cloud data and the second point cloud data to determine target point cloud data; wherein the preprocessing includes merging point cloud data, and the merging degree of the merged point cloud data is negatively correlated to the shape complexity of the merged area.

6. The system according to claim 5, characterized in that The defective area includes a defective surface, and the fourth determining module is further configured to: Based on the defective surface, determining a scanning angle and a scanning path of the second scanning parameter; A scanning speed of the second scanning parameter is determined based on the defect data.

7. A 3D reconstruction device using a truss 3D camera, characterized in that: The apparatus comprises at least one processor and at least one memory; The at least one memory is used to store computer instructions; The at least one processor is used to execute at least part of the computer instructions to implement the truss 3D camera three-dimensional reconstruction method as described in any one of claims 1 to 3.

8. A computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the truss 3D camera three-dimensional reconstruction method according to any one of claims 1 to 3.

Images