Scanning path optimization method and system, electronic equipment and storage medium

Abstract

The invention provides a scanning path optimization method and system, electronic equipment and a storage medium. The scanning path optimization method comprises the steps of obtaining multi-frame image data obtained when a scanning device scans a to-be-scanned object; according to any frame of image data, determining pose data when the scanning device acquires any frame of image data; scanning the virtual object based on the pose data corresponding to any frame of image data to obtain point cloud data of the virtual object; performing three-dimensional reconstruction on the virtual object according to the point cloud data to obtain first size data of the virtual object corresponding to any pose data; determining a size error corresponding to any pose data according to the first size data and second size data of the to-be-scanned object; and determining a scanning path of the scanning equipment from the multiple pieces of pose data according to the size errors corresponding to the multiple pieces of pose data. The invention relates to the technical field of three-dimensional scanning, and can improve the accuracy of three-dimensional reconstruction of a target.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional scanning technology, and in particular to a scanning path optimization method, system, electronic device and storage medium. Background Art

[0002] Currently, when performing cross-scale scanning of objects in large-scale scenes, multiple scanning devices are typically used to scan the object to be scanned to ensure the integrity of the point cloud data. For example, handheld laser scanning devices are often used to ensure global data integrity, while high-precision fixed equipment is used to obtain high-precision detailed data. Then, during the point cloud reconstruction process, different point cloud data from multiple devices are spliced ​​together, and point clouds in the same area are merged based on point cloud weights. However, this method is prone to obvious over-reconstruction and mesh connection breaks after point cloud data reconstruction, resulting in low accuracy of scan path optimization data. Summary of the Invention

[0003] In view of the above, it is necessary to propose a scanning path optimization method, system, electronic device and storage medium to solve the technical problem of low accuracy of three-dimensional reconstruction data.

[0004] The present application provides a scanning path optimization method, which is applied to an electronic device, wherein the electronic device is communicatively connected to a scanning device, and the scanning device is used to scan an object to be scanned. The method includes: obtaining multiple frames of image data obtained when the scanning device scans the object to be scanned; determining, based on any frame of image data, the posture data of the scanning device when the any frame of image data is obtained; scanning a virtual object based on the posture data corresponding to any frame of image data to obtain point cloud data of the virtual object; wherein the virtual object has the same size data as the object to be scanned; performing three-dimensional reconstruction on the virtual object based on the point cloud data to obtain first size data of the virtual object corresponding to any pose data; determining a size error corresponding to any pose data based on the first size data and the second size data of the object to be scanned; and determining the scanning path of the scanning device from the multiple pose data based on the size errors corresponding to the multiple pose data.

[0005] In some embodiments, determining the posture data of the scanning device when acquiring any frame of image data based on any frame of image data includes: determining the image coordinates of the landmark point of the object to be scanned based on the any frame of image data; and determining the posture data of the scanning device when acquiring the any frame of image data based on the image coordinates and the world coordinates of the object to be scanned.

[0006] In some embodiments, determining the posture data of the scanning device when acquiring any one frame of image data based on the image coordinates and the world coordinates of the object to be scanned includes: determining the device coordinates of the marker point in the device coordinate system of the scanning device based on the internal reference data of the scanning device and the image coordinates; determining the world coordinates of the marker point based on the world coordinates of the object to be scanned and the pre-stored coordinate difference; wherein the pre-stored coordinate difference is used to indicate the coordinate difference between the object to be scanned and the marker point; and determining the posture data of the scanning device by calculating the mapping relationship between the device coordinates and the world coordinates of the marker point.

[0007] In some embodiments, the three-dimensional reconstruction of the virtual object based on the point cloud data to obtain the first size data of the virtual object corresponding to any pose data includes: performing point cloud segmentation on the point cloud data to determine a first point cloud and a second point cloud; wherein the first point cloud is used to indicate the point cloud data belonging to the virtual object; and determining the first size data of the virtual object based on the virtual coordinates of the first point cloud and the virtual coordinates of the virtual object.

[0008] In some embodiments, determining the size error corresponding to any one of the posture data based on the first size data and the second size data of the object to be scanned includes: when there are multiple first point clouds, determining the size error corresponding to any one of the first point clouds based on the first size data and the second size data corresponding to any one of the first point clouds; and determining the size error corresponding to any one of the posture data based on the average of all size errors.

[0009] In some embodiments, determining the scanning path of the scanning device from the multiple posture data based on the size errors corresponding to the multiple posture data includes: determining multiple candidate posture data corresponding to any one posture data from the multiple posture data based on a preset step size parameter; determining the candidate posture data with the smallest corresponding size error from the multiple candidate posture data, and obtaining the target posture data corresponding to the any one posture data; and determining the scanning path of the scanning device based on the target posture data corresponding to all the posture data.

[0010] In some embodiments, the method further includes: the step size parameter includes a rotation step size and / or a translation step size; wherein the rotation step size is used to indicate the angle of rotation of the scanning device, and the translation step size is used to indicate the distance the scanning device moves.

[0011] An embodiment of the present application also provides a scanning path optimization system, which includes: a scanning device and an electronic device; the scanning device is used to scan an object to be scanned; the electronic device is used to obtain multiple frames of image data obtained when the scanning device scans the object to be scanned; based on any frame of image data, determine the posture data of the scanning device when obtaining the any frame of image data; based on the posture data corresponding to any frame of image data, scan a virtual object to obtain point cloud data of the virtual object; wherein the size data of the virtual object is the same as that of the object to be scanned; three-dimensionally reconstruct the virtual object based on the point cloud data to obtain first size data of the virtual object corresponding to any posture data; based on the first size data and the second size data of the object to be scanned, determine the size error corresponding to any posture data; based on the size errors corresponding to multiple posture data, determine the scanning path of the scanning device from the multiple posture data.

[0012] An embodiment of the present application further provides an electronic device, comprising: a memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the scan path optimization method.

[0013] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the scan path optimization method.

[0014] It can be seen from the above technical solutions that the embodiment of the present application can quantitatively characterize the impact of different postures on scanning blind spots and data overlap rates through three-dimensional reconstruction error analysis of virtual objects, generate a path with complete coverage and reasonable steps, and reduce invalid scanning actions. Dynamically adjust the scanning density according to the geometric features of the object to be scanned to improve data integrity. Replace actual repeated scanning with virtual object simulation of digital twins to reduce equipment wear, energy consumption and the cost of manual intervention. By comparing the virtual reconstructed size (first size data) with the actual size of the object to be scanned (second size data), the source of the size error in the posture data dimension is located, and closed-loop correction can be achieved in the process of planning the scanning path. In addition, environmental interference (for example, vibration, lighting changes) can be preset in the virtual environment, and the scanning path can be optimized to avoid high-noise areas, improve the signal-to-noise ratio of the actual scanning data, and thus improve the accuracy of the scanning device when scanning according to the scanning path. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of a scanning path optimization system provided in one embodiment of the present application.

[0016] Figure 2This is a flowchart of a scan path optimization method provided in one embodiment of the present application.

[0017] Figure 3 This is a flowchart of a method for determining posture data provided by an embodiment of the present application.

[0018] Figure 4 This is a flowchart of a method for determining posture data provided by another embodiment of the present application.

[0019] Figure 5 This is a flowchart of a method for determining first size data provided by an embodiment of the present application.

[0020] Figure 6 This is a flowchart of a method for determining dimensional error provided by another embodiment of the present application.

[0021] Figure 7 This is a flowchart of a method for determining a scan path provided by another embodiment of the present application.

[0022] Figure 8 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to more clearly understand the purpose, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In the following description, many specific details are set forth to facilitate a full understanding of the present application. The embodiments described are only a part of the embodiments of the present application, rather than all of the embodiments.

[0024] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the described features. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0026] An embodiment of the present application provides a scan path optimization method that can be applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0027] An electronic device can be any electronic product that can interact with a user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, Internet Protocol Television (IPTV), smart wearable device, etc.

[0028] The electronic device may also include a network device and / or a client device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0029] The networks where electronic devices are located include but are not limited to the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0030] like Figure 1As shown, the present application provides a scanning path optimization system 10 including an electronic device 100 and a scanning device 200. The electronic device 100 is configured to acquire multiple frames of image data obtained when the scanning device 200 scans an object 300 to be scanned. The electronic device 100 also determines, based on any frame of image data, the pose data of the scanning device 200 when acquiring any frame of image data. The electronic device 100 also constructs a virtual object with the same physical properties as the object 300 to be scanned based on digital twin technology, and scans the virtual object based on the pose data corresponding to any frame of image data to obtain point cloud data of the virtual object. The electronic device 100 also performs three-dimensional reconstruction of the virtual object based on the point cloud data to obtain first dimension data of the virtual object corresponding to any frame of pose data. The electronic device 100 also determines the dimensional error corresponding to any frame of pose data based on the first dimension data and the second dimension data of the object to be scanned, and determines the scanning path of the scanning device 200 from the plurality of pose data based on the dimensional errors corresponding to the plurality of pose data. In this way, digital twin technology can be implemented based on the electronic device 100 to simulate the operation of the scanning device 200 scanning the object 300 to be scanned, thereby optimizing the scanning path of the scanning device 200 based on the digital twin technology, and improving the accuracy of the point cloud data obtained when the scanning device 200 performs the scanning task based on the scanning path.

[0031] like Figure 2 , is a flow chart of a scan path optimization method provided in one embodiment of the present application. Depending on different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. A scan path optimization method provided in one embodiment of the present application includes the following steps.

[0032] S20: Acquire multiple frames of image data obtained when the scanning device scans the object to be scanned.

[0033] In one embodiment of the present application, in order to simulate the scanning path of a scanning device scanning an object to be scanned in an electronic device based on a digital twin method, and determine the scanning path with the highest accuracy to improve the accuracy of three-dimensional reconstruction of the object to be scanned, multiple frames of image data obtained when the scanning device scans the object to be scanned can be first obtained.

[0034] In one embodiment of the present application, when a scanning device is used to scan an object to be scanned, the resulting multiple frames of images may be spatial sequence images. For example, the spatial sequence images may be medical imaging, used to indicate spatial tomographic slices, and may be three-dimensional data sets generated along different angles or layer thicknesses (e.g., axial, sagittal, and coronal planes). Spatial sequence images may also be industrial three-dimensional scanning images, which are generated by laser or structured light scanning to generate two-dimensional projections at multiple angles for three-dimensional reconstruction. Spatial sequence images may also be geological exploration (e.g., seismic scanning): reflection signals from strata collected at different depths or locations form a two-dimensional profile sequence.

[0035] In one embodiment of the present application, when a scanning device is used to scan an object to be scanned, the multi-frame images obtained may also be time-series images. For example, time-series images may be images obtained from dynamic process monitoring (such as industrial assembly line inspection, medical angiography), which continuously capture multiple frames of images to record changes in the object over time (such as deformation, fluid movement). Time-series images may also be video scans, where a video stream is generated by an ordinary camera or a high-speed camera, with each frame being an independent image. Time-series images may also be multi-parameter / multi-modal images, such as medical MRI, which generates multi-frame images of different contrasts by adjusting scanning parameters (such as T1 and T2 weighting). Time-series images may also be spectral scans, which record reflection or transmission data at different wavelengths to form multi-channel spectral images.

[0036] In one embodiment of the present application, multiple image frames contain both basic pixel and voxel data. For example, basic pixel values ​​can be grayscale values, representing the density or reflective intensity of the scanned object. Basic pixel values ​​can also be RGB values, representing the pixel color information of color scanned objects (e.g., documents or artwork). Voxel data can be depth values, representing the spatial coordinates of each point in the point cloud data obtained by 3D scanning.

[0037] In one embodiment of the present application, the multi-frame image also includes geometric and topological information. For example, the multi-frame image may include point cloud data, which indicates a set of three-dimensional coordinates of discrete points in a 3D scan. The multi-frame image may also include a mesh model, which is a three-dimensional surface composed of triangular or quadrilateral facets. The multi-frame image may also include contour lines, which are edge or boundary information extracted from a two-dimensional scan.

[0038] S21, according to any frame of image data, determining the posture data when the scanning device acquires the any frame of image data.

[0039] In one embodiment of the present application, since any frame of image output by the scanning device can indicate the position of the object to be scanned within the imaging range of the scanning device, the position information of the scanning device in the world coordinate system can be determined based on the position information of the object to be scanned in the world coordinate system and the position information of the object to be scanned in any frame of image, and then the posture data of the scanning device when acquiring any frame of image data can be determined.

[0040] In one embodiment of the present application, when a scanning device is used to scan an object to be scanned, the posture data of the scanning device may be a set of parameters used to describe the position and posture of the scanning device in three-dimensional space. The posture data can affect the accuracy of the scanning process, the registration and fusion of multiple image frames, and the reconstruction of the final three-dimensional model. Specifically, the posture data generally includes position data and posture data. The position data includes three-dimensional coordinates, which are used to represent the specific position of the scanning device in space, and are generally represented by a Cartesian coordinate system (X, Y, Z). The position data also includes a reference coordinate system, such as a global coordinate system (e.g., the world coordinate system or the origin of the scanning scene) and a local coordinate system (e.g., the device's own reference point or the position of the previous frame). The posture data includes rotation parameters that describe changes in the orientation of the scanning device, and also includes translation parameters that describe changes in the position of the scanning device.

[0041] Pose data can be used for registration and 3D reconstruction of multi-frame point cloud data. After converting point cloud data obtained from scans at different angles or positions into a unified coordinate system, errors in overlapping areas are eliminated and the pose data is used to merge the multi-frame point cloud data into a complete 3D model (for example, digital scanning of cultural relics).

[0042] In one embodiment of the present application, the specific method for determining the posture data when the scanning device obtains the arbitrary frame of image data according to the arbitrary frame of image data can be found in Figure 3 and Figure 4 Corresponding detailed description.

[0043] S22, scanning the virtual object based on the posture data corresponding to any frame of image data to obtain point cloud data of the virtual object; wherein the size data of the virtual object is the same as that of the object to be scanned.

[0044] In one embodiment of the present application, in order to simulate the scanning path of a scanning device for scanning an object to be scanned in an electronic device based on a digital twin method, and determine the scanning path with the highest accuracy to improve the accuracy of three-dimensional reconstruction of the object to be scanned, a virtual object can be scanned based on the posture data corresponding to any frame of image data to obtain point cloud data of the virtual object; wherein, the size data of the virtual object is the same as that of the object to be scanned.

[0045] In one embodiment of the present application, a virtual object can be a digital object generated from the object to be scanned using digital twin technology. Specifically, the virtual object is a precise mapping of the object to be scanned in digital space. Essentially, it is a digital model that integrates geometric, physical, material, and behavioral properties. To simulate the physical properties of the object to be scanned, such as the surface material and reflectivity, a 3D reconstruction or CAD model can be used to import the virtual object, ensuring that geometric parameters such as length, width, height, and surface curvature are strictly consistent with the object to be scanned. The virtual object's topology can also be configured to simulate the object's internal structure (e.g., holes, supports) and surface details (e.g., textures, microfeatures) to improve the accuracy of subsequent scanning path optimization. The virtual object's physical properties can also be parameterized by configuring its physical parameters. Physical properties include material properties, which define physical parameters such as density, elastic modulus, and thermal conductivity, providing a foundation for mechanical simulation or thermal analysis. Physical properties also include optical properties (e.g., diffuse reflectivity, specular reflectivity, roughness, refractive index, etc.), which can directly impact the quality of the point cloud data generated by the scanning device. Physical properties also include behavioral models to simulate the object's deformation (e.g., flexible deformation), motion (e.g., vibration), or environmental interaction (e.g., lighting changes) during scanning.

[0046] In one embodiment of the present application, to improve the accuracy of 3D reconstruction of virtual objects, parametric modeling of the virtual object's material properties can be performed based on the bidirectional reflectance distribution function to define the material's reflective characteristics at different incident and observation angles. For example, parametric modeling can be used to determine the diffuse reflection of the virtual object's surface to simulate the uniform scattering of a rough surface (e.g., matte plastic); the specular reflection of the virtual object's surface can be determined to simulate the specular highlights of a smooth surface (e.g., metal); and the anisotropic reflection of the virtual object's surface can be determined to simulate the directional reflection of brushed metal or fibrous materials.

[0047] In one embodiment of the present application, markers with known reflectivity are placed in a virtual scene. By comparing the scan data of the virtual markers with the real markers, material parameters can be determined. Optimization algorithms (e.g., genetic algorithms, gradient descent, etc.) can also be used to adjust the virtual material parameters to minimize the error between the virtual and real scan data.

[0048] In one embodiment of the present application, the surface microstructure of a virtual object can be simulated based on a preset simulation algorithm. For example, surface microscopic bumps and concavities can be simulated using a normal map to enhance the representation of geometric details. Random textures (such as wood rings and metal scratches) can also be generated based on an arbitrary texture generation algorithm to enhance material realism.

[0049] S23 , performing three-dimensional reconstruction on the virtual object according to the point cloud data to obtain first size data of the virtual object corresponding to any pose data.

[0050] In one embodiment of the present application, in order to determine the accuracy of the point cloud data obtained when scanning a virtual object according to any posture data, and then determine the accuracy of the scanning device in scanning the object to be scanned when it is in the state of the posture data, the virtual object can be three-dimensionally reconstructed according to the point cloud data corresponding to any posture data to obtain the first size data of the virtual object corresponding to any posture data.

[0051] In one embodiment of the present application, the first dimension data may be used to indicate the size of the virtual object. For example, when the object to be scanned and the virtual object are spheres, the first dimension data may be the radius or diameter of the virtual object. When the object to be scanned and the virtual object are cubes, the first dimension data may be the side length of the virtual object. When the object to be scanned and the virtual object are irregular objects, the first dimension data may be the distance from the center of mass of the virtual object to any point on the surface of the virtual object.

[0052] In one embodiment of the present application, the method for performing three-dimensional reconstruction of the virtual object based on the point cloud data to obtain the first size data of the virtual object corresponding to any pose data can be found in Figure 5 Corresponding detailed description.

[0053] S24: Determine a size error corresponding to any one of the posture data according to the first size data and the second size data of the object to be scanned.

[0054] In one embodiment of the present application, the second dimension data of the object to be scanned is used to indicate the size of the object to be scanned. For example, when the object to be scanned and the virtual object are spheres, the second dimension data may be the radius of the object to be scanned or the diameter of the object to be scanned. When the object to be scanned and the virtual object are cubes, the second dimension data may be the side length of the object to be scanned. When the object to be scanned and the virtual object are irregular objects, the second dimension data may be the distance from the center of mass of the object to be scanned to any point on the surface of the object to be scanned.

[0055] In one embodiment of the present application, in order to determine the difference between the three-dimensional reconstructed data obtained by simulating the scanning of a virtual object based on digital twin technology and the real object to be scanned, the size error can be determined based on the first size data and the second size data of the object to be scanned. Among them, any posture data corresponds to a size error. The size error is used to characterize the error between the size of the virtual object obtained when the virtual object is scanned with the posture data based on the digital twin technology and the real size. In the case of a larger size error, it indicates that when the scanning device scans the object to be scanned based on the posture data, the accuracy of the point cloud data obtained is lower, and the accuracy of the scanning result obtained when the scanning device performs the scanning task based on the posture data is lower; in the case of a smaller size error, it indicates that when the scanning device scans the object to be scanned based on the posture data, the accuracy of the point cloud data obtained is higher, and the accuracy of the scanning result obtained when the scanning device performs the scanning task based on the posture data is higher.

[0056] In one embodiment of the present application, the specific method for determining the size error corresponding to any one of the pose data according to the first size data and the second size data of the object to be scanned is shown in FIG. Figure 6 Corresponding detailed description.

[0057] S25 , determining a scanning path of the scanning device from the plurality of posture data according to the size errors corresponding to the plurality of posture data.

[0058] In one embodiment of the present application, the scanning path corresponding to the scanning device can be determined based on the dimensional errors corresponding to the multiple posture data. The scanning path is a collection of multiple posture data. In order to improve the accuracy of the scanning device when scanning the object to be scanned along the scanning path, the quality of each posture can be evaluated based on the dimensional error corresponding to each posture data. When the dimensional error corresponding to any posture data is smaller than the dimensional error corresponding to the adjacent posture data, it indicates that the accuracy of the point cloud data obtained by scanning the object to be scanned when any scanning device is in this posture data is higher. Therefore, the scanning path of the scanning device can be determined from the multiple posture data based on the dimensional errors corresponding to the multiple posture data.

[0059] In one embodiment of the present application, the specific method of determining the scanning path of the scanning device from the multiple posture data according to the size error corresponding to the multiple posture data can be found in Figure 7 Corresponding detailed description.

[0060] It can be seen from the above technical solutions that the embodiment of the present application can quantitatively characterize the impact of different postures on scanning blind spots and data overlap rates through three-dimensional reconstruction error analysis of virtual objects, generate a path with complete coverage and reasonable steps, and reduce invalid scanning actions. Dynamically adjust the scanning density according to the geometric features of the object to be scanned to improve data integrity. Replace actual repeated scanning with virtual object simulation of digital twins to reduce equipment wear, energy consumption and the cost of manual intervention. By comparing the virtual reconstructed size (first size data) with the actual size of the object to be scanned (second size data), the source of the size error in the posture data dimension is located, and closed-loop correction can be achieved in the process of planning the scanning path. In addition, environmental interference (for example, vibration, lighting changes) can be preset in the virtual environment, and the scanning path can be optimized to avoid high-noise areas, improve the signal-to-noise ratio of the actual scanning data, and thus improve the accuracy of the scanning device when scanning according to the scanning path.

[0061] like Figure 3 , is a flow chart of a method for determining posture data provided in an embodiment of the present application. Depending on different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for determining posture data provided in an embodiment of the present application includes the following steps.

[0062] S30: Determine the image coordinates of the marker points of the object to be scanned according to the arbitrary frame of image data.

[0063] In one embodiment of the present application, in order to improve the accuracy of determining the position of the object to be scanned, the image coordinates of the marker point of the object to be scanned can be determined based on any frame of image data. The marker point of the object to be scanned is used to assist in positioning the object to be scanned. Specifically, the marker point can be any point on the surface of the object to be scanned, or any point whose relative position to the object to be scanned is fixed, and this application does not limit this. For example, the marker point can be a black dot affixed to the surface of the object to be scanned, or a black dot whose relative position to the object to be scanned is fixed.

[0064] In one embodiment of the present application, the position of the marker point of the object to be scanned in the image can be determined based on the amount of change in the pixel value in the image data. For example, when the amount of change in the grayscale value of one or more pixels in any image data is large, it indicates that there is a large grayscale value difference at one or more pixels, so the one or more pixels can be determined as marker points, and the image coordinates of the marker point can be determined based on the image coordinates of the one or more pixels. For example, when the amount of change in the grayscale value of the pixel at the 10th row and 10th column in the image data is large, it can be determined that the pixel at the 10th row and 10th column represents the marker point, and the image coordinates of the marker point can be determined to be (10, 10).

[0065] S31 , determining, according to the image coordinates and the world coordinates of the object to be scanned, the posture data when the scanning device acquires the arbitrary frame of image data.

[0066] In one embodiment of the present application, the world coordinates of the object to be scanned are used to indicate the position of the object to be scanned in a real-world world coordinate system. The world coordinate system can be any pre-set coordinate system. For example, the initial position of the scanning device when it is started can be set as the origin of the world coordinate system, the direction perpendicular to the sea level can be set as the Z axis of the world coordinate system, the east direction can be set as the X axis of the world coordinate system, and the north direction can be set as the Y axis of the world coordinate system.

[0067] In one embodiment of the present application, the world coordinates of the marker points can be determined based on the world coordinates of the object to be scanned and the relative positional relationship between the object to be scanned and the marker points. By determining the mapping relationship between the world coordinates of the marker points and the image coordinates of the marker points, the pose data of any frame of image data acquired by the scanning device can be determined.

[0068] like Figure 4 , is a flow chart of a method for determining posture data provided in an embodiment of the present application. Depending on different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for determining posture data provided in an embodiment of the present application includes the following steps.

[0069] S40 , determining the device coordinates of the marker point in the device coordinate system of the scanning device according to the internal reference data of the scanning device and the image coordinates.

[0070] In one embodiment of the present application, when determining the mapping relationship between the world coordinates of the marker point and the image coordinates of the marker point to determine the posture data when the scanning device acquires any frame of image data, the device coordinates of the marker point in the device coordinate system of the scanning device can be first determined based on the internal reference data of the scanning device and the image coordinates of the marker point.

[0071] In one embodiment of the present application, the intrinsic parameter data of a scanning device is used to indicate the imaging parameters of the scanning device. This intrinsic parameter data includes optical resolution, which indicates the actual resolution achievable by the scanning device and is expressed in dots per inch (DPI), such as 600×1200 DPI. Specifically, optical resolution includes horizontal and vertical resolution, which characterizes the clarity of the scanned image and is a core indicator of scanning quality. The intrinsic parameter data also includes color bit count, which indicates the richness of color information the scanning device can capture. The intrinsic parameter data also includes dynamic range, which indicates the scanning device's ability to display tonal variations. The intrinsic parameters of the scanning device also include camera intrinsic parameters. Specifically, these include focal length, which indicates the distance from the lens optical center to the imaging plane; principal point position, which indicates the coordinates of the intersection of the optical axis and the imaging plane; and distortion coefficient, which is used to correct lens distortion (such as barrel and pincushion distortion). The intrinsic parameter data of the scanning device also includes scanning speed, which indicates the time required for the scanning device to complete a scan and characterizes the scanning device's operating efficiency.

[0072] S41, determining the world coordinates of the marker point according to the world coordinates of the object to be scanned and a pre-stored coordinate difference; wherein the pre-stored coordinate difference is used to indicate the coordinate difference between the object to be scanned and the marker point.

[0073] In one embodiment of the present application, a pre-stored coordinate difference is used to indicate the relative positional relationship between the scanned object and the marker. Specifically, the coordinate difference may be the difference between the world coordinates of the scanned object and the world coordinates of the marker. For example, the coordinate difference may be in the form of "X-axis coordinate difference 3, Y-axis coordinate difference 3, Z-axis coordinate difference 3."

[0074] In one embodiment of the present application, the world coordinates of the marker point can be determined based on the coordinate difference and the world coordinates of the object to be scanned. For example, if the world coordinates of the object to be scanned are (x=10, y=10, z=10), and the coordinate difference is "X-axis coordinate difference of 3, Y-axis coordinate difference of 3, Z-axis coordinate difference of 3", the world coordinates of the marker point can be determined to be (x=13, y=13, z=13).

[0075] S42, determining the posture data of the scanning device by calculating the mapping relationship between the device coordinates and the world coordinates of the marker point.

[0076] In one embodiment of the present application, in order to determine the pose data of the scanning device when acquiring image data, the mapping relationship between the device coordinates and the world coordinates can be determined based on the device coordinates of the marker point and the world coordinates of the marker point. The pose data of the scanning device includes translation parameters and rotation parameters. Specifically, the method for calculating the mapping relationship between the device coordinates and the world coordinates satisfies the following relationship: ; Among them, R is the rotation parameter in the pose data, T is the translation parameter in the pose data; X C is the X-axis coordinate of the marker point in the device coordinates, C is the Y-axis coordinate of the marker point in the device coordinates, Z C is the Z-axis coordinate of the marker point in the device coordinates; X W is the X-axis coordinate of the marker point in the world coordinates, Y W is the Y-axis coordinate of the marker point in the world coordinates, Z W is the Z-axis coordinate of the marker point in the world coordinates. After determining the device coordinates and world coordinates of the marker point, the device coordinates and world coordinates can be substituted into the above relationship to obtain the rotation parameters and translation parameters in the pose parameters.

[0077] like Figure 5 1 is a flowchart of a method for determining first size data provided in an embodiment of the present application. Depending on different requirements, the order of the steps in the flowchart may be changed, and some steps may be omitted. The method for determining first size data provided in an embodiment of the present application includes the following steps.

[0078] S50, performing point cloud segmentation on the point cloud data to determine a first point cloud and a second point cloud; wherein the first point cloud is used to indicate point cloud data belonging to the virtual object.

[0079] In one embodiment of the present application, to determine the size of a virtual object, a first point cloud and a second point cloud in the point cloud data may be first determined. The first point cloud indicates point cloud data belonging to the virtual object. The size of the virtual object is then determined based on the coordinates of the first point cloud and the coordinates of the center of mass of the virtual object.

[0080] In one embodiment of the present application, neighborhood statistical characteristics (e.g., mean distance, standard deviation) of each point cloud in the point cloud data can be calculated to remove outliers that deviate significantly from the mean. The point cloud data space is then divided into voxel grids, and the center of gravity within the voxel grids is used to replace the original points, reducing the amount of point cloud data. The normal direction or curvature of each point cloud is then calculated, and an angle threshold (e.g., normal angle > 60°) is used to distinguish different planes or curved surfaces in the point cloud data. A model fitting algorithm (e.g., RANSAC or Euclidean clustering) is used to fit a basic geometric model of a virtual object (e.g., a plane or cylinder) to extract point clouds that conform to the model. Semantic segmentation can also be performed on the point clouds in the point cloud data using a deep learning algorithm to determine the semantics of each point cloud in the point cloud data. For example, the semantics of a point cloud can be either "virtual object" or "background." If the semantics of any point cloud is "virtual object," it can be identified as the first point cloud; if the semantics of any point cloud is "background," it can be identified as the second point cloud. Among them, the deep learning algorithm can be a neural network algorithm with point cloud classification function such as PointNet, PointNet++ or RandLA-Net. This application does not limit the specific type of deep learning algorithm.

[0081] S51 : Determine first size data of the virtual object based on the virtual coordinates of the first point cloud and the virtual coordinates of the virtual object.

[0082] In one embodiment of the present application, the virtual coordinates of the first point cloud are used to indicate the specific location of the point cloud on the surface of the virtual object, and the virtual coordinates of the virtual object are used to indicate the specific location of the center of mass of the virtual object. To determine the size of the virtual object, the Euclidean distance between the virtual coordinates of the first point cloud and the virtual coordinates of the virtual object can be determined to obtain first size data of the virtual object. The first size data indicates the size of the virtual object.

[0083] like Figure 6 , which is a flow chart of a method for determining dimensional error according to an embodiment of the present application. Depending on different requirements, the order of the steps in the flow chart may be changed, and some steps may be omitted. The method for determining dimensional error according to an embodiment of the present application includes the following steps.

[0084] S60: When there are multiple first point clouds, determine a size error corresponding to any one of the first point clouds based on the first size data and the second size data corresponding to the any one of the first point clouds.

[0085] In one embodiment of the present application, if there are multiple first point clouds, this indicates that multiple point clouds of the virtual object's surface were collected when scanning the virtual object using digital twin technology. To improve the accuracy of determining the size of the virtual object, the size error corresponding to any one of the first point clouds can be determined based on the first size data and second size data corresponding to any one of the first point clouds.

[0086] S61: Determine the size error corresponding to any one of the pose data according to the mean of all size errors.

[0087] In one embodiment of the present application, in order to improve the accuracy of determining the size error, the size error corresponding to any one of the posture data may be determined based on the average of all the size errors.

[0088] like Figure 7 , is a flow chart of a method for determining a scanning path provided in an embodiment of the present application. Depending on different requirements, the order of the steps in the flow chart may be changed, and some steps may be omitted. The method for determining a scanning path provided in an embodiment of the present application includes the following steps.

[0089] S70, determining a plurality of candidate pose data corresponding to any one pose data from the plurality of pose data according to a preset step size parameter.

[0090] In one embodiment of the present application, in order to ensure that the scanning path determined from multiple posture data is the optimal scanning path, the dimensional error corresponding to each posture data can be evaluated, and the scanning path can be constructed based on multiple posture data with smaller dimensional errors.

[0091] In one embodiment of the present application, the step size parameter includes a rotation step size and / or a translation step size; wherein the rotation step size is used to indicate the angle of rotation of the scanning device, and the translation step size is used to indicate the distance the scanning device moves.

[0092] In one embodiment of the present application, the pose data can be traversed in sequence according to the order in which the scanning device scans the virtual object, and multiple pose data adjacent to any pose data can be determined as candidate pose data. The number of candidate poses is determined by the step size parameter. For example, when the rotation step size is 10 degrees, other pose data that differ from any pose data by a rotation angle of 10 degrees can be determined as candidate pose data corresponding to the any pose data; when the translation step size is 10 mm, other pose data that differ from any pose data by 10 mm can be determined as candidate pose data corresponding to the any pose data.

[0093] S71, determining the candidate pose data with the smallest size error from the multiple candidate pose data, and obtaining the target pose data corresponding to any one of the pose data.

[0094] In one embodiment of the present application, when the dimensional error corresponding to any candidate pose data is minimized, it indicates that the accuracy of the point cloud data obtained by the scanning device when scanning the object under the candidate pose data is high. Therefore, the candidate pose data with the minimized dimensional error can be determined as the target pose data corresponding to the candidate pose data.

[0095] S72: Determine a scanning path of the scanning device according to target posture data corresponding to all posture data.

[0096] In one embodiment of the present application, after determining the target posture data corresponding to all posture data, the target posture data corresponding to all posture data can be combined to obtain a scanning path of the scanning device.

[0097] See Figure 8 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Electronic device 100 includes a memory 12 and a processor 13. Memory 12 is used to store computer-readable instructions, and processor 13 is used to execute the computer-readable instructions stored in the memory to implement a scan path optimization method described in any of the above embodiments.

[0098] In an embodiment of the present application, the electronic device 100 further includes a bus, and a computer program stored in the memory 12 and executable on the processor 13 , such as a scan path optimization program.

[0099] Figure 8 Only the electronic device 100 having the memory 12 and the processor 13 is shown. It can be understood by those skilled in the art that Figure 8 The structure shown does not constitute a limitation on the electronic device 100 , and the electronic device 100 may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0100] Combine Figure 2, the memory 12 in the electronic device 100 stores multiple computer-readable instructions to implement a scanning path optimization method, and the processor 13 can execute the multiple instructions to achieve: obtaining multiple frames of image data obtained when the scanning device scans the object to be scanned; determining the posture data of the scanning device when obtaining the any frame of image data based on any frame of image data; scanning the virtual object based on the posture data corresponding to any frame of image data to obtain point cloud data of the virtual object; wherein the size data of the virtual object is the same as that of the object to be scanned; performing three-dimensional reconstruction on the virtual object based on the point cloud data to obtain first size data of the virtual object corresponding to any pose data; determining the size error corresponding to any pose data based on the first size data and the second size data of the object to be scanned; and determining the scanning path of the scanning device from the multiple pose data based on the size errors corresponding to the multiple pose data.

[0101] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 2 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0102] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 100 and does not constitute a limitation on the electronic device 100. The electronic device 100 may have a bus structure or a star structure. The electronic device 100 may also include more or less other hardware or software than shown in the figure, or a different arrangement of components. For example, the electronic device 100 may also include input and output devices, network access devices, etc.

[0103] It should be noted that the electronic device 100 is only an example. Other existing or future electronic products that are suitable for this application should also be included in the scope of protection of this application and incorporated herein by reference.

[0104] The memory 12 includes at least one type of readable storage medium, which can be either non-volatile or volatile. The readable storage medium includes a flash memory, a removable hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 100, such as a removable hard disk of the electronic device 100. In other embodiments, the memory 12 can also be an external storage device of the electronic device 100, such as a plug-in removable hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. The memory 12 can be used not only to store application software installed in the electronic device 100 and various types of data, such as the code of a scan path optimization program, but can also be used to temporarily store data that has been output or is about to be output.

[0105] In some embodiments, the processor 13 may be comprised of an integrated circuit, such as a single packaged integrated circuit or a combination of multiple packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control core (control unit) of the electronic device 100, connecting the various components of the electronic device 100 using various interfaces and circuits. It executes programs or modules stored in the memory 12 (e.g., a scan path optimization program) and accesses data stored in the memory 12 to perform various functions and process data.

[0106] The processor 13 executes the operating system of the electronic device 100 and various installed applications. The processor 13 executes the applications to implement the steps in each of the above-mentioned scanning path optimization method embodiments, for example Figure 2 Steps shown.

[0107] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device 100.

[0108] The above-mentioned integrated unit implemented in the form of a software functional module can be stored in a computer-readable storage medium. The above-mentioned software functional module stored in a storage medium includes a number of instructions for causing a computer device (which can be a personal computer, computer device, or network device, etc.) or a processor to execute a portion of a scan path optimization method described in various embodiments of the present application.

[0109] If the modules / units integrated in the electronic device 100 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment methods, and can also instruct the relevant hardware devices to complete them through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments.

[0110] The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, and other memory.

[0111] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0112] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The diagram is represented by only one arrow, but it does not mean that there is only one bus or one type of bus. The bus is configured to implement connection and communication between the memory 12 and at least one processor 13, etc.

[0113] An embodiment of the present application also provides a computer-readable storage medium (not shown), in which computer-readable instructions are stored. The computer-readable instructions are executed by a processor in an electronic device to implement a scan path optimization method described in any of the above embodiments.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0115] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0116] In addition, the functional modules in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0117] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices listed in the specification may also be implemented by a single unit or device through software or hardware. Terms such as first and second are used to indicate names and do not imply any particular order.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A scanning path optimization method, applied to electronic equipment, characterized in that: The electronic device is communicatively connected to a scanning device, the scanning device is used to scan an object to be scanned, and the method includes: Acquiring multiple frames of image data obtained when the scanning device scans the object to be scanned; Determining, based on any frame of image data, the posture data when the scanning device acquires the any frame of image data; Scanning a virtual object based on the posture data corresponding to any frame of image data to obtain point cloud data of the virtual object; wherein the size data of the virtual object is the same as that of the object to be scanned; Performing three-dimensional reconstruction on the virtual object according to the point cloud data to obtain first size data of the virtual object corresponding to any pose data; determining a size error corresponding to any one of the pose data according to the first size data and the second size data of the object to be scanned; According to the dimensional errors corresponding to the multiple posture data, a scanning path of the scanning device is determined from the multiple posture data.

2. The scanning path optimization method according to claim 1, wherein: The determining, based on any frame of image data, the posture data when the scanning device acquires the any frame of image data comprises: Determining the image coordinates of the marker point of the object to be scanned according to the arbitrary frame of image data; The position and posture data of the scanning device when acquiring the arbitrary frame of image data are determined according to the image coordinates and the world coordinates of the object to be scanned.

3. The scanning path optimization method according to claim 2, wherein: The determining, based on the image coordinates and the world coordinates of the object to be scanned, the posture data when the scanning device acquires the arbitrary frame of image data comprises: determining the device coordinates of the marker point in the device coordinate system of the scanning device according to the internal reference data of the scanning device and the image coordinates; Determining the world coordinates of the marker point according to the world coordinates of the object to be scanned and the pre-stored coordinate difference; wherein the pre-stored coordinate difference is used to indicate the coordinate difference between the object to be scanned and the marker point; The position and posture data of the scanning device are determined by calculating the mapping relationship between the device coordinates and the world coordinates of the marker points.

4. The scanning path optimization method according to claim 1, wherein: The three-dimensional reconstruction of the virtual object according to the point cloud data to obtain first size data of the virtual object corresponding to any pose data includes: Performing point cloud segmentation on the point cloud data to determine a first point cloud and a second point cloud; wherein the first point cloud is used to indicate point cloud data belonging to the virtual object; First size data of the virtual object is determined based on the virtual coordinates of the first point cloud and the virtual coordinates of the virtual object.

5. The scanning path optimization method according to claim 4, wherein: The determining, based on the first size data and the second size data of the object to be scanned, a size error corresponding to any one of the pose data comprises: In the case where there are multiple first point clouds, determining a size error corresponding to any one of the first point clouds based on the first size data and the second size data corresponding to the any one of the first point clouds; The size error corresponding to any one of the pose data is determined according to the mean of all size errors.

6. The scanning path optimization method according to claim 1, wherein: Determining the scanning path of the scanning device from the plurality of posture data according to the size errors corresponding to the plurality of posture data includes: Determining, according to a preset step size parameter, a plurality of candidate pose data corresponding to any one pose data from the plurality of pose data; Determine the candidate pose data with the smallest size error from the multiple candidate pose data, and obtain the target pose data corresponding to any one of the pose data; The scanning path of the scanning device is determined according to the target posture data corresponding to all the posture data.

7. The scanning path optimization method according to claim 6, wherein: The method further includes: the step size parameter includes a rotation step size and / or a translation step size; wherein the rotation step size is used to indicate the rotation angle of the scanning device, and the translation step size is used to indicate the movement distance of the scanning device.

8. A scanning path optimization system, characterized in that: The system includes: a scanning device and an electronic device; the scanning device is used to scan an object to be scanned; The electronic device is used to obtain multiple frames of image data obtained when the scanning device scans the object to be scanned; determine the posture data of the scanning device when obtaining the arbitrary frame of image data based on any frame of image data; scan the virtual object based on the posture data corresponding to any frame of image data to obtain point cloud data of the virtual object; wherein, the virtual object has the same size data as the object to be scanned; perform three-dimensional reconstruction on the virtual object based on the point cloud data to obtain first size data of the virtual object corresponding to any pose data; determine the size error corresponding to any pose data based on the first size data and the second size data of the object to be scanned; and determine the scanning path of the scanning device from the multiple pose data based on the size errors corresponding to the multiple pose data.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, and the processor is configured to implement the scan path optimization method according to any one of claims 1 to 7 when executing a computer program stored in the memory.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the scan path optimization method according to any one of claims 1 to 7 is implemented.

Images