Scanning processing method and device, equipment and medium

Abstract

The invention discloses a scanning processing method and device, equipment and a medium. The method comprises the steps that a first scanning frame and first splicing data of a scanned object are acquired; carrying out first sequence splicing on the first scanning frame and the first splicing data; and if a first sequence splicing result obtained by the first sequence splicing does not meet a preset requirement, re-positioning a target area in the first splicing data for splicing to obtain an intermediate splicing result, and continuously obtaining a verification scanning frame set based on the intermediate splicing result for re-splicing verification, or storing the first splicing data to obtain a second sequence splicing result. And generating new second splicing data based on the first scanning frame. According to the method, the spelling-back error caused by similar characteristics or physiological jitter can be effectively identified and corrected, so that the accuracy of scanning processing is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of scanning technology, and in particular to a scanning processing method, apparatus, device, and medium. Background Technology

[0002] In the field of 3D scanning and real-time modeling, especially in medical applications such as dental restoration and implantology, tracking loss during the scanning process is a key issue leading to data frame loss and real-time stitching interruptions. While feature-based re-assembly algorithms can reposition the camera pose to some extent, in special cases such as edentulous jaws and immediate implantation, the physiological jitter of gingival tissue and the high similarity of local geometric features greatly increase the risk of re-assembly errors, resulting in low accuracy in scanning processing. Existing technologies mainly attempt to correct errors by adjusting re-assembly parameters or introducing AI verification. However, parameter adjustment methods are difficult to cover the individualized situations of various complex cases and lack universality; while AI verification methods have low accuracy, especially in uncommon cases. In addition, AI verification methods also suffer from long computation time, high computing power requirements, and high costs for training data collection and annotation. Therefore, the reliability of existing scanning processing is low. Summary of the Invention

[0003] This invention provides a scanning processing method, apparatus, device, and medium, aiming to solve the problem of low accuracy in existing scanning processing methods.

[0004] In a first aspect, embodiments of the present invention provide a scanning processing method, including: Acquire the first scan frame and the first stitched data of the scanned object; The first scan frame and the first spliced ​​data are spliced ​​together in a first order; If the first sequence stitching result obtained does not meet the preset requirements, the target region is repositioned in the first stitching data and stitched to obtain an intermediate stitching result. Based on the intermediate stitching result, a set of verification scan frames is continuously acquired for back-stitching verification. Alternatively, the first stitching data is saved, and new second stitching data is generated based on the first scan frame.

[0005] Secondly, embodiments of the present invention also provide a scanning processing apparatus, comprising: The acquisition unit is used to acquire the first scan frame and the first stitched data of the scanned object. A splicing unit is used to splice the first scan frame and the first splicing data in a first order; The back-assembly verification unit is used to, if the first-order assembly result obtained by the first-order assembly does not meet the preset requirements, relocate the target area in the first-order assembly data and perform assembly to obtain an intermediate assembly result, and continuously acquire a set of verification scan frames based on the intermediate assembly result for back-assembly verification, or save the first-order assembly data and generate new second-order assembly data based on the first scan frame.

[0006] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-described method.

[0007] Fourthly, embodiments of the present invention also provide an electronic device readable storage medium, the storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0008] This invention provides a scanning processing method, apparatus, device, and medium. The method includes: acquiring a first scan frame and first stitched data of a scanned object; The first scan frame and the first stitched data are stitched together in a first order. If the first stitched result does not meet the preset requirements, the target region is repositioned in the first stitched data and stitched together to obtain an intermediate stitched result. Based on the intermediate stitched result, a set of verification scan frames is continuously acquired for back-stitching verification. Alternatively, the first stitched data is saved, and new second stitched data is generated based on the first scan frame. This technical solution of the present invention, when the first stitched result obtained by stitching the first scan frame and the first stitched data does not meet the preset requirements, effectively identifies and corrects back-stitching errors caused by similar features or physiological jitter by repositioning the target region in the first stitched data and stitching together to obtain an intermediate stitched result. Based on the intermediate stitched result, a set of verification scan frames is continuously acquired for back-stitching verification. Alternatively, the first stitched data is saved, and new second stitched data is generated based on the first scan frame. This significantly improves the accuracy of the scanning process. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic flowchart of a scanning processing method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a sub-process of a scanning processing method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of another sub-process of a scanning processing method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of another sub-process of a scanning processing method provided in an embodiment of the present invention; Figure 5 This is a schematic block diagram of a scanning processing device provided in an embodiment of the present invention; Figure 6 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0013] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0014] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0015] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0016] This invention proposes a scanning processing method, apparatus, device, and medium to address the problem of low accuracy in existing scanning processing. In this embodiment, when the first sequential stitching result obtained by stitching the first scan frame of the scanned object with the first stitching data does not meet the preset requirements, an intermediate stitching result is obtained by repositioning the target region in the first stitching data and stitching it. Based on the intermediate stitching result, a set of verification scan frames is continuously acquired for re-stitching verification. Alternatively, the first stitching data is saved, and new second stitching data is generated based on the first scan frame. This effectively identifies and corrects re-stitching errors caused by similar features or physiological jitter, thereby significantly improving the accuracy of scanning processing.

[0017] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0018] This invention proposes a scanning processing method applicable to 3D scanners such as dental scanners, facial scanners, industrial scanners, professional scanners, handheld scanners, and fixed scanners. It can realize 3D reconstruction of objects or scenes such as teeth, faces, bodies, industrial products, industrial equipment, cultural relics, artworks, prostheses, medical devices, and buildings. The electronic device can be understood as, for example, a 3D scanner, mobile phone, tablet computer, laptop computer, desktop computer, smart TV, etc.

[0019] Optionally, the scanned object includes at least one scanning body. In one example, the scanned object can be a patient's oral cavity, the patient can be an edentulous patient or a patient with partial tooth loss, and the scanning body can be a standard part or a scanned object with the same geometric features, such as an implant rod, scanning cap, abutment, etc. In another example, the scanned object can also be industrial equipment or components in an industrial setting, and the scanning body can be a standard part or a scanned object with the same geometric features, such as a bolt. This application can avoid the reassembly error of these standard parts or objects with the same geometric features.

[0020] Please refer to Figure 1 , Figure 1 A flowchart illustrating the scanning processing method according to an embodiment of the present invention is shown, as follows: Figure 1 As shown, the scanning processing method includes steps S110-S130.

[0021] S110. Obtain the first scan frame and the first stitched data of the scanned object.

[0022] In some embodiments, the object being scanned may include any object for which data stitching needs to be performed. In some embodiments, a first scan frame of the object being scanned may be obtained in response to a user's scanning operation using a scanner. The first scan frame represents the latest scan data (current scan frame) obtained from scanning the object. For example, if the object being scanned is a patient's oral cavity, the first scan frame may represent the latest captured image frame or point cloud frame obtained from scanning the patient's oral cavity.

[0023] In some embodiments, the first stitching data includes: a first stitching model (or a first reconstructed model), and / or, the last frame of the first stitching model, and / or, the region where the last frame of the first stitching model is located. The first stitching model represents a 3D model stitched (or reconstructed) before obtaining the first scan frame. In one example, during the acquisition of the first stitching data, the process can respond to a user's scanning operation using a scanner. For example, the user can follow a preset scanning route, such as a route along a preset direction of the dental arch in the patient's mouth (e.g., from left to right), or freely choose a scanning route to obtain scan frames (e.g., point cloud frames) at each position along the scanning direction of the dental arch, and stitch scan frames (e.g., point cloud frames) at adjacent time points. As another example, along a preset scanning route of industrial equipment, the preset scanning route can follow the equipment outline, weld seam, or assembly boundary, or the operator can freely plan the scanning route, acquiring continuous point cloud frames and stitching adjacent frames in real time based on common features such as bolts and interfaces. In this way, the latest captured scan frame (such as a point cloud frame, which may contain new point cloud data) can be stitched together with the last frame of data in the existing stitching model to update or expand the existing 3D stitching model, thereby obtaining the first stitching data.

[0024] S120. The first scan frame and the first splicing data are spliced ​​together in the first order.

[0025] In some embodiments, the first scan frame and the first stitched data are sequentially stitched together, which may be a stitching method such as partial stitching. For example, the first scan frame is sequentially stitched together with the first reconstructed model, the last frame of the first reconstructed model, and the region where the last frame of the first reconstructed model is located.

[0026] In one example, taking the partial stitching of the first scan frame with the last frame of the first reconstructed model as an example, feature points of the first scan frame first_frame and the last frame last_frame can be extracted, and a preset matching algorithm can be used to determine the corresponding points that match each other between the feature points of the first_frame and the last_frame. The preset matching algorithm includes, but is not limited to, feature descriptor matching algorithms.

[0027] In one example, when determining the sequential stitching error, the distance from each point in the first_frame to the nearest point in the last_frame can be calculated, and the sequential stitching error can be determined based on the average of all calculated distances. In another example, if the first stitched data is a mesh model, the nearest distance between each point in the first_frame and the surface of the current stitched data can also be calculated, and the sequential stitching error can be determined based on the average of all nearest distances. This application does not impose specific limitations on the method for determining the sequential stitching error.

[0028] In some embodiments, the constructed first stitched data can integrate point cloud data acquired by the scanned object. Therefore, the first stitched data may include point cloud data corresponding to the scanned object.

[0029] S130. If the first-order stitching result obtained from the first-order stitching does not meet the preset requirements, the target region is repositioned in the first stitched data and stitched to obtain an intermediate stitching result. Based on the intermediate stitching result, a set of verification scan frames is continuously acquired for back-stitching verification. Alternatively, the first stitching data is saved, and new second stitching data is generated based on the first scan frames. In some embodiments, it can be determined whether the first-order stitching result obtained from the first-order stitching meets the preset requirements. In one example, the first-order stitching result can be evaluated in multiple dimensions, including but not limited to stitching accuracy, completeness, consistency, and cumulative error.

[0030] In some embodiments, the sequential stitching error between the first scan and the first stitched data can be determined based on the first sequential stitching result. If the sequential stitching error is greater than a preset error threshold, it is determined that the first sequential stitching result does not meet the preset requirements. If the first sequential stitching error is less than or equal to the preset error threshold, it is determined that the first sequential stitching result meets the preset requirements. The preset error threshold can be set according to actual needs, and this application does not impose specific limitations on it. In some embodiments, if the first sequential stitching result meets the preset requirements, the first sequential stitching result is used as the updated first stitched data. Using the first sequential stitching result as the updated first stitched data includes: fusing the feature point cloud corresponding to the first scan frame into the first stitched data; extracting feature points from the fused first stitched data to obtain a global feature point cloud; and integrating the global feature point cloud with the feature point cloud corresponding to the first stitched data before fusion to obtain the updated first stitched data.

[0031] In some embodiments, if the first-order stitching result obtained from the first-order stitching does not meet the preset requirements, the target region is repositioned in the first stitched data and stitched to obtain an intermediate stitching result. When repositioning the stitching region from the first stitched data, the feature similarity between each region in the first stitched data and the current scan frame can be determined, and the region corresponding to the feature similarity greater than the preset similarity threshold is taken as the target region. In one example, local features can be extracted from the first scan frame and the first stitched data. For example, local features include, but are not limited to, feature histograms, fast point feature histograms, key point features, etc. Based on the local features of the first scan frame and the first stitched data, a feature descriptor matching algorithm is used to determine the feature similarity between the local features of the first scan frame and the first stitched data, such as Euclidean distance, Mahalanobis distance, cosine similarity, Hamming distance, etc., and the region with a feature similarity greater than the preset similarity threshold is selected as the target region. The preset similarity threshold can be set according to actual needs, and this application does not impose specific restrictions on it.

[0032] Please see Figure 2 , Figure 2 This is a schematic diagram of a sub-process of a scanning processing method provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the steps for continuously acquiring a set of verification scan frames based on the intermediate splicing results and performing reassembly verification include the following steps S210-S260: S210. Continuously acquire the second scan frame and put the second scan frame into the verification scan frame set, wherein the verification scan frame set may or may not include the first scan frame.

[0033] In some embodiments, after step S210 and before S220, intermediate stitching results are also displayed. Understandably, displaying intermediate stitching results allows the operator to immediately confirm the initial positioning of the re-stitching attempt.

[0034] S220. Based on the set of verification scan frames, obtain the first verification splicing result.

[0035] In some embodiments, all frame data in the verification scan frame set are acquired, and local feature points are extracted for each scan frame within the set. Then, within the verification scan frame set, the relative pose transformation relationship between each scan frame is calculated and accumulated through feature matching and point cloud registration between adjacent scan frames, thereby constructing a local, self-consistent feature point cloud map and pose trajectory. This independently calculated local stitching result is the first verification stitching result used for subsequent verification and comparison. The entire process does not depend on or update the first stitching data, ensuring the objectivity of the verification benchmark.

[0036] S230. Determine whether the set of verification scan frames or the first verification splicing result meets the first verification re-splicing condition. If the first verification re-splicing condition is met, then execute step S240.

[0037] In some embodiments, step S230 includes: if the number of frames in the verification scan frame set reaches a dynamic threshold, then it is determined that the first verification reassembly condition is met; and / or, if the scan area corresponding to the first verification stitching result reaches a first preset size, then it is determined that the first verification reassembly condition is met. The dynamic threshold and the first preset size can both be set according to actual needs, and this application does not impose specific limitations on them.

[0038] In some embodiments, if the first verification reassembly condition is not met, the second scan frame is acquired, and the first verification reassembly condition is determined again.

[0039] S240. Perform a verification reassembly between the first verification splicing result and the intermediate splicing result to obtain the first verification reassembly result.

[0040] In some embodiments, the feature point clouds corresponding to the first verification stitching result and the feature point clouds corresponding to the intermediate stitching results are quickly coarsely matched using feature descriptors. A preliminary transformation matrix is ​​robustly estimated using a random sampling consensus algorithm, and erroneous matches are filtered out. An iterative nearest-point algorithm is used for fine registration to minimize the distance error between the two point clouds. The first verification stitching result is comprehensively determined as "stitching successful" or "stitching failed" by evaluating indicators such as the proportion of matched intrapoints, reprojection error, and consistency of overlapping areas. It should be noted that the verification stitching is feature stitching.

[0041] S250. If the first verification reassembly result is a reassembly failure, then discard the intermediate splicing results and restore the first splicing data.

[0042] In some embodiments, when the first verification reassembly result is a reassembly failure, it indicates that the intermediate splicing result has failed. All frames in the verification scan frame set are defused with the first splicing data to restore the first splicing data, thereby effectively avoiding temporary modifications to the first splicing data during the verification process. The verification scan frame set is cleared, ensuring the integrity and consistency of the first splicing data, thereby reverting to the normal real-time splicing process, that is, reverting to step S100, and finding a new splicing position to splice.

[0043] In another embodiment, discarding the intermediate splicing result can also be equivalent to deleting the intermediate splicing result. The deletion operation has several implementation schemes, including but not limited to: one implementation scheme is to delete only on the display interface, selectively retaining the displayed result rather than completely deleting it; another implementation scheme is to directly and completely delete it from the source data.

[0044] In some embodiments, the intermediate stitching result is continuously updated based on the first stitched data. The verification scan frame set and the intermediate stitching result are maintained and stored independently, and stitching / verification are performed at different locations. Optionally, the user can see multiple second scan frames continuously stitching and updating based on the intermediate stitching result in the interactive interface. When the first verification re-stitching result shows a re-stitching failure, the user directly sees the restoration to the first stitching result. Afterwards, the first scan frame continues to search for and relocate the target area for stitching. If it cannot be found, a new map can be generated, and the user can see two similar models generated. When the first verification re-stitching result shows a successful re-stitching, the user can see that the scanning continues uninterrupted. This setup allows for uninterrupted scanning and multiple verification re-stitchings, resulting in a high-precision model with smaller errors.

[0045] S260. If the first verification reassembly result is successful, then the intermediate splicing result and the first verification splicing result are merged and used as the updated first splicing data.

[0046] In some embodiments, when the first verification reassembly result is successful, the feature point cloud of the intermediate stitching result and the feature point cloud corresponding to the first verification stitching result are fused into the first stitched data. Feature points are extracted from the fused first stitched data to obtain a global feature point cloud. The global feature point cloud is then integrated with the feature point cloud corresponding to the first stitched data before fusion to obtain the updated first stitched data. The fused and updated first stitched data replaces the original first stitched data, thereby enhancing the integrity and reliability of the first stitched data in the corresponding region and effectively correcting possible accumulated errors.

[0047] In some embodiments, after obtaining the updated first stitched data, the method further includes: calculating the stitching residual of the updated first stitched data; if the stitching residual is greater than a preset threshold, then performing the step of restoring the first stitched data. The preset threshold can be set according to actual needs, and this application does not impose specific limitations on it. In one example, based on the feature point cloud of the updated first stitched data, for all successfully matched feature point pairs, their three-dimensional spatial distance or two-dimensional reprojection error is calculated, and the root mean square value of the three-dimensional spatial distance or two-dimensional reprojection error is calculated as the stitching residual.

[0048] Please see Figure 3 , Figure 3 This is a schematic diagram of another sub-process of a scanning processing method provided in an embodiment of the present invention, as shown below. Figure 3 As shown, after the step of generating new second stitched data based on the first scan frame, steps S310-S360 are also included: S310. Continuously acquire the second scan frame and perform a second sequence splicing of the second scan frame and the second splicing data.

[0049] In some embodiments, the second sequential splicing is implemented in the same way as the first sequential splicing. Please refer to the implementation process of the first sequential splicing. For the sake of simplicity, it will not be repeated here.

[0050] S320. If the second-order splicing result obtained by the second-order splicing meets the preset requirements, the second-order splicing result is used as the updated second splicing data.

[0051] In some embodiments, the determination of whether the second sequential stitching result meets the preset requirements is the same as the determination of whether the first sequential stitching result meets the preset requirements. For the sake of simplicity, it will not be repeated here. Understandably, if the first stitched data is used as the main map, the second stitched data is the first new map created, and the second sequential stitching result is used as the updated second stitched data, that is, the stitching and fusion are performed on the first new map.

[0052] S330. Determine whether the updated second spliced ​​data meets the second verification re-splicing condition. If it does, proceed to step S340.

[0053] In some embodiments, if the number of valid points in the currently sequentially stitched second scan frames reaches a dynamic point count threshold, the second verification re-stitching condition is determined to be met; and / or, if the number of second scan frames corresponding to the updated second stitched data reaches a dynamic threshold, the second verification re-stitching condition is determined to be met; and / or, if the scan area corresponding to the updated second stitched data reaches a first preset size, the second verification re-stitching condition is determined to be met; and / or, if the stitching residual of the updated second stitched data is greater than a preset residual value, the second verification re-stitching condition is determined to be met. The dynamic point count threshold, dynamic threshold, first preset size, and preset residual value can all be set according to actual needs, and this application does not impose specific limitations on them. In one example, the number of valid points in the second scan frame refers to the number of feature points successfully stitched with the second stitched data and usable for calculating the pose of the second scan frame.

[0054] S340. Verify and reassemble the updated second spliced ​​data with the first spliced ​​data to obtain the second verification and reassembly result.

[0055] In some embodiments, the verification reassembly process in step S340 is the same as the verification reassembly process in step S240, and will not be described again here for the sake of simplicity. It should be noted that the verification reassembly is feature reassembly.

[0056] S350. If the second verification reassembly result is successful, the updated second splicing data and the first splicing data are merged to form the updated first splicing data.

[0057] In some embodiments, when the second verification re-pasting result is successful, the first new map is merged with the main map to update the main map.

[0058] S360. If the second verification re-attachment result is a re-attachment failure, then continue to acquire the second scan frame to update the second splicing data, and continue to determine whether the second verification re-attachment condition is met.

[0059] In some embodiments, when the second verification re-attempt result is a re-attempt failure, the second stitched data (the first new map) is not discarded simply because of a failed re-attempt. Instead, the second stitched data is allowed to continue accumulating and optimizing, expanding its range and enriching its features by acquiring more scan frames. This creates better conditions for subsequent attempts to stitch with the main map (the first stitched data), thereby significantly improving the success rate and robustness of future cross-map merging.

[0060] Please see Figure 4 , Figure 4 This is a schematic diagram of another sub-process of a scanning processing method provided in an embodiment of the present invention, as shown below. Figure 4 As shown, after the step of performing a second sequential stitching of the second scan frame and the second stitched data, steps S410-S470 are further included: S410. If the second sequential splicing result does not meet the preset requirements, generate new third splicing data based on the second scan frame spliced ​​in the current sequence.

[0061] In some embodiments, when the second sequential stitching result does not meet the preset requirements, a third stitching data is generated based on the second scan frame, that is, a new map is regenerated, which is the second new map.

[0062] Optionally, if the second sequential stitching result does not meet the preset requirements, the second scan frame and the first stitched data are repositioned and stitched together in the target area. If stitching fails, a new third stitched data is generated based on the second scan frame stitched in the current order. If stitching succeeds, the first stitched data is updated. Specifically, when the second sequential stitching result does not meet the preset requirements, the second scan frame and the first stitched data (such as the main map) are repositioned and stitched together in the target area. If stitching fails, a new third stitched data is generated based on the second scan frame stitched in the current order. If stitching succeeds, the second scan frame and the first stitched data (such as the main map) are repositioned and stitched together in the target area. If stitching succeeds, the first stitched data is updated.

[0063] S420. Continuously acquire the third scan frame and then perform a third sequence of splicing the third scan frame with the third splicing data.

[0064] In some embodiments, the third sequential splicing is implemented in the same way as the second and first sequential splicing. Please refer to the implementation process of the first sequential splicing. For the sake of simplicity, it will not be described again here.

[0065] S430. If the third-order splicing result obtained by the third-order splicing meets the preset requirements, the third-order splicing result shall be used as the updated third-order splicing data.

[0066] In some embodiments, the determination of whether the third-order stitching result meets the preset requirements is the same as the determination of whether the first-order stitching result and the second-order stitching result meet the preset requirements. For the sake of simplicity, it will not be repeated here. Understandably, using the third-order stitching result as the updated third stitching data means stitching and merging it in the second new map.

[0067] S440. Determine whether the updated third spliced ​​data meets the third verification re-splicing condition. If it does, proceed to step S450.

[0068] In some embodiments, if the number of valid points in the third scan frame currently stitched together reaches a dynamic point count threshold, it is determined that the third verification re-stitching condition is met; and / or, if the number of third scan frames corresponding to the updated third stitched data reaches a dynamic threshold, it is determined that the third verification re-stitching condition is met; and / or, if the scan area corresponding to the updated third stitched data reaches a first preset size, it is determined that the third verification re-stitching condition is met; and / or, if the stitching residual of the updated third stitched data is greater than a preset residual value, it is determined that the third verification re-stitching condition is met.

[0069] S450. Perform verification and reassembly on any two of the third spliced ​​data, the first spliced ​​data, and the second spliced ​​data to obtain the third verification and reassembly result.

[0070] In some embodiments, the verification reassembly process in step S450 is the same as the verification reassembly process in step S340 and the verification reassembly process in step S240, and will not be described in detail here for simplicity. In one example, the third concatenated data is verified and reassembled with the first concatenated data; in another example, the third concatenated data is verified and reassembled with the second concatenated data. It should be noted that the verification reassembly is feature reassembly.

[0071] S460. If the third verification reassembly result is successful, then the two successfully reassembled data will be merged and used as the updated first reassembled data, and the remaining unreassembled data will continue to be saved.

[0072] In some embodiments, when the third verification re-pasting result is successful, the two successfully piecing together maps are used as the main map, and the unpiecing maps are kept.

[0073] S470. If the third verification reassembly result is a reassembly failure, then continue to acquire the third scan frame to update the third stitching data, and continue to determine whether the third verification reassembly condition is met.

[0074] In some embodiments, when the third verification re-attempt result is a re-attempt failure, the third stitched data (second new map) is not discarded due to a single failed re-attempt. Instead, the second new map is allowed to continue accumulating and optimizing, expanding its range and enriching its features by acquiring more scan frames, so that it can be subsequently fused and stitched with the first new map and the main map.

[0075] In some embodiments, it is determined whether the third stitched data, the first stitched data, and / or the second stitched data meet preset deletion conditions. Specifically, if any data fails to pass the verification and re-stitching within a preset time, it is determined that the preset deletion conditions are met; and / or, if the number of scan frames corresponding to any data is less than a preset number, it is determined that the preset deletion conditions are met; and / or, if any data does not increase its scanning range within a preset time, it is determined that the preset deletion conditions are met; and / or, if the scanning area corresponding to any data is smaller than a second preset size, it is determined that the preset deletion conditions are met. If so, the data that meets the preset deletion conditions is deleted. Understandably, if any map among the main map, the first new map, and the second new map cannot be successfully stitched with other maps within a preset time, or if the amount of scan frame data corresponding to it is too small, or if its scanning range does not increase for a long time, it is determined to be "invalid" or "isolated" data. Such maps will be automatically deleted to release computing and storage resources and ensure that the focus is always on valid data that is expected to be integrated into the main map, thereby maintaining the efficiency and stability of the overall operation.

[0076] The deletion operation has several implementation schemes, including but not limited to: one implementation scheme is to delete only on the display interface, which is a selective retention of the display result rather than a complete deletion; another implementation scheme is to completely delete it from the source data.

[0077] In the above embodiments, after obtaining the updated first stitched data, the method may further include: calculating the stitching residual of the updated first stitched data; if the stitching residual is greater than a preset threshold, then performing the step of restoring the first stitched data. Optionally, multiple models (maps) may be displayed on the user interface. The updated first stitched data is the main map displayed on the user interface in the largest area.

[0078] Optionally, this application may maintain a separate set of verification frames for the continuously acquired second scan frames or the continuously acquired third scan frames, or the second and third stitched data may be maintained and updated separately. This application continuously verifies the stitching residuals of the updated first stitched data. If the residuals are found to be too high, defusion can be performed based on the separately stored set of verification frames or the separately stored stitching results (map) to restore the first stitched data. This allows for rapid rollback and ultimately yields a map or model with smaller cumulative errors.

[0079] To facilitate understanding, the specific implementation process of the scanning processing method is illustrated below with an example: In practical applications of oral scanning modeling, when a scanner scans a patient's jaw, it first tracks the acquired first scan frame in real time. If tracking is successful, it attempts to perform a first-order stitching. If the first-order stitching is successful, the first scan frame is fused into the main map (fusion field), and the global feature point cloud corresponding to the fusion field is updated. At this point, the target global feature point cloud and the new fusion field are obtained. If the first-order stitching fails, feature re-stitching verification is initiated, that is, the acquired second scan frame is used to perform feature re-stitching verification with the fusion field to obtain the first-verified stitching result. Since there may be geometrically similar regions in the jaw, this result may be incorrect. Therefore, the first scan frame, the second scan frame, and several subsequent successfully stitched scan frames are temporarily stored in a preset frame group (verification scan frame set), and a local feature point cloud is maintained. The first verification re-stitching condition is checked based on the preset frame group or the first verification stitching result. When the first verification re-stitching condition is met, the local feature point cloud and the global feature point cloud corresponding to the fusion site are verified and re-stitched (feature re-stitching) to obtain the first verification re-stitching result. If the first verification re-stitching result is a re-stitching failure, all frames in the preset frame group are defused with the fusion site, and the preset frame group is cleared. If the first verification re-stitching result is a re-stitching success, the local feature point cloud corresponding to the preset frame group is added to the global feature point cloud, and the preset frame group is cleared.

[0080] If a serious tracking failure occurs during scanning (e.g., the scanner rapidly moves to a completely new area with no overlap with the main map), and the first scan frame meets preset conditions (the preset conditions are that the number of valid points in the first scan frame is greater than a preset number of valid points, or the stitching residual of the first scan frame is greater than a preset residual, or the scanning range corresponding to the first scan frame is greater than a preset scanning range), a new map will be created based on the first scan frame, i.e., the first new map. Subsequent scans will be stitched and merged within the first new map.

[0081] If the stitching fails during the second sequential stitching of the second scan frame and the first new map, a new map, the second new map, is created based on the second scan frame. Understandably, at this point, the main map, the first new map, and the second new map each have their own fusion field and feature point cloud. When the scanner returns to the boundary between the different maps, it will attempt to stitch them back together using the feature point cloud. If the stitching is successful, the successfully stitched maps will be merged. Maps that fail to merge successfully for an extended period and may be invalid (e.g., due to temporary occlusion) will be automatically deleted when preset deletion conditions are met to conserve resources.

[0082] Figure 5 This is a schematic block diagram of a scanning processing apparatus 200 provided in an embodiment of the present invention. For example... Figure 5As shown, corresponding to the above scanning processing method, the present invention also provides a scanning processing apparatus 200. This scanning processing apparatus 200 includes a unit for performing the above scanning processing method, and the apparatus can be configured in an electronic device. Specifically, please refer to... Figure 5 The scanning processing device 200 includes an acquisition unit 201, a stitching unit 202, and a back-stitching verification unit 203. Detailed descriptions of each functional module are as follows: Acquisition unit 201 is used to acquire the first scan frame and the first stitched data of the scanned object; The splicing unit 202 is used to splice the first scan frame and the first splicing data in a first order; The back-assembly verification unit 203 is used to, if the first-order splicing result obtained by the first-order splicing does not meet the preset requirements, relocate the target area in the first splicing data and splice it to obtain an intermediate splicing result, and continuously obtain a set of verification scan frames based on the intermediate splicing result for back-assembly verification, or save the first splicing data and generate new second splicing data based on the first scan frame.

[0083] In one embodiment, the back-assembly verification unit 203 is specifically used for: The second scan frame is continuously acquired and placed into the verification scan frame set, wherein the verification scan frame set may or may not include the first scan frame; Based on the set of scanned verification frames, the first verification splicing result is obtained; Determine whether the set of scanned frames or the first verification splicing result meets the first verification re-splicing condition; If the first verification reassembly condition is met, the first verification reassembly result and the intermediate reassembly result are reassembly to obtain the first verification reassembly result. If the first verification reassembly result is a failure, then discard the intermediate splicing results and restore the first splicing data; If the first verification reassembly result is successful, the intermediate splicing result and the first verification splicing result are merged and used as the updated first splicing data.

[0084] In one embodiment, the back-scraping verification unit 203 is further configured to: Display the intermediate splicing results.

[0085] In one embodiment, the back-scraping verification unit 203 is further configured to: If the number of frames in the scanned frame set reaches the dynamic threshold, then the first verification reassembly condition is satisfied. And / or, if the scanning area corresponding to the first verification stitching result reaches the first preset size, then it is determined that the first verification stitching condition is met.

[0086] In one embodiment, the back-scraping verification unit 203 is further configured to: Continuously acquire the second scan frame, and then perform a second sequential concatenation of the second scan frame and the second concatenation data; If the second-order splicing result obtained by the second-order splicing meets the preset requirements, the second-order splicing result will be used as the updated second splicing data; Determine whether the updated second spliced ​​data meets the second verification re-sponsoring condition; If the second verification and reassembly condition is met, the updated second spliced ​​data and the first spliced ​​data will be verified and reassembled to obtain the second verification and reassembly result. If the second verification reassembly result is successful, the updated second spliced ​​data and the first spliced ​​data will be merged to form the updated first spliced ​​data. If the second verification reassembly result is a reassembly failure, then continue to acquire the second scan frame to update the second stitching data, and continue to determine whether the second verification reassembly condition is met.

[0087] In one embodiment, the back-scraping verification unit 203 is further configured to: If the number of valid points in the second scan frame being stitched in the current sequence reaches the dynamic point count threshold, then the second verification reassembly condition is satisfied. And / or, if the number of second scan frames corresponding to the updated second splicing data reaches the dynamic threshold, then the second verification re-splicing condition is satisfied; And / or, if the scanning area corresponding to the updated second stitching data reaches the first preset size, then the second verification re-stitching condition is satisfied; And / or, if the splicing residual of the updated second splicing data is greater than the preset residual value, then the second verification re-sponsoring condition is satisfied.

[0088] In one embodiment, the back-scraping verification unit 203 is further configured to: If the second sequential splicing result does not meet the preset requirements, a new third splicing data is generated based on the second scan frame spliced ​​in the current sequence. Continuously acquire the third scan frame, and then concatenate the third scan frame with the third stitched data in a third sequence; If the third-order splicing result meets the preset requirements, the third-order splicing result will be used as the updated third-order splicing data. Determine whether the updated third-concatenation data meets the third verification reassembly condition; If the third verification reassembly condition is met, then any two data points from the third spliced ​​data, the first spliced ​​data, and the second spliced ​​data will be verified and reassembled to obtain the third verification reassembly result. If the third verification result is successful, the two successfully concatenated data will be merged and used as the first concatenated data after the update, and the remaining unconcatenated data will continue to be saved. If the third verification reassembly result is a failure, then the third scan frame is acquired to update the third stitching data, and the third verification reassembly condition is still met.

[0089] In one embodiment, the back-scraping verification unit 203 is further configured to: If the number of valid points in the third scan frame currently being stitched together reaches the dynamic point count threshold, then the third verification reassembly condition is satisfied. And / or, if the number of third scan frames corresponding to the updated third stitched data reaches the dynamic threshold, then the third verification re-stitching condition is satisfied; And / or, if the scanning area corresponding to the updated third stitching data reaches the first preset size, then the third verification re-stitching condition is satisfied; And / or, if the splicing residual of the updated third splicing data is greater than the preset residual value, then the third verification re-sponsoring condition is satisfied.

[0090] In one embodiment, the back-scraping verification unit 203 is further configured to: Determine whether the third spliced ​​data, the first spliced ​​data, and / or the second spliced ​​data meet the preset deletion conditions; If so, the data that meets the preset deletion conditions will be deleted.

[0091] In one embodiment, the back-scraping verification unit 203 is further configured to: If any data fails to pass the verification and reconstruction within the preset time, it is determined that the preset deletion condition is met; And / or, if the number of scan frames corresponding to any data is less than the preset number, then the preset deletion condition is met; And / or, if any data does not increase the scan range within a preset time, the preset deletion condition is determined to be met; And / or, if the scanned area corresponding to any data is smaller than the second preset size, then the preset deletion condition is satisfied.

[0092] In one embodiment, the back-scraping verification unit 203 is further configured to: If the first sequence of splicing results meets the preset requirements, then the first sequence of splicing results will be used as the updated first splicing data.

[0093] In one embodiment, the back-scraping verification unit 203 is further configured to: Calculate the splicing residual of the updated first spliced ​​data; If the splicing residual is greater than the preset threshold, then the step of restoring the first spliced ​​data is executed.

[0094] The aforementioned scanning processing apparatus can be implemented as a computer program, which can, for example... Figure 6 It runs on the electronic device shown.

[0095] Please see Figure 6 , Figure 6 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. The electronic device 300 is separate from the scanning device and communicates with it via wired or wireless means. The electronic device 300 can be a computer, tablet computer, and / or mobile phone, etc., and the scanning device can be an oral scanner (intraoral scanner or extraoral scanner) or a camera mounted on an implantable handpiece, etc. In some scenarios, the electronic device 300 can be housed within the scanning device. In some scenarios, other types of devices may also be included. In summary, the embodiments of this application do not limit the specific application scenarios of the three-dimensional reconstruction method.

[0096] See Figure 6 The electronic device 300 includes a processor 302, a memory, and a network interface 305 connected via a system bus 301. The memory may include a non-volatile storage medium 303 and internal memory 304.

[0097] The non-volatile storage medium 303 may store an operating system 3031 and a computer program 3032. When the computer program 3032 is executed, it causes the processor 302 to perform a scanning processing method.

[0098] The processor 302 provides computing and control capabilities to support the operation of the entire electronic device 300.

[0099] The internal memory 304 provides an environment for the execution of the computer program 3032 in the non-volatile storage medium 303. When the computer program 3032 is executed by the processor 302, the processor 302 can perform a scanning processing method.

[0100] This network interface 305 is used for network communication with other devices. Those skilled in the art will understand that... Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device 300 to which the present invention is applied. The specific electronic device 300 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0101] The processor 302 is used to run a computer program 3032 stored in a memory to implement any embodiment of the above-described scanning processing method.

[0102] It should be understood that, in this embodiment of the invention, the processor 302 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0103] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a storage medium, which is an electronic device readable storage medium. The computer program is executed by at least one processor in the electronic device system to implement the process steps of the embodiments of the above methods.

[0104] Therefore, the present invention also provides a storage medium. This storage medium can be an electronically readable storage medium. The storage medium stores a computer program. When executed by a processor, the computer program causes the processor to perform any embodiment of the scanning processing method described above.

[0105] The storage medium can be any electronic device readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0106] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, electronic device software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0107] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0108] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0109] If the integrated unit example is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This electronic device software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0110] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0111] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Since these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0112] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A scanning processing method, characterized in that, include: Acquire the first scan frame and the first stitched data of the scanned object; The first scan frame and the first spliced ​​data are spliced ​​together in a first order; If the first sequence stitching result obtained does not meet the preset requirements, the target region is repositioned in the first stitching data and stitched to obtain an intermediate stitching result. Based on the intermediate stitching result, a set of verification scan frames is continuously acquired for back-stitching verification. Alternatively, the first stitching data is saved, and new second stitching data is generated based on the first scan frame.

2. The method according to claim 1, characterized in that, The step of continuously acquiring a set of verification scan frames based on the intermediate splicing results and performing re-splicing verification includes: The second scan frame is continuously acquired and placed into the verification scan frame set, wherein the verification scan frame set may or may not include the first scan frame; Based on the set of verification scan frames, the first verification splicing result is obtained; Determine whether the set of verification scan frames or the first verification splicing result satisfies the first verification re-splicing condition; If the first verification reassembly condition is met, then the first verification splicing result and the intermediate splicing result are reassembled for verification to obtain the first verification reassembly result. If the first verification reassembly result is a reassembly failure, then the intermediate splicing result is discarded and the first spliced ​​data is restored; If the first verification reassembly result is successful, then the intermediate splicing result and the first verification splicing result are merged and used as the updated first splicing data.

3. The method according to claim 2, characterized in that, Before the step of obtaining the first verification splicing result based on the set of verification scan frames, the method further includes: The intermediate splicing results are displayed.

4. The method according to claim 2, characterized in that, The step of determining whether the set of verification scan frames or the first verification splicing result satisfies the first verification reassembly condition includes: If the number of frames in the set of verification scan frames reaches the dynamic threshold, then the first verification reassembly condition is satisfied. And / or, if the scanning area corresponding to the first verification stitching result reaches the first preset size, then it is determined that the first verification stitching condition is met.

5. The method according to claim 1, characterized in that, After the step of generating new second stitched data based on the first scan frame, the following steps are included: Continuously acquire the second scan frame, and then perform a second sequential concatenation of the second scan frame and the second concatenation data; If the second sequence splicing result obtained by the second sequence splicing meets the preset requirements, the second sequence splicing result is used as the updated second splicing data; Determine whether the updated second spliced ​​data meets the second verification re-splicing condition; If the second verification and reassembly condition is met, the updated second spliced ​​data and the first spliced ​​data are verified and reassembled to obtain the second verification and reassembly result. If the second verification reassembly result is successful, then the updated second spliced ​​data and the first spliced ​​data are merged to form the updated first spliced ​​data; If the second verification reassembly result is a reassembly failure, then continue to acquire the second scan frame to update the second spliced ​​data, and continue to determine whether the second verification reassembly condition is met.

6. The method according to claim 5, characterized in that, The step of determining whether the updated second spliced ​​data meets the second verification re-splicing condition includes: If the number of valid points in the second scan frame currently being stitched together reaches the dynamic point count threshold, then the second verification reassembly condition is satisfied. And / or, if the number of the second scan frames corresponding to the updated second splicing data reaches a dynamic threshold, then it is determined that the second verification re-splicing condition is met; And / or, if the scanning area corresponding to the updated second stitching data reaches the first preset size, then it is determined that the second verification re-stitching condition is met; And / or, if the splicing residual of the updated second splicing data is greater than the preset residual value, then it is determined that the second verification re-sponsoring condition is met.

7. The method according to claim 5, characterized in that, After the step of performing a second sequential stitching of the second scan frame and the second stitched data, the method further includes: If the second sequential splicing result does not meet the preset requirements, a new third splicing data is generated based on the second scan frame spliced ​​in the current order; Continuously acquire the third scan frame, and then concatenate the third scan frame with the third splicing data in a third sequence; If the third-order splicing result obtained by the third-order splicing meets the preset requirements, the third-order splicing result is used as the updated third splicing data; Determine whether the updated third spliced ​​data meets the third verification re-sponsoring condition; If the third verification and reassembly condition is met, then any two of the third spliced ​​data, the first spliced ​​data, and the second spliced ​​data are verified and reassembled to obtain the third verification and reassembly result. If the third verification reassembly result is successful, the two successfully reassembled data will be merged and used as the updated first reassembled data, and the remaining unreassembled data will continue to be saved. If the third verification reassembly result is a reassembly failure, then the third scan frame is acquired to update the third splicing data, and it is further determined whether the third verification reassembly condition is met.

8. The method according to claim 5, characterized in that, After the step of performing a second sequential stitching of the second scan frame and the second stitched data, the method further includes: If the second sequential splicing result does not meet the preset requirements, then the second scan frame and the first splicing data are repositioned and spliced ​​in the target area. If splicing fails, new third splicing data is generated based on the second scan frames spliced ​​in the current order; When the stitching is successful, update the first stitched data.

9. The method according to claim 7, characterized in that, The step of determining whether the updated third spliced ​​data meets the third verification re-splicing condition includes: If the number of valid points in the third scan frame currently being stitched together reaches the dynamic point count threshold, then the third verification reassembly condition is satisfied. And / or, if the number of third scan frames corresponding to the updated third splicing data reaches a dynamic threshold, then the third verification re-splicing condition is determined to be satisfied; And / or, if the scanning area corresponding to the updated third stitching data reaches the first preset size, then it is determined that the third verification re-stitching condition is met; And / or, if the splicing residual of the updated third splicing data is greater than the preset residual value, then it is determined that the third verification re-sponsoring condition is met.

10. The method according to claim 7, characterized in that, The method further includes: Determine whether the third spliced ​​data, the first spliced ​​data, and / or the second spliced ​​data meet the preset deletion conditions; If so, the data that meets the preset deletion conditions will be deleted.

11. The method according to claim 10, characterized in that, The step of determining whether the third spliced ​​data, the first spliced ​​data, and / or the second spliced ​​data meet the preset deletion conditions includes: If any data fails to pass the verification and reconstruction within a preset time, it is determined that the preset deletion condition is met; And / or, if the number of scan frames corresponding to any data is less than a preset number, then the preset deletion condition is satisfied; And / or, if any data does not increase the scan range within a preset time, it is determined that the preset deletion condition is met; And / or, if the scanned area corresponding to any data is smaller than the second preset size, then the preset deletion condition is determined to be met.

12. The method according to claim 1, characterized in that, After the step of performing a first sequential stitching of the first scan frame and the first stitched data, the method further includes: If the first sequential splicing result meets the preset requirements, then the first sequential splicing result is used as the updated first splicing data.

13. The method according to claim 2, 5, 7, or 12, characterized in that, After obtaining the updated first spliced ​​data, the process further includes: Calculate the splicing residual of the updated first spliced ​​data; If the splicing residual is greater than a preset threshold, then the step of restoring the first spliced ​​data is executed.

14. A scanning processing apparatus, characterized in that, include: The acquisition unit is used to acquire the first scan frame and the first stitched data of the scanned object. A splicing unit is used to splice the first scan frame and the first splicing data in a first order; The back-assembly verification unit is used to, if the first-order assembly result obtained by the first-order assembly does not meet the preset requirements, relocate the target area in the first-order assembly data and perform assembly to obtain an intermediate assembly result, and continuously acquire a set of verification scan frames based on the intermediate assembly result for back-assembly verification, or save the first-order assembly data and generate new second-order assembly data based on the first scan frame.

15. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-13.

16. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-13.

Images