Method, apparatus, processor, and scanning system for processing scanning results

Abstract

A method, apparatus, processor and scanning system for processing scan results, the processing method includes the steps of: obtaining scan results of a measurement object, the scan results including a two-dimensional image and / or a three-dimensional model (S101); calling a smart recognition function to recognize the scan results and obtain a classification result, the smart recognition function being a classification model obtained by training image samples (S102); and determining invalid data in the scan results based on the classification result, the invalid data being scan results of non-target areas of the measurement object (S103).

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to Chinese Patent Application No. 202011057266.1, filed with the China Patent Office on September 29, 2020, entitled "Method, Apparatus, Processor and Scanning System for Processing Scanning Results," the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the technical field of scanners, and in particular to a method, an apparatus, a computer readable storage medium, a processor and a scanning system for processing scan results. [Background technology]

[0003] In the application of oral digital restoration design, the intraoral dental data of the patient is extracted, and mainly collected by scanning with an intraoral scanner to collect data of teeth and gums. Because the intraoral space is relatively small, during scanning, it is very easy to capture images of the tongue, lips, buccal side, auxiliary tools, etc., resulting in the generated model data containing invalid data, which mainly has the following effects:

[0004] 1. During scanning, if there is data other than teeth and gums, interference will occur when stitching new scanned data, making it difficult to stitch new data in certain areas.

[0005] 2. In the optimization of grid data, invalid data at a certain interval from the teeth can be removed by the relevant algorithm, but the connected parts need to be actively removed, otherwise they will be retained, which adds the steps of checking, selecting and removing, and affects the overall scanning efficiency.

[0006] The process of generating the digital data for the model is from the original depth map → point cloud data (reconstruction) → grid model (fusion). Therefore, if a frame of data contains images of the labial and buccal sides, the fused grid data will contain these triangular meshes, i.e., unnecessary data.

[0007] In the process of scanning and collecting oral data, due to the constraints of the oral space, data other than teeth and gums is likely to appear in the scanning results, which may affect the scanning of new data and the optimization of the entire data. The scanning of dental data is a real-time and continuous process, and since the frame rate of scanning is relatively high, the generation of unnecessary data also affects the operator's scanning process. Regarding the generated unnecessary data, most of this data can be thinned out by using algorithms such as "isolation removal" and "strong connectivity". However, since such algorithm processing takes relatively long time, it has to be executed intermittently. If there is data that cannot be removed by the algorithm, it needs to be removed manually.

[0008] Existing methods for removing unnecessary data mainly analyze the generated data and, through the correlation relationship of data blocks (points), for example, whether there are some isolated data blocks (points), or by some restrictions or strategies, find invalid data points that do not meet the conditions, and then remove these points from the entire data.

[0009] However, the scanning of oral data is highly real-time. When the overall grid data increases and the removal and optimization algorithms are executed, it is necessary to pause the scanning and wait for the optimization to be completed, which affects the scanning efficiency. Furthermore, this algorithm has specific strategies and depends on specific calculation problems, targeting only general case data and unable to remove invalid data in relatively special cases.

[0010] The above information disclosed in the background art is merely for deepening the understanding of the background art of the technology described in this specification. Therefore, the background art may contain certain information that does not form the prior art known in the country for those skilled in the art.

Summary of the Invention

Problems to be Solved by the Invention

[0011] The main object of the present application is to provide a method for processing scanning results, an apparatus, a computer-readable storage medium, a processor, and a scanning system to solve the problem of inefficient scanning caused by the method for processing scanning results for determining invalid data through data analysis in the prior art.

Means for Solving the Problems

[0012] According to an aspect of an embodiment of the present invention, there is provided a method for processing scanning results, including: obtaining a scanning result of a measurement object, where the scanning result includes a two-dimensional image and / or a three-dimensional model; calling a smart recognition function to recognize the scanning result and obtain a classification result, where the smart recognition function is a classification model obtained by training an image sample; and determining invalid data in the scanning result based on the classification result, where the invalid data is a scanning result of a non-target area of the measurement object.

[0013] Preferably, the scanning result is the two-dimensional image, the two-dimensional image includes a texture image, and the classification result includes first image data corresponding to a target area of the measurement object in the texture image and second image data corresponding to a non-target area of the measurement object.

[0014] Preferably, the scanning result further includes a reconstructed image corresponding to the texture image, and the step of determining invalid data in the scanning result based on the classification result includes the steps of constructing a 3D point cloud based on the reconstructed image and determining an invalid point cloud based on a correspondence between the reconstructed image and the texture image, wherein the invalid point cloud is a point cloud corresponding to the second image data in the 3D point cloud, removing the invalid point cloud in the 3D point cloud, and stitching a valid 3D model of the measurement object based on the remaining point cloud in the 3D point cloud.

[0015] Preferably, if a 3D model of the measured object is successfully reconstructed, the scanning result is the 3D model, and the step of invoking a smart recognition function to recognize the scanning result and obtain a classification result further includes the steps of obtaining 3D point cloud data for reconstructing the 3D model, and invoking the smart recognition function to analyze the 3D point cloud data and recognize a classification result for the 3D point cloud data, and the classification result includes first point cloud data in the 3D point cloud data corresponding to a target area in the measured object and second point cloud data corresponding to a non-target area of ​​the measured object.

[0016] Preferably, when the second point cloud data is determined to be the invalid data, the invalid data is removed from the three-dimensional point cloud data to determine point cloud data of a valid area in the three-dimensional model.

[0017] Preferably, the method further includes, prior to the step of acquiring 3D point cloud data for reconstructing the 3D model, a step of collecting 2D images of the measurement object, three-dimensionally reconstructing 3D point cloud data based on the 2D images, and stitching the 3D model based on the reconstructed 3D point cloud data.

[0018] Preferably, the method further includes the step of initiating and initializing a scanning process and an AI recognition process prior to the step of obtaining a scan result of the measurement object, wherein the scanning process is used to perform a scan on the measurement object and the AI ​​recognition process is used to recognize and classify the scan result.

[0019] Preferably, during the process of initializing the scanning process and the AI ​​recognition process, it is monitored whether communication is successfully established between the scanning process and the AI ​​recognition process, and after confirming that the connection is successful, if the scanning result is detected, the scanning process issues a processing command to the AI ​​recognition process, and the AI ​​recognition process calls the smart recognition function based on the processing command to recognize the scanning result.

[0020] Preferably, in the process of monitoring whether communication is successfully established between the scanning process and the AI ​​recognition process, the AI ​​recognition process runs in parallel, and when the operating environment meets certain conditions, the AI ​​recognition process initializes a recognition algorithm, receives the processing instruction, and executes the smart recognition function after the recognition algorithm is successfully initialized.

[0021] According to another aspect of the embodiment of the present invention, a scanning result processing device includes: an acquisition unit for acquiring a scanning result of a measurement object, the scanning result including a two-dimensional image and / or a three-dimensional model; a first recognition unit for invoking a smart recognition function to recognize the scanning result and obtain a classification result, the smart recognition function being a classification model obtained by training image samples; and a first determination unit for determining invalid data in the scanning result based on the classification result, the invalid data being classified by the measurement object. of non-target area of A first determining unit that is a scanning result and a processing device for the scanning result that includes the first determining unit are also provided.

[0022] According to a further aspect of an embodiment of the present invention there is provided a storage medium including a program stored thereon, the program performing any one of the processing methods described above.

[0023] According to a further aspect of an embodiment of the present invention there is provided a processor adapted to execute a program, the program being adapted to perform any one of the processing methods described above when executed.

[0024] According to a further aspect of an embodiment of the present invention there is provided a scanning system comprising a scanner and a scan result processing device, the scan result processing device being adapted to perform any one of the processing methods. Effect of the Invention

[0025] In the above processing method, firstly, obtain the scanning result of the measurement object, i.e., obtain the 2D image and / or 3D model obtained from the scanner, then invoke the smart recognition function to recognize the scanning result and obtain the classification result, i.e., classify the 2D image and / or 3D model by the trained classification model, and finally determine the invalid data in the scanning result based on the classification result, i.e., remove the invalid data from the 2D image and / or 3D model based on the classification result, thereby eliminating the need to interrupt the scanning to determine the invalid data through data analysis, and further improving the scanning efficiency. [Brief description of the drawings]

[0026] The accompanying drawings of the specification forming a part of this application are used to provide a further understanding of the application, and the illustrative embodiments and the description thereof are used to explain the application and are not intended to unduly limit the application. [Figure 1] 4 is a flow chart of a method for processing a scan result according to an embodiment of the present application. [Diagram 2] FIG. 2 is a schematic diagram of a three-dimensional model of teeth and gums according to one embodiment of the present application. [Diagram 3] 1 is a flowchart of an AI recognition process according to one embodiment of the present application. [Figure 4] 1 is a flowchart for initiating a scanning process and an AI recognition process according to an embodiment of the present application. [Diagram 5] 4 is a flowchart for constructing a three-dimensional model of a measurement object according to an embodiment of the present application. [Figure 6] 4 is a flow chart of a scanning result processing apparatus according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] In addition, the features in the examples and embodiments of the present application can be combined with each other without contradiction. The present application will now be described in detail with reference to the accompanying drawings and examples.

[0028] The following clearly and completely describes the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application, in order to allow those skilled in the art to better understand the solutions of the present application, but it is clear that the described embodiments are only a part of the embodiments of the present application, and are not all of them. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts shall be included in the protection scope of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned attached drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that the data used in this manner are interchangeable, as appropriate, for the purposes of the examples of this application described herein. Furthermore, the terms "comprise" and "comprise", as well as any variations thereof, are intended to be non-exclusive inclusive, e.g., a process, method, system, product, or apparatus consisting of a series of steps or units need not be limited to those steps or units expressly described, but may include other steps or units not expressly described or inherent to those processes, methods, products, or apparatus.

[0030] When an element (such as a layer, film, region, or substrate) is described as being "on" another element, it is understood that the element may be directly on the other element, or there may be intermediate components between them. Further, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element, or may be "connected" to the other element by a third element.

[0031] As described in the Background section, the prior art scanning result processing method determines invalid data through data analysis, which reduces scanning efficiency. To solve the above problems, the exemplary embodiments of the present application provide a scanning result processing method, an apparatus, a computer-readable storage medium, a processor and a scanning system.

[0032] According to one embodiment of the present application, a method for processing scan results is provided.

[0033] FIG. 1 is a flow chart of a method for processing a scanning result according to an embodiment of the present application. As shown in FIG. 1, the method includes: S101, acquiring a scan of a measurement object, the scan including a two-dimensional image and / or a three-dimensional model; Step S102: invoking a smart recognition function to recognize the scanning result and obtain a classification result, the smart recognition function being a classification model obtained by training image samples; The method includes determining invalid data in the scanning result based on the classification result, where the invalid data is a scanning result of a non-target area of ​​the measurement object.

[0034] In the above processing method, firstly, obtain the scanning result of the measurement object, i.e., obtain the 2D image and / or 3D model obtained from the scanner, then invoke the smart recognition function to recognize the scanning result and obtain the classification result, i.e., classify the 2D image and / or 3D model by the trained classification model, and finally determine the invalid data in the scanning result based on the classification result, i.e., remove the invalid data from the 2D image and / or 3D model based on the classification result, thereby eliminating the need to interrupt the scanning to determine the invalid data through data analysis, and further improving the scanning efficiency.

[0035] It should be noted that the steps depicted in the flowcharts of the accompanying drawings may be executed in a computer system as a set of computer-executable instructions, and that although a logical order is shown in the flowcharts, in some cases the steps shown or described may be executed in a different order than as shown here.

[0036] In one embodiment of the present application, the scanning result is the two-dimensional image, the two-dimensional image includes a texture image, and the classification result includes first image data corresponding to the target area of ​​the measurement object in the texture image and second image data corresponding to the non-target area of ​​the measurement object. Specifically, by recognizing the texture image with a smart recognition function, the texture image is divided into first image data and second image data, the first image data is image data corresponding to the target area of ​​the measurement object in the texture image, and the second image data is image data corresponding to the non-target area of ​​the measurement object in the texture image, and the target area and the non-target area are predetermined, for example, when the measurement object is an oral cavity, the target area includes teeth and gums, and the non-target area includes tongue, lips, cheeks, etc., the first image data is image data corresponding to teeth and gums, and the second image data is image data corresponding to areas such as tongue, lips, cheeks, etc., of course, the target area and the non-target area can be adjusted according to actual needs, for example, when only tooth data is required, the gums can be preset as the non-target area. The classification results include teeth, gums, tongue, lips, and cheeks, but of course, if the tongue, lips, and cheeks are classified as others, the classification results can also include teeth, gums, and others, and if the teeth and gums are classified as target classes and the tongue, lips, and cheeks are classified as non-target classes (i.e., others), the classification results can also be classified as target classes and non-target classes.

[0037] In one embodiment of the present application, the two-dimensional image further includes a reconstructed image corresponding to the texture image, and the step of determining invalid data in the scanning result based on the classification result includes the steps of constructing a three-dimensional point cloud based on the reconstructed image and determining an invalid point cloud based on a correspondence between the reconstructed image and the texture image, where the invalid point cloud is a point cloud corresponding to the second image data in the three-dimensional point cloud; removing the invalid point cloud in the three-dimensional point cloud; and stitching a valid three-dimensional model of the measurement object based on the remaining point cloud in the three-dimensional point cloud. Specifically, a three-dimensional point cloud is constructed based on the reconstructed image, and a point cloud corresponding to the second image data in the three-dimensional point cloud data is determined according to the correspondence between the reconstructed image and the texture image, thereby determining an invalid point cloud, and after removing the invalid point cloud, the remaining point cloud is stitched to obtain a valid three-dimensional model of the measurement object. For example, a two-dimensional image of a first frame and a two-dimensional image of a second frame are obtained, the remaining point cloud of the first frame is obtained based on the two-dimensional image of the first frame, the remaining point cloud of the second frame is obtained based on the two-dimensional image of the second frame, and the remaining point cloud of the first frame and the remaining point cloud of the second frame are stitched together. More specifically, when the measurement object is an oral cavity, the target region includes teeth and gums, and the non-target region includes tongue, lips, cheeks, etc., the point clouds corresponding to regions such as tongue, lips, cheeks, etc. are removed, and the remaining point clouds are stitched together to obtain a three-dimensional model of the teeth and gums, as shown in FIG. 2.

[0038] In one embodiment of the present application, if the 3D model of the measurement object is successfully reconstructed, the scanning result is the 3D model. The step of invoking a smart recognition function to recognize the scanning result and obtain a classification result further includes: obtaining 3D point cloud data for reconstructing the 3D model; and invoking the smart recognition function to analyze the 3D point cloud data and recognize a classification result of the 3D point cloud data, where the classification result includes first point cloud data corresponding to a target area of ​​the measurement object in the 3D point cloud data and second point cloud data corresponding to a non-target area of ​​the measurement object. Specifically, by recognizing 3D point cloud data used to reconstruct the 3D model using a smart recognition function, the 3D point cloud data is divided into first point cloud data and second point cloud data, the first point cloud data is point cloud data corresponding to a target area of ​​the measurement object in the 3D point cloud data, and the second point cloud data is point cloud data corresponding to a non-target area of ​​the measurement object in the 3D point cloud data. For example, when the measurement object is an oral cavity, the target area includes teeth and gums, The non-target areas include the tongue, lips, cheeks, etc., the first point cloud data is point cloud data corresponding to the teeth and gums, the second point cloud data is point cloud data corresponding to areas such as the tongue, lips, cheeks, etc., and the classification result includes teeth, gums, tongue, lips, and cheeks, but of course, if the tongue, lips, and cheeks are classified as others, the classification result can also include teeth, gums, and others, and if the teeth and gums are classified as target classes and the tongue, lips, and cheeks are classified as non-target classes (i.e. others), the classification result can also be classified as target classes and non-target classes.

[0039] In one embodiment of the present application, when the second point cloud data is determined to be the invalid data, the invalid data is removed from the 3D point cloud data to determine the point cloud data of the valid area in the 3D model. Specifically, after the second point cloud data is determined to be the invalid data, the invalid data in the 3D model is removed, thereby determining the valid area in the 3D model, i.e., the area corresponding to the measurement object.

[0040] In one embodiment of the present application, before the step of obtaining 3D point cloud data for reconstructing the 3D model, the method further includes a step of collecting a 2D image of the object to be measured by scanning the object to be measured, and a step of three-dimensionally reconstructing 3D point cloud data based on the 2D image and stitching the 3D model based on the reconstructed 3D point cloud data. There is a correspondence between the pixel points of the 2D image and the 3D point cloud data. Specifically, before the step of obtaining 3D point cloud data for reconstructing the 3D model, based on the collected 2D image of the object to be measured, 3D point cloud data corresponding to the object to be measured is three-dimensionally reconstructed, and the 3D model of the object to be measured can be stitched from the 3D point cloud data corresponding to the object to be measured.

[0041] In one embodiment of the present application, before the step of obtaining the scanning result of the object to be measured, the method further includes a step of starting and initializing the scanning process and the AI recognition process. The scanning process is used to scan the object to be measured, and the AI recognition process is used to recognize and classify the scanning result. Specifically, before the step of obtaining the scanning result of the object to be measured, the scanning process and the AI recognition process are started and initialized. The scanning process scans the object to be measured to obtain a scanning result, and the AI recognition process recognizes and classifies the scanning result to obtain a classification result. The initialization process clears the previous scanning result and classification result to avoid interference with the current processing process.

[0042] In one embodiment of the present application, in the process of initializing the scanning process and the AI ​​recognition process, it monitors whether the communication between the scanning process and the AI ​​recognition process is successfully established; after confirming the successful connection, if the scanning result is detected, the scanning process issues a processing command to the AI ​​recognition process, and the AI ​​recognition process calls the smart recognition function according to the processing command to recognize the scanning result. Specifically, in the process of initializing the scanning process and the AI ​​recognition process, it monitors whether the communication between the scanning process and the AI ​​recognition process is successfully established; if the communication connection is not established, it connects the two to communication; if the connection is successful, after the scanning result is detected, the scanning process issues a processing command to the AI ​​recognition process, and the AI ​​recognition process calls the smart recognition function to recognize the scanning result.

[0043] In one embodiment of the present application, in the process of monitoring whether the communication between the scanning process and the AI ​​recognition process is successfully established, when the AI ​​recognition process is executed in parallel and the operating environment meets a predetermined condition, the AI ​​recognition process initializes the recognition algorithm, receives the processing command, and executes the smart recognition function after the recognition algorithm is successfully initialized. Specifically, after the AI ​​recognition process is started, when the AI ​​recognition process is executed in parallel and the operating environment meets a predetermined condition, the AI ​​recognition process initializes the recognition algorithm to avoid the subsequent failure of the smart recognition function, and executes the smart recognition function to recognize the scanning result only when the recognition algorithm is successfully initialized and the processing command is received. That is, before the recognition algorithm is successfully initialized, even if the processing command is received, the AI ​​recognition process does not execute the smart recognition function.

[0044] It should be noted that the above AI recognition process is executed as an independent process. Of course, AI recognition and scanning can be set to be executed serially in the same process, but compared with the case where the AI ​​recognition process and the scanning process are serially executed together, when the AI ​​recognition process and the scanning process are independent of each other, the logic is clearer, and the AI ​​recognition function as a functional module reduces software coupling, and maintenance and modification are also easier. In addition, when requirements are added / changed, it is only necessary to add / change the corresponding communication protocol, which is highly flexible, and the AI ​​recognition function depends on the host hardware configuration and needs to perform checks and algorithm initialization, so it takes time to start up, and as an independent process, it is also reasonable in terms of software configuration.

[0045] During actual operation, the scanning process and the AI ​​recognition process establish communication and exchange data in the form of a shared memory, and in a specific embodiment of the present application, as shown in Figure 3, the AI ​​recognition process includes the steps of reading the texture image acquired by the scanning process, storing the read image data in a shared memory, inputting the image data into an AI recognition algorithm, outputting a result label, and writing the result label into the shared memory, and the result label corresponds to the point corresponding to the image data one-to-one. For example, when the measurement object is an oral cavity, the target area includes teeth and gums, and a result label of 0 represents others, a result label of 1 represents teeth, and a result label of 2 represents gums.

[0046] In a specific embodiment of the present application, as shown in Figure 4, the startup steps of the scanning process and the AI ​​recognition process are as follows: when the scanning process is started, the AI ​​recognition process is pulled up, that is, the AI ​​recognition process is started, the scanning process and the AI ​​recognition process are initialized, communication is established between the scanning process and the AI ​​recognition process, and after the connection is confirmed, the scanning process obtains the scanning result and issues a processing command to the AI ​​recognition process, and the AI ​​recognition process calls a smart recognition function according to the processing command to recognize the scanning result, writes the recognized result label into the shared memory, and the scanning process applies the result label to process the scanning result.

[0047] In a specific embodiment of the present application, as shown in Fig. 5, the steps of constructing a 3D model of a measurement object are as follows: obtain one frame of image data, reconstruct 3D point cloud data three-dimensionally based on the image data, activate the AI ​​smart recognition function if the reconstruction is successful, otherwise, return to obtain the image data of the next frame, if the activation of the AI ​​smart recognition function fails, directly stitch the 3D point cloud data to obtain the 3D model of the measurement object, if the activation of the AI ​​smart recognition function is successful, obtain the AI ​​recognition result, if the acquisition of the result times out, return to obtain the image data of the next frame, if the timeout does not occur, apply the AI ​​recognition result to process the 3D point cloud data, remove invalid point clouds, and stitch the remaining point clouds to obtain the 3D model of the measurement object.

[0048] In addition, the embodiments of the present application provide a scanning result processing device. It should be noted that the scanning result processing device in the embodiments of the present application can be used to implement the scanning result processing method provided in the embodiments of the present application. The scanning result processing device provided in the embodiments of the present application will be described below.

[0049] 6 is a flow chart of a scanning result processing device according to an embodiment of the present application. As shown in FIG. 6, the device includes: an acquisition unit 10 for acquiring a scan result of a measurement object, said scan result including a 2D image and / or a 3D model; a first recognition unit 20 for invoking a smart recognition function to recognize the scanning result and obtain a classification result, the smart recognition function being a classification model obtained by training image samples; The apparatus further comprises a first determining unit 30 for determining invalid data in the scanning result based on the classification result, where the invalid data is a scanning result of a non-target area of ​​the measurement object.

[0050] In the above processing device, the acquisition unit acquires the scanning result of the measurement object, i.e., the 2D image and / or the 3D model scanned by the scanner; the recognition unit calls the smart recognition function to recognize the scanning result and obtain the classification result, i.e., classify the 2D image and / or the 3D model according to the trained classification model; Decision Unit determines invalid data in the scanning result based on the classification result, i.e., determines invalid data in the two-dimensional image and / or the three-dimensional model based on the classification result, thereby eliminating the need to pause scanning to determine invalid data through data analysis, and improving scanning efficiency.

[0051] In one embodiment of the present application, the scanning result is the two-dimensional image, the two-dimensional image includes a texture image, and the classification result includes first image data corresponding to a target area of ​​the measurement object in the texture image and second image data corresponding to a non-target area of ​​the measurement object. Specifically, by recognizing the texture image with a smart recognition function, the texture image is divided into first image data and second image data, the first image data is image data corresponding to a target area of ​​the measurement object in the texture image, the second image data is image data corresponding to a non-target area of ​​the measurement object in the texture image, the target area and the non-target area are determined in advance, for example, when the measurement object is an oral cavity, the target area includes teeth and gums, the non-target area includes tongue, lips, cheeks, etc., and the first image data is image data corresponding to teeth and gums. the first image data is image data corresponding to areas such as the tongue, lips, cheeks, etc., and of course the target areas and non-target areas can be adjusted according to actual needs, for example, when only tooth data is required, the gums can be preset as a non-target area, and the classification result includes teeth, gums, tongue, lips, and cheeks, but of course, if the tongue, lips, and cheeks are classified as others, the classification result can also include teeth, gums, and others, and if the teeth and gums are classified as a target class and the tongue, lips, and cheeks are classified as a non-target class (i.e. others), the classification result can be classified as a target class and a non-target class.

[0052] In one embodiment of the present application, the two-dimensional image further includes a reconstructed image corresponding to the texture image; Decision UnitThe method includes a determination module, a first processing module, and a second processing module, the determination module is used to construct a three-dimensional point cloud based on the reconstructed image, and determine an invalid point cloud based on a correspondence between the reconstructed image and the texture image, the invalid point cloud being a point cloud corresponding to the second image data in the three-dimensional point cloud, the first processing module is used to remove the invalid point cloud in the three-dimensional point cloud, and the second processing module is used to stitch a valid three-dimensional model of the measurement object based on the remaining point cloud in the three-dimensional point cloud. Specifically, a three-dimensional point cloud is constructed based on the reconstructed image, and a point cloud corresponding to the second image data in the three-dimensional point cloud data is determined according to the correspondence between the reconstructed image and the texture image, thereby determining the invalid point cloud, and after removing the invalid point cloud, the remaining point cloud can be stitched to obtain a valid three-dimensional model of the measurement object. For example, a 2D image of a first frame and a 2D image of a second frame are obtained, the remaining point cloud of the first frame is obtained based on the 2D image of the first frame, the remaining point cloud of the second frame is obtained based on the 2D image of the second frame, and the remaining point cloud of the first frame and the remaining point cloud of the second frame are stitched together. More specifically, when the measurement object is the oral cavity, the target area includes the teeth and gums, and the non-target area includes the tongue, lips, cheeks, etc., and the point clouds corresponding to areas such as the tongue, lips, and cheeks are removed, and the remaining point clouds are stitched together to obtain a 3D model of the teeth and gums, as shown in FIG. 2.

[0053] In one embodiment of the present application, when the 3D model of the measurement object is successfully reconstructed, the scanning result is the 3D model, and the device also includes a second recognition unit, the second recognition unit includes a first acquisition module and a recognition module, the first acquisition module is used to acquire 3D point cloud data for reconstructing the 3D model, and the recognition module is used to call the smart recognition function to analyze the 3D point cloud data and recognize a classification result of the 3D point cloud data, the classification result includes a first point cloud data corresponding to a target area of ​​the measurement object in the 3D point cloud data, and a second point cloud data corresponding to a non-target area of ​​the measurement object. Specifically, the smart recognition function recognizes the 3D point cloud data used to reconstruct the 3D model, thereby dividing the 3D point cloud data into a first point cloud data and a second point cloud data, the first point cloud data being point cloud data corresponding to a target area of ​​the measurement object in the 3D point cloud data, and the second point cloud data being point cloud data corresponding to a non-target area of ​​the measurement object in the 3D point cloud data. For example, when the measurement object is the oral cavity, the target area includes the teeth and gums, and the non-target area includes the tongue, lips, cheeks, etc., the first point cloud data is point cloud data corresponding to the teeth and gums, the second point cloud data is point cloud data corresponding to areas such as the tongue, lips, cheeks, etc., and the classification result includes the teeth, gums, tongue, lips, and cheeks, but of course, if the tongue, lips, and cheeks are classified as others, the classification result can also include the teeth, gums, and others, and if the teeth and gums are classified as targets and the tongue, lips, and cheeks are classified as non-targets (i.e. others), the classification result can also be classified as targets and non-targets.

[0054] In one embodiment of the present application, the apparatus also includes a second determination unit, which is used to determine point cloud data of a valid area in the 3D model by removing the invalid data from the 3D point cloud data when the second point cloud data is determined to be the invalid data. Specifically, after the second point cloud data is determined to be the invalid data, the valid area in the 3D model, i.e., the area corresponding to the measurement object, is determined by removing the invalid data in the 3D model.

[0055] In one embodiment of the present application, the apparatus also includes a reconstruction unit, the reconstruction unit includes a second acquisition module and a reconstruction module, the second acquisition module is used to collect a two-dimensional image of the measurement object by scanning the measurement object before obtaining three-dimensional point cloud data for reconstructing the three-dimensional model, the reconstruction module is used to three-dimensionally reconstruct the three-dimensional point cloud data based on the two-dimensional image, and stitch the three-dimensional model based on the reconstructed three-dimensional point cloud data, and there is a correspondence between the pixel points of the two-dimensional image and the three-dimensional point cloud data. Specifically, before obtaining the three-dimensional point cloud data for reconstructing the three-dimensional model, the three-dimensional point cloud data corresponding to the measurement object can be three-dimensionally reconstructed based on the collected two-dimensional image of the measurement object, and the three-dimensional model of the measurement object can be stitched from the three-dimensional point cloud data corresponding to the measurement object.

[0056] In one embodiment of the present application, the apparatus also includes a control unit, which is used to start and initialize a scanning process and an AI recognition process before obtaining a scanning result of a measurement object, the scanning process is used to scan the measurement object, and the AI ​​recognition process is used to recognize and classify the scanning result. Specifically, before obtaining a scanning result of a measurement object, the scanning process and the AI ​​recognition process are started and initialized, the scanning process scans the measurement object to obtain a scanning result, the AI ​​recognition process recognizes and classifies the scanning result to obtain a classification result, and the initialization process clears previous scanning results and classification results to avoid interference with the current processing process.

[0057] In one embodiment of the present application, the control unit includes a first control module, which monitors whether communication between the scanning process and the AI ​​recognition process is successfully established during the process of initializing the scanning process and the AI ​​recognition process, and after confirming the successful connection, if the scanning result is detected, the scanning process issues a processing command to the AI ​​recognition process, and the AI ​​recognition process calls the smart recognition function according to the processing command to recognize the scanning result. Specifically, during the process of initializing the scanning process and the AI ​​recognition process, it monitors whether communication between the scanning process and the AI ​​recognition process is successfully established, and if a communication connection is not established, it connects the two to communication, and if the connection is successful, after the scanning result is detected, the scanning process issues a processing command to the AI ​​recognition process, and the AI ​​recognition process calls the smart recognition function to recognize the scanning result.

[0058] In one embodiment of the present application, the control unit includes a second control module, and in the process of monitoring whether the communication between the scanning process and the AI ​​recognition process is successfully established, when the AI ​​recognition process is executed in parallel and the operating environment meets a predetermined condition, the AI ​​recognition process initializes the recognition algorithm, receives the processing command, and executes the smart recognition function after the recognition algorithm is successfully initialized. Specifically, after starting the AI ​​recognition process, when the AI ​​recognition process is executed in parallel and the operating environment meets a predetermined condition, the AI ​​recognition process initializes the recognition algorithm to avoid the subsequent startup failure of the smart recognition function, and executes the smart recognition function to recognize the scanning result only when the initialization of the recognition algorithm is successful and the processing command is received. That is, before the initialization of the recognition algorithm is successful, the AI ​​recognition process does not execute the smart recognition function even if it receives a processing command.

[0059] Although the above AI recognition process is executed as an independent process, of course, AI recognition and scanning can also be set to run serially in the same process. Compared with the case where the AI ​​recognition process and the scanning process are executed serially together, when the AI ​​recognition process and the scanning process are independent, the logic is clearer. The AI ​​recognition function as a functional module reduces software coupling and makes maintenance and changes easier. In addition, when requirements are added / changed, it is only necessary to add / change the corresponding communication protocol, which is highly flexible. The AI ​​recognition function depends on the host hardware configuration and needs to perform checks and algorithm initialization, which takes time to start up. As an independent process, it is also reasonable in terms of software configuration.

[0060] According to an embodiment of the present invention there is further provided a scanning system comprising a scanner and a scanning result processing device, said scanning result processing device being adapted to perform any one of the above processing methods.

[0061] The scanning system includes a scanner and a scanning result processing device, in which the acquisition unit acquires the scanning result of the measurement object, i.e., acquires the 2D image and / or 3D model obtained from the scanner; the recognition unit calls the smart recognition function to recognize the scanning result and obtain the classification result, i.e., classifies the 2D image and / or 3D model according to the trained classification model; Decision Unit determines invalid data in the scanning result based on the classification result, i.e., determines invalid data in the two-dimensional image and / or the three-dimensional model based on the classification result, thereby eliminating the need to interrupt scanning to determine invalid data through data analysis, and further improving scanning efficiency.

[0062] the above Scanning result processing device has a processor and a memory, and the acquisition unit, the first recognition unit, the first determination unit, etc. are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.

[0063] The processor is provided with a kernel, and the kernel calls the corresponding program unit from the memory. One or more kernels can be set, and the problem of reduced scanning efficiency caused by the prior art method of processing the scanning result, which determines invalid data through data analysis, is solved by adjusting the parameters of the kernel.

[0064] The memory may include a form of non-volatile memory in a computer-readable medium, such as random access memory (RAM), and / or read only memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.

[0065] An embodiment of the present invention provides a storage medium having a program stored thereon, which, when executed by a processor, implements the above processing method.

[0066] An embodiment of the present invention provides a processor adapted to execute a program, which, when executed, performs the processing method described above.

[0067] An embodiment of the present invention provides an apparatus comprising a processor, a memory, and a program stored in the memory and executable on the processor, the program, when executed on the processor, performing at least: S101, acquiring a scan of a measurement object, the scan including a two-dimensional image and / or a three-dimensional model; Step S102: invoking a smart recognition function to recognize the scanning result and obtain a classification result, the smart recognition function being a classification model obtained by training image samples; and determining invalid data in the scanning result based on the classification result, where the invalid data is a scanning result of a non-target area of ​​the measurement object.

[0068] The device in this specification may be a server, a PC, a PAD, a mobile phone, etc.

[0069] The present application also provides a computer program product, which when executed on a data processing device, comprises at least: S101, acquiring a scan of a measurement object, the scan including a two-dimensional image and / or a three-dimensional model; Step S102: invoking a smart recognition function to recognize the scanning result and obtain a classification result, the smart recognition function being a classification model obtained by training image samples; A step S103 of determining invalid data in the scanning result based on the classification result, where the invalid data is the scanning result of the non-target area of ​​the measurement object is suitable for executing a program to initialize the step S103.

[0070] In the above embodiments of the present invention, the description of each embodiment is focused on its own, and for parts of the embodiments that are not described in detail, reference can be made to the relevant descriptions of other embodiments.

[0071] It should be understood that the disclosed technical contents in some embodiments provided in the present application can be implemented in other ways. Here, the above-mentioned device embodiments are merely examples, and for example, the division of the above units can be a logical division of functions, and can be actually implemented in other ways, for example, multiple units or components can be combined, or can be incorporated into another system, or some functions can be ignored or not performed. In other respects, the illustrated or discussed mutual couplings or direct couplings or communication connections can be via indirect couplings or communication connections of some interfaces, units or modules, and can be made in electrical or other ways.

[0072] The above units described as separate components may or may not be physically separated, and the parts shown as units may or may not be physical units, i.e., they may be located in one place or in multiple units. Some or all of these units may be selected according to practical needs to achieve the purpose of the solution of the present embodiment.

[0073] In addition, each functional unit in each embodiment of the present invention may be integrated into a single processing unit, each unit may exist physically independent, or two or more units may be integrated into a single unit. The integrated unit may be implemented in the form of hardware or a software functional unit.

[0074] The above integrated unit may be implemented as a software functional unit and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the essence of the technical solution of the present invention or a part that contributes to the prior art, or the whole or part of the technical solution, may be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes a plurality of commands for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above method in each embodiment of the present invention. The storage medium includes various media that can store program code, such as USB memory, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk, optical disk, etc.

[0075] From the above description, it is apparent that the above embodiments of the present application achieve the following technical effects.

[0076] 1) In the processing method of the present application, firstly, obtain the scanning result of the measurement object, i.e., obtain the 2D image and / or 3D model obtained from the scanner, then invoke the smart recognition function to recognize the scanning result and obtain the classification result, i.e., classify the 2D image and / or 3D model by the trained classification model, and finally determine the invalid data in the scanning result based on the classification result, i.e., remove the invalid data from the 2D image and / or 3D model based on the classification result, thereby eliminating the need to interrupt the scanning to determine the invalid data through data analysis, and further improving the scanning efficiency.

[0077] 2) In the processing device of the present application, the acquisition unit acquires the scanning result of the measurement object, i.e., acquires the 2D image and / or 3D model obtained from the scanner; the recognition unit calls the smart recognition function to recognize the scanning result and obtain the classification result, i.e., classifies the 2D image and / or 3D model by the trained classification model; Decision Unit determines invalid data in the scanning result based on the classification result, i.e., determines invalid data in the two-dimensional image and / or the three-dimensional model based on the classification result, thereby eliminating the need to interrupt scanning to determine invalid data through data analysis, and further improving scanning efficiency.

[0078] 3) The scanning system of the present application includes a scanner and a scanning result processing device, in which the acquisition unit acquires the scanning result of the measurement object, i.e., acquires the 2D image and / or 3D model obtained from the scanner; the recognition unit calls the smart recognition function to recognize the scanning result and obtain the classification result, i.e., classifies the 2D image and / or 3D model according to the trained classification model; Decision Unit determines invalid data in the scanning result based on the classification result, i.e., determines invalid data in the two-dimensional image and / or the three-dimensional model based on the classification result, thereby eliminating the need to interrupt scanning to determine invalid data through data analysis, and further improving scanning efficiency.

[0079] The above is only a preferred embodiment of the present application, and does not limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application. [Industrial Applicability]

[0080] The solution provided by the embodiments of the present application is to obtain a two-dimensional image and / or a three-dimensional model scanned by a scanner, classify the two-dimensional image and / or the three-dimensional model by a trained classification model, and remove invalid data from the two-dimensional image and / or the three-dimensional model based on the classification result. Therefore, it is not necessary to temporarily stop the scanning to determine invalid data by data analysis, the scanning efficiency is improved, and the problem of reducing the scanning efficiency by temporarily stopping the scanning, performing data analysis on the scanning result to determine invalid data, which is caused by the method for processing the scanning result in the prior art, is solved.

Claims

1. 1. A method for processing a scan result, comprising: acquiring a scan of a measurement object, the scan including a two-dimensional image and / or a three-dimensional model obtained by reconstructing the two-dimensional image; calling a smart recognition function to recognize the scanning result and obtain a classification result, the smart recognition function being a classification model obtained by training image samples, the smart recognition function being used to classify 2D images and / or 3D models according to the trained classification model; determining invalid data in the scan result based on the classification result, the invalid data being a scan result of a non-target area of ​​the measurement object, the non-target area being predetermined, the scan result being the two-dimensional image, the two-dimensional image including a texture image, and the classification result including first image data corresponding to the target area of ​​the measurement object in the texture image and second image data corresponding to the non-target area of ​​the measurement object; the two-dimensional image further includes a reconstructed image corresponding to the texture image; determining invalid data in the scanning result based on the classification result, constructing a three-dimensional point cloud based on the reconstructed image, and determining an invalid point cloud based on a correspondence between the reconstructed image and the texture image, the invalid point cloud being a point cloud corresponding to the second image data within the three-dimensional point cloud; removing invalid points in the three-dimensional point cloud; and stitching a valid 3D model of the measurement object based on the remaining points in the 3D point cloud. How to process the scan results.

2. If the three-dimensional model of the measurement object is successfully reconstructed, the scanning result is the three-dimensional model; Invoking a smart recognition function to recognize the scanning result and obtain a classification result further includes: acquiring three-dimensional point cloud data for reconstructing the three-dimensional model; Invoking the smart recognition function to analyze the 3D point cloud data and recognize a classification result for the 3D point cloud data; The classification result includes first point cloud data corresponding to a target area of ​​the measurement object in the three-dimensional point cloud data, and second point cloud data corresponding to a non-target area of ​​the measurement object.

2. A method for processing a scan result according to claim 1.

3. When the second point cloud data is determined to be the invalid data, the invalid data is removed from the three-dimensional point cloud data to determine point cloud data of a valid area in the three-dimensional model.

3. A method for processing a scan result according to claim 2.

4. Prior to the step of acquiring three-dimensional point cloud data for reconstructing the three-dimensional model, acquiring a two-dimensional image of the measurement object; and three-dimensionally reconstructing three-dimensional point cloud data based on the two-dimensional image, and stitching the three-dimensional model based on the reconstructed three-dimensional point cloud data. A method for processing a scan result according to claim 3.

5. Prior to the step of obtaining the scan result of the measurement object, the method further includes the step of initiating and initializing the scanning process and the AI ​​recognition process; The scanning process is used to perform a scan on the measurement object, and the AI ​​recognition process is used to recognize and classify the scan results. A method for processing a scanning result according to any one of claims 1 to 4.

6. In the process of initializing the scanning process and the AI ​​recognition process, monitor whether communication between the scanning process and the AI ​​recognition process is successfully established. After confirming the success of the connection, if the scanning process detects the scanning result, issue a processing command to the AI ​​recognition process; The AI ​​recognition process calls the smart recognition function based on the processing command to recognize the scan result. A method for processing a scan result according to claim 5.

7. In the process of monitoring whether communication has been successfully established between the scanning process and the AI ​​recognition process, the AI ​​recognition process is executed in parallel, and when an operating environment satisfies a predetermined condition, the AI ​​recognition process initializes a recognition algorithm, receives the processing command, and executes the smart recognition function after the recognition algorithm is successfully initialized. A method for processing a scan result according to claim 6.

8. Establishing communication between the scanning process and the AI ​​recognition process and exchanging data in the form of a shared memory; The AI ​​recognition process includes: reading a texture image obtained by said scanning process; storing said texture image in a shared memory; inputting the texture image into an AI recognition algorithm and outputting result labels that correspond one-to-one to corresponding points in the texture image; and writing the result label to the shared memory. A method for processing a scan result according to claim 5.

9. Constructing the three-dimensional model of the measurement object includes: acquiring a frame of image data; three-dimensionally reconstructing three-dimensional point cloud data based on the image data; If the reconstruction is successful, the AI ​​smart recognition function is activated; if the reconstruction is unsuccessful, the image data of the next frame is acquired; If the activation of the AI ​​smart recognition function fails, stitching the 3D point cloud data to obtain the 3D model of the measurement object; If the AI ​​smart recognition function is successfully activated, obtaining an AI recognition result; When the acquisition of the AI ​​recognition result times out, acquiring image data of the next frame; If the acquisition of the AI ​​recognition result has not timed out, applying the AI ​​recognition result to process the 3D point cloud data, removing invalid points, and stitching the remaining points in the 3D point cloud data to obtain the 3D model of the measurement object.

2. A method for processing a scan result according to claim 1.

10. The AI ​​recognition process is executed as an independent process, and the AI ​​recognition process and the scanning process are independent of each other. A method for processing a scan result according to claim 5.

11. 1. A scanning result processing device, comprising: an acquisition unit for acquiring a scan result of a measurement object, the scan result including a two-dimensional image and / or a three-dimensional model obtained by reconstructing the two-dimensional image; a first recognition unit for calling a smart recognition function to recognize the scanning result and obtain a classification result, the smart recognition function being a classification model obtained by training image samples, the smart recognition function being used to classify two-dimensional images and / or three-dimensional models according to the trained classification model; a first determination unit for determining invalid data in the scanning result based on the classification result, the invalid data being a scanning result of a non-target area of ​​the measurement object, the non-target area being predetermined, the scanning result being the two-dimensional image, the two-dimensional image including a texture image, and the classification result including first image data corresponding to the target area of ​​the measurement object in the texture image and second image data corresponding to the non-target area of ​​the measurement object; the two-dimensional image further includes a reconstructed image corresponding to the texture image; determining invalid data in the scanning result based on the classification result, constructing a three-dimensional point cloud based on the reconstructed image, and determining an invalid point cloud based on a correspondence relationship between the reconstructed image and the texture image, the invalid point cloud being a point cloud corresponding to the second image data within the three-dimensional point cloud; removing invalid points in the three-dimensional point cloud; and stitching a valid 3D model of the measurement object based on the remaining points in the 3D point cloud. A device for processing the scan results.

12. A computer readable storage medium containing a program stored thereon, The program is characterized in that it realizes the method for processing a scanning result according to any one of claims 1 to 10. A computer-readable storage medium.

13. A processor used to execute a program, The program, when executed, realizes the method for processing a scanning result according to any one of claims 1 to 10. Processor.

14. A scanning system comprising a scanner and a processing device for processing the scan results, The scanning result processing device is characterized in that it is used to realize the scanning result processing method according to any one of claims 1 to 10. Scanning system.

Images