Deformation detection method, device, equipment and medium for multi-step orthodontic appliance

Abstract

The present invention provides a method, device, equipment and medium for detecting deformation of a multi-step appliance. The embodiment of the present invention obtains images to be detected of n appliances respectively, and respectively extracts features of the image to be detected of the i-th appliance, the image to be detected of the i-1-th appliance and the image to be detected of the i+1-th appliance among the n appliances, and fuses the obtained features to obtain fused features, and then performs classification based on the fused features to obtain a classification result of whether the i-th appliance is deformed, and then, based on the classification result of whether the i-th appliance is deformed, determines whether the i-th appliance is deformed, and can accurately and effectively identify whether the multi-step appliance is deformed.

Description

[Technical field]

[0001] The present invention relates to the field of medical devices, and in particular to quality inspection technology for medical devices, and in particular to a deformation inspection method, device, equipment and medium for a multi-step orthodontic appliance. [Background technology]

[0002] Shell-shaped orthodontic appliances based on polymer materials (such as invisible braces) are becoming more and more popular due to their advantages of beauty, convenience and ease of cleaning. Shell-shaped orthodontic appliances use the resilience generated by deformation to reposition teeth from one configuration to another.

[0003] The production process of shell-shaped orthodontic appliances (hereinafter referred to as: appliances) usually needs to go through the following procedures: light-curing mold molding -> using light-curing mold to press the appliance -> appliance cutting and shelling -> appliance grinding and cleaning -> appliance sorting and packaging.

[0004] During the process of cutting, shelling, grinding, and cleaning, there is a certain chance that the braces will be deformed due to external forces or problems introduced by the light-curing mold. Deformed braces are defective products that cannot be worn to achieve the purpose of correcting teeth and need to be identified and processed.

[0005] In the prior art, the deformation of the orthodontic appliance is mainly identified in the following ways: during the production process, the operator performs a visual inspection; during the quality inspection stage, the quality inspectors perform a visual inspection; during the finished product quality inspection stage, the quality inspectors perform a single-step visual inspection (up and down warping detection) and a multi-step comparative visual inspection (arch expansion and retraction detection).

[0006] However, the above-mentioned method of mainly manually identifying defects of light-curing molds and orthodontic appliances has at least the following problems: due to production demand, quality control requirements and the limited number of quality inspectors, visual inspection is very arduous, which brings huge work pressure to operators and quality inspectors; the visual inspection of orthodontic appliance deformation is affected by subjective factors of operators and quality inspectors (such as vision, experience, fatigue level, etc.), and it is impossible to form an objective and unified evaluation standard, which is not conducive to quality control; some deformations are very difficult to judge, or are very subtle and difficult to detect.

[0007] Therefore, the existing technology cannot accurately and effectively identify the deformation of the orthodontic appliance by manually identifying the deformation of the orthodontic appliance, and it needs to occupy a large amount of human resources, the quality inspection cost is high, the time is long, the efficiency is low, and the quality of the orthodontic appliance cannot be effectively controlled. [Summary of the invention]

[0008] Various aspects of the present invention provide a method, device, equipment and computer-readable storage medium for detecting deformation of a multi-step appliance, for accurately and effectively identifying whether the multi-step appliance is deformed.

[0009] In one aspect of the present invention, a deformation detection method for a multi-step orthodontic appliance is provided, wherein the multi-step orthodontic appliance comprises n orthodontic appliances corresponding to n orthodontic steps, and the method comprises:

[0010] respectively acquiring the images to be detected of the n appliances;

[0011] For the i-th appliance among the n appliances, feature extraction is performed on the image to be detected of the i-th appliance, the image to be detected of the i-1-th appliance, and the image to be detected of the i+1-th appliance, respectively, to obtain features of the image to be detected of the i-1-th appliance, features of the image to be detected of the i-th appliance, and features of the image to be detected of the i+1-th appliance; wherein the value of i is any integer in [2, n-1] in sequence, and the value of n is greater than 2;

[0012] Fusing the features of the image to be detected of the (i-1)th appliance, the features of the image to be detected of the (i)th appliance, and the features of the image to be detected of the (i+1)th appliance to obtain a fused feature;

[0013] Performing classification based on the fusion features to obtain a classification result of whether the i-th appliance is deformed;

[0014] Based on the classification result of whether the i-th orthodontic appliance is deformed, it is determined whether the i-th orthodontic appliance is deformed.

[0015] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein after determining whether the i-th orthodontic appliance is deformed based on the classification result of whether the i-th orthodontic appliance is deformed, the method further includes:

[0016] using the determination result of whether the second orthodontic appliance is deformed as the determination result of whether the first orthodontic appliance is deformed;

[0017] The determination result of whether the n-1th orthodontic appliance is deformed is used as the determination result of whether the nth orthodontic appliance is deformed.

[0018] According to the aspects described above and any possible implementation, an implementation is further provided, wherein the deformation includes any one or more of the following: expansion of the arch, contraction of the arch.

[0019] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein after determining whether the i-th orthodontic appliance is deformed based on the classification result of whether the i-th orthodontic appliance is deformed, the method further includes:

[0020] Based on the determination result of whether the i-th appliance is deformed, a deformation detection result of the multi-step appliance is determined.

[0021] According to the above aspect and any possible implementation, an implementation is further provided, wherein respectively acquiring the images to be detected of the n appliances comprises:

[0022] The image acquisition system is used to acquire images of the n orthodontic appliances respectively in a top-view image acquisition manner to obtain the images to be detected of the n orthodontic appliances.

[0023] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance are fused to obtain the fused features, including:

[0024] Based on the attention mechanism, the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance are fused to obtain a fused feature.

[0025] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein based on the attention mechanism, the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance are fused to obtain fused features, including:

[0026] Fusing the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain an intermediate fusion feature;

[0027] Using the spatial attention module, obtaining spatial attention weight values ​​corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance;

[0028] Based on the spatial attention weight values ​​corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance, the intermediate fusion features are weighted to obtain the fusion features.

[0029] According to the aspects described above and any possible implementation method, an implementation method is further provided, using a pre-trained convolutional neural network to perform the operation of respectively extracting features from the image to be detected of the i-th appliance among the n appliances, the image to be detected of the i-th appliance, the image to be detected of the i-1-th appliance, and the image to be detected of the i+1-th appliance, to obtain features of the image to be detected of the i-1-th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1-th appliance, the operation of fusing the features of the image to be detected of the i-1-th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1-th appliance to obtain fused features, and the operation of performing classification based on the fused features to obtain a classification result of whether the i-th appliance is deformed.

[0030] According to the above aspects and any possible implementation, there is further provided an implementation, further comprising:

[0031] A training set is obtained, wherein the training set includes sample images of a plurality of samples, each sample including: m-2 groups of subsamples from m subsamples corresponding to m correction steps, where m is an integer greater than ; wherein each group of subsamples includes a k-th subsample, a k-1-th subsample, a k-th subsample, and a k+1-th subsample, and the value of k is any integer in [2, m-1] in sequence; in each group of subsamples, when a subsample in each group of subsamples is deformed, the k-th subsample is annotated as a positive sample with annotation information indicating the presence of deformation;

[0032] Inputting the k-th subsample, the k-1-th subsample, the k-th subsample and the k+1-th subsample included in each group of subsamples in the training set into the convolutional neural network to be trained respectively, and the convolutional neural network to be trained outputs a prediction classification result of whether the k-th subsample is deformed;

[0033] According to the labeling information and predicted classification results corresponding to each group of sub-samples, the convolutional neural network to be trained is trained until a preset training completion condition is met to obtain the convolutional neural network.

[0034] Another aspect of the present invention provides a deformation detection device for a multi-step orthodontic appliance, wherein the multi-step orthodontic appliance comprises n orthodontic appliances corresponding to n orthodontic steps, and the device comprises:

[0035] A first acquisition module is used to respectively acquire the images to be detected of the n orthodontic appliances;

[0036] A feature extraction module, for extracting features of the image to be detected of the i-th appliance among the n appliances, the image to be detected of the i-th appliance, the image to be detected of the i-1-th appliance, and the image to be detected of the i+1-th appliance, respectively, to obtain features of the image to be detected of the i-1-th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1-th appliance; wherein the value of i is any integer in [2, n-1] in sequence, and the value of n is greater than 2;

[0037] a feature fusion module, used for fusing the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain a fused feature;

[0038] A classification module, used for performing classification based on the fusion features to obtain a classification result of whether the i-th appliance is deformed;

[0039] The first determination module is used to determine whether the i-th orthodontic appliance is deformed based on the classification result of whether the i-th orthodontic appliance is deformed.

[0040] According to the above aspect and any possible implementation manner, an implementation manner is further provided, wherein the first determining module is further configured to:

[0041] using the determination result of whether the second orthodontic appliance is deformed as the determination result of whether the first orthodontic appliance is deformed;

[0042] The determination result of whether the n-1th orthodontic appliance is deformed is used as the determination result of whether the nth orthodontic appliance is deformed.

[0043] According to the aspects described above and any possible implementation, an implementation is further provided, wherein the deformation includes any one or more of the following: expansion of the arch, contraction of the arch.

[0044] According to the above aspects and any possible implementation, there is further provided an implementation, further comprising:

[0045] The second determination module is used to determine the deformation detection result of the multi-step orthodontic appliance based on the determination result of whether the i-th orthodontic appliance is deformed.

[0046] According to the above aspect and any possible implementation manner, an implementation manner is further provided, wherein the first acquisition module is specifically configured to:

[0047] The image acquisition system is used to acquire images of the n orthodontic appliances respectively in a top-view image acquisition manner to obtain the images to be detected of the n orthodontic appliances.

[0048] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the feature fusion module is specifically used to:

[0049] Based on the attention mechanism, the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance are fused to obtain a fused feature.

[0050] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the feature fusion module includes:

[0051] a feature fusion unit, used to fuse the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain an intermediate fusion feature;

[0052] A spatial attention module, used to obtain spatial attention weight values ​​corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance;

[0053] A weighting module is used to weight the intermediate fusion feature based on the spatial attention weight values ​​corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain the fusion feature.

[0054] As described above, the aspects and any possible implementation methods further provide an implementation method, wherein the feature extraction module, the feature fusion module and the classification module are implemented by a pre-trained convolutional neural network.

[0055] According to the above aspects and any possible implementation, there is further provided an implementation, further comprising:

[0056] A second acquisition module is used to acquire a training set, wherein the training set includes sample images of multiple samples, each sample including: m-2 groups of subsamples from m subsamples corresponding to m correction steps, where m is an integer greater than ; wherein each group of subsamples includes a k-th subsample, a k-1-th subsample, a k-th subsample, and a k+1-th subsample, and the value of k is any integer in [2, m-1] in sequence; in each group of subsamples, when a subsample in each group of subsamples is deformed, the k-th subsample is annotated as a positive sample with annotation information indicating the presence of deformation;

[0057] The training module is used to input the k-th subsample included in each group of subsamples in the training set into the k-1-th subsample, the k-th subsample and the k+1-th subsample into the convolutional neural network to be trained, and the convolutional neural network to be trained outputs the predicted classification result of whether the k-th subsample is deformed; and according to the labeling information and the predicted classification result corresponding to each group of subsamples, the convolutional neural network to be trained is trained until the preset training completion condition is met to obtain the convolutional neural network.

[0058] Another aspect of the present invention provides a device, comprising:

[0059] one or more processors;

[0060] a storage device for storing one or more programs,

[0061] When the one or more programs are executed by the one or more processors, the one or more processors implement the deformation detection method of the multi-step orthodontic appliance provided in the above aspect.

[0062] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the deformation detection method for a multi-step orthodontic appliance as provided in the above aspect.

[0063] It can be seen from the above technical scheme that the embodiment of the present invention obtains the images to be detected of n appliances respectively, and respectively extracts features of the images to be detected of the i-th appliance among the n appliances, the i-1-th appliance and the i+1-th appliance, obtains the features of the image to be detected of the i-1-th appliance, the i-th appliance and the i+1-th appliance, and fuses them to obtain fused features, and then classifies them based on the fused features to obtain a classification result of whether the i-th appliance is deformed, and then determines whether the i-th appliance is deformed based on the classification result of whether the i-th appliance is deformed. Therefore, the embodiment of the present invention realizes the detection of deformation of multi-step appliances based on computer vision technology, and can accurately and effectively identify whether the multi-step appliances are deformed.

[0064] In addition, by adopting the technical solution provided by the present invention, deformation detection of multi-step orthodontic appliances is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of multi-step orthodontic appliances, avoiding the occupation of operators and quality inspectors, saving human resources, reducing quality inspection costs, reducing quality inspection time, improving quality inspection efficiency, and reducing the workload of operators and quality inspectors, thereby improving production efficiency and helping to increase production capacity.

[0065] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance, effectively controlling the factory quality of the multi-step appliance, and contributing to the formation of a complete quality traceability and related feedback mechanism.

[0066] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance. Deformations that are difficult for the human eye can be detected, thereby effectively handling difficult-to-identify examples and improving the level of quality control.

[0067] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance, with high detection accuracy and reliability in complex production environments.

Brief Description of the Drawings

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0069] Figure 1 A schematic diagram of a flow chart of a deformation detection method for a multi-step appliance provided in one embodiment of the present invention;

[0070] Figure 2 A schematic diagram of a process of a multi-step appliance deformation detection method provided by another embodiment of the present invention;

[0071] Figure 3 A schematic structural diagram of a deformation detection device for a multi-step orthodontic appliance provided by another embodiment of the present invention;

[0072] Figure 4 A schematic structural diagram of a deformation detection device for a multi-step orthodontic appliance provided by another embodiment of the present invention;

[0073] Figure 5 is a block diagram of an exemplary computer system / server that is suitable for use in implementing embodiments of the present invention. [Specific implementation method]

[0074] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] It should be noted that the terminals involved in the embodiments of the present invention may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers, personal computers (PCs), MP3 players, MP4 players, wearable devices (for example, smart glasses, smart watches, smart bracelets, etc.), etc.

[0076] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0077] The main idea of ​​the present invention is to perform deformation detection on a multi-step appliance based on computer vision technology. By classifying the fused features obtained by fusing the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance and the features of the image to be detected of the i+1th appliance, it is determined whether the i-th appliance is deformed, thereby accurately and effectively identifying whether the appliance is deformed.

[0078] Figure 1 The flowchart of the deformation detection method of the multi-step orthodontic appliance provided in one embodiment of the present invention is as follows. The multi-step orthodontic appliance in the embodiment of the present invention includes n orthodontic appliances corresponding to n orthodontic steps. Figure 1 shown.

[0079] 101. Obtain n images of orthodontic appliances to be inspected respectively.

[0080] 102. For the ith appliance among the n appliances, feature extraction is performed on the image to be detected of the ith appliance, the image to be detected of the i-1th appliance, and the image to be detected of the i+1th appliance, respectively, to obtain features of the image to be detected of the i-1th appliance, features of the image to be detected of the ith appliance, and features of the image to be detected of the i+1th appliance.

[0081] The value of i is any integer in [2, n-1], and the value of n is greater than 2.

[0082] 103. Fuse the features of the image to be detected of the (i-1)th appliance, the features of the image to be detected of the (i)th appliance, and the features of the image to be detected of the (i+1)th appliance to obtain a fused feature.

[0083] 104. Classify based on the fusion features to obtain a classification result of whether the i-th appliance is deformed.

[0084] Optionally, in a possible implementation of the present embodiment, the classification result of whether the i-th appliance is deformed may be: the i-th appliance is deformed or the i-th appliance is not deformed, or it may be the probability that the i-th appliance is deformed and the probability that the i-th appliance is not deformed.

[0085] 105. Based on the classification result of whether the ith orthodontic appliance is deformed, determine whether the ith orthodontic appliance is deformed.

[0086] Optionally, in a possible implementation of the present embodiment, if the classification result of whether the i-th appliance is deformed is: the i-th appliance is deformed or the i-th appliance is not deformed, then the classification result is directly used as the result of whether the i-th appliance is deformed; if the classification result of whether the i-th appliance is deformed is: the i-th appliance is deformed or the i-th appliance is not deformed, or it can be the probability of the i-th appliance being deformed and the probability of the i-th appliance not being deformed, then whether the i-th appliance is deformed can be determined based on the size between the probability of deformation and the probability of non-deformation, and the classification result with a larger probability is selected as the result of whether the i-th appliance is deformed.

[0087] It should be noted that part or all of the execution entities of 101 to 105 may be applications located in the local terminal, i.e., the terminal device of the service provider, or may also be functional units such as plug-ins or software development kits (SDK) set in the applications located in the local terminal, or may also be processing engines located in the network-side server, or may also be distributed systems located on the network side, and this embodiment does not specifically limit this.

[0088] It is understandable that the application may be a local program (nativeApp) installed on the terminal, or may be a webpage program (webApp) of a browser on the terminal, and this embodiment does not specifically limit this.

[0089] In practical applications, according to whether one or more braces are needed to complete the entire correction process for each case, the braces can be divided into single-step braces and multi-step braces. Among them, a single-step brace is a brace that can complete the complete correction of a case with one brace; a multi-step brace includes multiple braces, which are used to complete the complete correction of a case in sequence according to the corresponding correction steps.

[0090] The inventors have found through research that arch expansion and arch retraction mainly occur in multi-step orthodontic appliance solutions. The orthodontic appliance has the phenomenon of arch expansion and arch retraction, that is, the orthodontic appliance has no up and down warping but the arch angle is too large or too small. Arch expansion and arch retraction are currently mainly judged by quality inspectors who hold a full set of orthodontic appliances for a certain case or a series of orthodontic appliances before entering the sorting line.

[0091] In this embodiment, the deformation of the multi-step appliance is detected based on computer vision technology, and whether the multi-step appliance is deformed can be accurately and effectively identified.

[0092] In addition, by adopting the technical solution provided by the present invention, deformation detection of multi-step orthodontic appliances is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of multi-step orthodontic appliances, avoiding the occupation of operators and quality inspectors, saving human resources, reducing quality inspection costs, reducing quality inspection time, improving quality inspection efficiency, and reducing the workload of operators and quality inspectors, thereby improving production efficiency and helping to increase production capacity.

[0093] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance, effectively controlling the factory quality of the multi-step appliance, and contributing to the formation of a complete quality traceability and related feedback mechanism.

[0094] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance. Deformations that are difficult for the human eye can be detected, thereby effectively handling difficult-to-identify examples and improving the level of quality control.

[0095] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance, with high detection accuracy and reliability in complex production environments.

[0096] Optionally, in a possible implementation of the present embodiment, after 105, the determination result of whether the second appliance is deformed can be used as the determination result of whether the first appliance is deformed; and the determination result of whether the n-1th appliance is deformed can be used as the determination result of whether the nth appliance is deformed.

[0097] Optionally, in a possible implementation of this embodiment, the deformation may include any one or more of the following: expansion of the arch, contraction of the arch.

[0098] Optionally, in a possible implementation of the present embodiment, after 105, the deformation detection result of the multi-step appliance can also be determined based on the determination result of whether the i-th appliance is deformed. When there is a deformed appliance among the n appliances, it is determined that the multi-step appliance is deformed; when there is no deformed appliance among the n appliances, it is determined that the multi-step appliance is not deformed.

[0099] Based on this embodiment, whether the multi-step appliance is deformed can be determined by detecting whether the i-th appliance is deformed.

[0100] Optionally, in a possible implementation of this embodiment, in 101, an image acquisition system can be used to acquire images of n braces respectively by using a top-view image acquisition method to obtain images of n braces to be detected. In a specific implementation, the aperture and focal length of the industrial camera can be adjusted according to the different materials of the braces. In addition, a backlight is used for the braces, and the brightness and color of the light source are adjusted so that the industrial camera can acquire clear images to be detected.

[0101] Optionally, in a possible implementation of the present embodiment, in 103, based on the attention mechanism, the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance may be fused to obtain fused features.

[0102] In a specific implementation, the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance can be first fused to obtain intermediate fused features, and then, the spatial attention module can be used to obtain the spatial attention weight values ​​corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance, and then, based on the spatial attention weight values ​​corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance, the intermediate fused features are weighted to obtain fused features.

[0103] In a specific example, the fusion feature is obtained by using the spatial attention module, which can be expressed as:

[0104] M s (F g )=σ(f 7×7 ([AvgPool(F g );MaxPool(F g )])

[0105] Among them, M s (F g ) represents fusion features, AvgPool represents average pooling, and MaxPool represents maximum pooling; f7*7 is a convolution operation with a convolution kernel of 7*7; σ is an activation function (sigmoid) operation, and the value after the sigmoid operation is between 0 and 1, which plays an attention effect.

[0106] The principle of attention is: the attention weight value is between 0 and 1. After the attention weight value is obtained, it will be multiplied by the corresponding feature to act as a soft mask. Intuitively speaking, where the attention weight value is relatively large (i.e. close to 1), the feature can be completely retained, and where the attention weight value is relatively large (i.e. close to 0), the feature will be eliminated (it tends to 0 after multiplication). The attention weight value can be obtained through pre-learning, so that the attention module can automatically adjust the position that deserves attention (i.e. the position that has a greater decisive effect on the final classification result).

[0107] Based on this embodiment, a feature fusion module based on the attention mechanism is proposed. By utilizing the spatial attention mechanism, it is possible to retain important information and eliminate redundant information in the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance, thereby achieving effective detection of whether the appliance is deformed.

[0108] Optionally, in a possible implementation of this embodiment, the convolutional neural network to be trained may be pre-trained using a training set to obtain a convolutional neural network, and the above operations 102 to 104 may be performed using the pre-trained convolutional neural network.

[0109] Optionally, in a possible implementation of this embodiment, the convolutional neural network may include a feature extraction module (full convolution module), a feature fusion module and a classification module (classifier) ​​to implement. Figure 2 , which is a schematic diagram of the process of a multi-step appliance deformation detection method provided by another embodiment of the present invention.

[0110] In a specific example, the feature extraction module can be implemented by a full convolution module, for example, it can be implemented by using the convolution layer in networks such as VGG, AlexNet, and ResNet.

[0111] In a specific example, the feature fusion module can be implemented by commonly used feature fusion methods, such as max pooling, average pooling or stacking, and can also be further implemented by a feature fusion module based on an attention mechanism.

[0112] In a specific example, the classification module can be implemented by a convolutional neural network consisting of a fully connected layer, an activation layer, a dropout layer, a sigmoid layer, or a softmax layer.

[0113] Optionally, in a possible implementation of this embodiment, the convolutional neural network may be trained in the following manner:

[0114] First, a training set is obtained, wherein the training set includes sample images of multiple samples, each sample includes: m-2 groups of subsamples from m subsamples corresponding to m correction steps, where m is an integer greater than ; wherein each group of subsamples includes a k-th subsample, a k-1-th subsample, a k-th subsample, and a k+1-th subsample, and the value of k is any integer in [2, m-1] in sequence; in each group of subsamples, when a subsample in each group of subsamples is deformed, the k-th subsample is annotated as a positive sample with annotation information of the deformation;

[0115] Then, the k-th subsample, the k-1-th subsample, the k-th subsample and the k+1-th subsample included in each group of subsamples in the training set are respectively input into the convolutional neural network to be trained, and the convolutional neural network to be trained outputs a prediction classification result of whether the k-th subsample has deformation;

[0116] Then, according to the labeling information and predicted classification results corresponding to each group of sub-samples, the convolutional neural network to be trained is trained until the preset training completion conditions are met to obtain a convolutional neural network.

[0117] Optionally, in a possible implementation of this embodiment, the preset training completion condition may include, but is not limited to: the number of training times for the convolutional neural network to be trained reaches a preset number of times, and / or the difference between the labeling information corresponding to the samples in the training set and the predicted classification results is less than a preset threshold.

[0118] Optionally, in a possible implementation of this embodiment, the training set may be obtained in the following manner:

[0119] The appliances of the complete correction steps in the multi-step correction appliances of a large number of cases can be collected in advance, wherein in some cases, one or more appliances used in the m correction steps have defects of arch expansion or arch retraction, and in some cases, the appliances used in all correction steps of the multi-step correction appliances are normal (without defects of arch expansion or arch retraction);

[0120] Each multi-step appliance in all cases is taken as a sample, and all samples are respectively imaged by an image acquisition system in a top-view image acquisition mode to obtain sample images of multiple samples;

[0121] For each sample, for the subsample applied to any correction step k, the value of k is any integer in [2, m-1] in turn. If there is arch expansion or arch contraction in the k-1th subsample, the kth subsample and the k+1th subsample, the k-1th subsample is marked as deformed, thereby transforming the m-step orthodontic appliance into m-2 groups of subsamples and their labels.

[0122] The above operation is performed on each sample to obtain the training set.

[0123] Optionally, in a possible implementation of the present embodiment, in the process of training the convolutional neural network to be trained, a transfer learning method can be used to initialize the parameters of the convolutional neural network to be trained to the parameter values ​​pre-trained by ImageNet, and then on this basis, a suitable learning update strategy is adopted to adjust the parameters of the convolutional neural network to be trained using a classified data set (i.e., a training set), thereby realizing the training of the convolutional neural network to be trained.

[0124] In the model training based on deep learning, since the model contains many parameters, in order to achieve the purpose of training convergence and better results, a large amount of training data (millions of data) is required, and it is usually impossible to collect and annotate such a large data set. ImageNet is a relatively large public data set. Researchers have done a lot of research on this data set, and a classification model with good results has been trained on this data set. In this embodiment, the transfer learning method is adopted to initialize the parameters of the convolutional neural network to be trained to the parameter values ​​of ImageNet pre-training. On this basis, a suitable learning update strategy is adopted to update the parameters of the convolutional neural network to be trained using the classification data set, which can accelerate the convergence of the convolutional neural network to be trained and improve the training effect.

[0125] The technical solution of the embodiment of the present invention based on deep learning can usually be implemented through the following process when applied:

[0126] a), acquiring a sample and collecting an electronic representation (i.e., an image) of the sample;

[0127] b), annotate the sample images with corresponding annotation information to form a data set, and divide the data set into a training set, a validation set, and a test set according to a certain ratio;

[0128] c) Design a deep learning model (i.e., the convolutional neural network to be trained), train the deep learning model using the training set; set evaluation indicators, and select the best deep learning model through the validation set;

[0129] d) Set evaluation indicators to evaluate the performance of the selected deep learning model test set. If the deployment requirements are not met, return to step a) to continue collecting samples and continue the process, or return to step c) to change the design of the deep learning model and continue the process; if the deployment requirements are met, enter the deployment phase of step e);

[0130] e), deploy the deep learning model, obtain the image to be detected, and apply the deep learning model deployed in d) to perform deformation detection of the multi-step orthodontic appliance.

[0131] f) Post-process the deformation detection results. If it is confirmed that the multi-step appliance is deformed, provide text feedback or alarm.

[0132] In this embodiment, a deep learning-based approach is adopted to utilize a pre-trained convolutional neural network to implement deformation detection of a multi-step appliance, which can accurately and effectively identify whether the appliance is deformed.

[0133] In this embodiment, the deformation of the multi-step appliance is detected based on computer vision technology, and whether the multi-step appliance is deformed can be accurately and effectively identified.

[0134] In addition, by adopting the technical solution provided by the present invention, deformation detection of multi-step orthodontic appliances is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of multi-step orthodontic appliances, avoiding the occupation of operators and quality inspectors, saving human resources, reducing quality inspection costs, reducing quality inspection time, improving quality inspection efficiency, and reducing the workload of operators and quality inspectors, thereby improving production efficiency and helping to increase production capacity.

[0135] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance, effectively controlling the factory quality of the multi-step appliance, and contributing to the formation of a complete quality traceability and related feedback mechanism.

[0136] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance. Deformations that are difficult for the human eye can be detected, thereby effectively handling difficult-to-identify examples and improving the level of quality control.

[0137] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance, with high detection accuracy and reliability in complex production environments.

[0138] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0139] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0140] Figure 3 A structural schematic diagram of a deformation detection device for a multi-step orthodontic appliance provided by another embodiment of the present invention is shown in FIG. Figure 3As shown. The multi-step orthodontic appliance in this embodiment includes n orthodontic appliances corresponding to n orthodontic steps. The deformation detection device of the multi-step orthodontic appliance in this embodiment may include a first acquisition module 301, a feature extraction module 302, a feature fusion module 303, a classification module 304 and a first determination module 305. Among them, the first acquisition module 301 is used to respectively acquire the images to be detected of the n orthodontic appliances; the feature extraction module 302 is used to respectively extract features of the image to be detected of the i-th orthodontic appliance among the n orthodontic appliances, the image to be detected of the i-1-th orthodontic appliance and the image to be detected of the i+1-th orthodontic appliance, and obtain the features of the image to be detected of the i-1-th orthodontic appliance, the features of the image to be detected of the i-th orthodontic appliance and the features of the image to be detected of the i+1-th orthodontic appliance; wherein the values ​​of i are [2, n -1], the value of n is greater than 2; a feature fusion module 303, used for fusing the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance and the features of the image to be detected of the i+1th appliance to obtain a fused feature; a classification module 304, used for performing classification based on the fused feature to obtain a classification result of whether the i-th appliance is deformed; a determination module 305, used for determining whether the i-th appliance is deformed based on the classification result of whether the i-th appliance is deformed.

[0141] It should be noted that part or all of the deformation detection device of the multi-step orthodontic appliance provided in this embodiment may be an application located in the local terminal, or may also be a functional unit such as a plug-in or software development kit (SDK) set in the application located in the local terminal, or may also be a search engine located in a network-side server, or may also be a distributed system located on the network side. This embodiment does not specifically limit this.

[0142] It is understandable that the application may be a local program (nativeApp) installed on the terminal, or may be a webpage program (webApp) of a browser on the terminal, and this embodiment does not specifically limit this.

[0143] Optionally, in a possible implementation of the present embodiment, the first determination module 305 is further used to: use the determination result of whether the second appliance is deformed as the determination result of whether the first appliance is deformed; use the determination result of whether the n-1th appliance is deformed as the determination result of whether the nth appliance is deformed.

[0144] Optionally, in a possible implementation of this embodiment, the deformation may include any one or more of the following: expansion of the arch, contraction of the arch, etc.

[0145] Figure 4 A schematic diagram of the structure of a deformation detection device for a multi-step orthodontic appliance provided by another embodiment of the present invention. Figure 4 As shown, the deformation detection device for a multi-step appliance of this embodiment further includes: a second determination module 401, which is used to determine the deformation detection result of the multi-step appliance based on the determination result of whether the i-th appliance is deformed.

[0146] Optionally, in a possible implementation of this embodiment, the first acquisition module 301 is specifically used to: utilize an image acquisition system to respectively acquire images of the n orthodontic appliances in a top-view image acquisition manner to obtain images of the n orthodontic appliances to be detected.

[0147] Optionally, in a possible implementation of the present embodiment, the feature fusion module 303 is specifically used to: based on the attention mechanism, fuse the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain a fused feature.

[0148] Optionally, in a possible implementation of the present embodiment, the feature fusion module 303 may include: a feature fusion unit, used to fuse the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain an intermediate fusion feature; a spatial attention module, used to obtain the spatial attention weight value corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance; a weighting module, used to weight the intermediate fusion feature based on the spatial attention weight value corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain the fusion feature.

[0149] Optionally, in a possible implementation of this embodiment, the feature extraction module 302, the feature fusion module 303 and the classification module 304 are implemented by a pre-trained convolutional neural network.

[0150] In addition, see Figure 4, the deformation detection device for the multi-step orthodontic appliance of the above embodiment may further include: a second acquisition module 402 and a training module 403. The second acquisition module 402 is used to acquire a training set, wherein the training set includes sample images of multiple samples, each sample including: m-2 groups of subsamples corresponding to m subsamples applied to m correction steps, where m is an integer greater than; wherein each group of subsamples includes the k-th subsample in the k-1-th subsample, the k-th subsample and the k+1-th subsample, and the value of k is any integer in [2, m-1] in sequence; in each group of subsamples, when there is a subsample in each group of subsamples that is deformed, the k-th subsample is annotated as a positive sample. There is deformation annotation information; a training module 403, which is used to input the k-th subsample included in each group of subsamples in the training set into the k-1-th subsample, the k-th subsample and the k+1-th subsample into the convolutional neural network to be trained, and the convolutional neural network to be trained outputs the predicted classification result of whether the k-th subsample has deformation; and according to the annotation information and the predicted classification result corresponding to each group of subsamples, the convolutional neural network to be trained is trained until the preset training completion condition is met to obtain the convolutional neural network.

[0151] It should be noted that Figure 1-2 The corresponding method in the embodiment can be obtained by Figure 3-4 The deformation detection device of the multi-step orthodontic appliance provided in the embodiment shown is implemented. Figure 1-2 The relevant contents in the corresponding embodiments will not be repeated here.

[0152] In this embodiment, the deformation of the multi-step appliance is detected based on computer vision technology, and whether the multi-step appliance is deformed can be accurately and effectively identified.

[0153] In addition, by adopting the technical solution provided by the present invention, deformation detection of multi-step orthodontic appliances is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of multi-step orthodontic appliances, avoiding the occupation of operators and quality inspectors, saving human resources, reducing quality inspection costs, reducing quality inspection time, improving quality inspection efficiency, and reducing the workload of operators and quality inspectors, thereby improving production efficiency and helping to increase production capacity.

[0154] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance, effectively controlling the factory quality of the multi-step appliance, and contributing to the formation of a complete quality traceability and related feedback mechanism.

[0155] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance. Deformations that are difficult for the human eye can be detected, thereby effectively handling difficult-to-identify examples and improving the level of quality control.

[0156] In addition, by adopting the technical solution provided by the present invention, deformation detection of the multi-step appliance is performed based on computer vision technology, thereby achieving efficient identification of deformation problems of the multi-step appliance, with high detection accuracy and reliability in complex production environments.

[0157] Figure 5 A block diagram of an exemplary computer system / server suitable for use in implementing embodiments of the present invention is shown. Figure 5 The computer system / server 12 shown is merely an example and should not bring any limitation to the functionality and scope of use of embodiments of the present invention.

[0158] like Figure 5 As shown, computer system / server 12 is represented in the form of a general-purpose computing device. Components of computer system / server 12 may include, but are not limited to: one or more processors or processing units 16, storage devices or system memory 28, and buses 18 that connect different system components (including system memory 28 and processing units 16).

[0159] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0160] Computer system / server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer system / server 12, including volatile and nonvolatile media, removable and non-removable media.

[0161] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer system / server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0162] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0163] The computer system / server 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer system / server 12, and / or any device that enables the computer system / server 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 44. In addition, the computer system / server 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer system / server 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer system / server 12, including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0164] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing Figure 1 The image processing method provided by the corresponding embodiment.

[0165] Another embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements Figure 1 The corresponding embodiment provides a deformation detection method for a multi-step orthodontic appliance.

[0166] Specifically, any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0167] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0168] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0169] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0170] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0171] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0172] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0174] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting deformation of a multi-step appliance, characterized in that: The multi-step orthodontic appliance comprises n orthodontic appliances corresponding to n orthodontic steps, and the method comprises: respectively acquiring the images to be detected of the n appliances; For the i-th appliance among the n appliances, feature extraction is performed on the image to be detected of the i-th appliance, the image to be detected of the i-1-th appliance, and the image to be detected of the i+1-th appliance, respectively, to obtain features of the image to be detected of the i-1-th appliance, features of the image to be detected of the i-th appliance, and features of the image to be detected of the i+1-th appliance; wherein the value of i is any integer in [2, n-1] in sequence, and the value of n is greater than 2; Fusing the features of the image to be detected of the (i-1)th appliance, the features of the image to be detected of the (i)th appliance, and the features of the image to be detected of the (i+1)th appliance to obtain a fused feature; Performing classification based on the fusion features to obtain a classification result of whether the i-th appliance is deformed; Based on the classification result of whether the i-th orthodontic appliance is deformed, it is determined whether the i-th orthodontic appliance is deformed.

2. The method according to claim 1, characterized in that After determining whether the i-th orthodontic appliance is deformed based on the classification result of whether the i-th orthodontic appliance is deformed, the method further includes: The determination result of whether the second appliance is deformed is used as the determination result of whether the first appliance is deformed; The determination result of whether the n-1th orthodontic appliance is deformed is used as the determination result of whether the nth orthodontic appliance is deformed.

3. The method according to claim 2, characterized in that The deformation includes any one or more of the following: expansion of the arch, contraction of the arch.

4. The method according to claim 2, characterized in that: After determining whether the i-th orthodontic appliance is deformed based on the classification result of whether the i-th orthodontic appliance is deformed, the method further includes: Based on the determination result of whether the i-th appliance is deformed, a deformation detection result of the multi-step appliance is determined.

5. The method according to claim 1, characterized in that The step of respectively acquiring the images to be detected of the n appliances comprises: The image acquisition system is used to acquire images of the n orthodontic appliances respectively in a top-view image acquisition manner to obtain the images to be detected of the n orthodontic appliances.

6. The method according to claim 1, characterized in that The step of fusing the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain the fused features includes: Based on the attention mechanism, the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance are fused to obtain a fused feature.

7. The method according to claim 6, characterized in that Based on the attention mechanism, the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance are fused to obtain fused features, including: Fusing the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain an intermediate fusion feature; Using the spatial attention module, obtaining spatial attention weight values ​​corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance; Based on the spatial attention weight values ​​corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance, the intermediate fusion features are weighted to obtain the fusion features.

8. The method according to any one of claims 1 to 7, characterized in that: Using the pre-trained convolutional neural network, perform the operations of respectively extracting features from the image to be detected of the i-th appliance among the n appliances, the image to be detected of the i-th appliance, the image to be detected of the i-1th appliance, and the image to be detected of the i+1th appliance, to obtain features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance, fuse the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain fused features, and perform classification based on the fused features to obtain a classification result of whether the i-th appliance is deformed.

9. The method according to claim 8, characterized in that Also includes: A training set is obtained, wherein the training set includes sample images of a plurality of samples, each sample including: m-2 groups of subsamples from m subsamples corresponding to m correction steps, where m is an integer greater than 2; wherein each group of subsamples includes a k-th subsample, a k-1-th subsample, a k-th subsample, and a k+1-th subsample, and the value of k is any integer in [2, m-1] in sequence; in each group of subsamples, when a subsample in each group of subsamples is deformed, the k-th subsample is annotated as a positive sample with annotation information indicating the presence of deformation; Inputting the k-th subsample, the k-1-th subsample, the k-th subsample and the k+1-th subsample included in each group of subsamples in the training set into the convolutional neural network to be trained respectively, and the convolutional neural network to be trained outputs a prediction classification result of whether the k-th subsample is deformed; According to the labeling information and predicted classification results corresponding to each group of sub-samples, the convolutional neural network to be trained is trained until a preset training completion condition is met to obtain the convolutional neural network.

10. A deformation detection device for a multi-step orthodontic appliance, characterized in that: The multi-step corrector comprises n correctors corresponding to n corrective steps, and the device comprises: A first acquisition module is used to respectively acquire the images to be detected of the n orthodontic appliances; A feature extraction module, for extracting features of the image to be detected of the i-th appliance among the n appliances, the image to be detected of the i-th appliance, the image to be detected of the i-1-th appliance, and the image to be detected of the i+1-th appliance, respectively, to obtain features of the image to be detected of the i-1-th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1-th appliance; wherein the value of i is any integer in [2, n-1] in sequence, and the value of n is greater than 2; a feature fusion module, used for fusing the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain a fused feature; A classification module, used for performing classification based on the fusion features to obtain a classification result of whether the i-th appliance is deformed; The first determination module is used to determine whether the i-th orthodontic appliance is deformed based on the classification result of whether the i-th orthodontic appliance is deformed.

11. The device according to claim 10, characterized in that The first determining module is further used for: The determination result of whether the second appliance is deformed is used as the determination result of whether the first appliance is deformed; The determination result of whether the n-1th orthodontic appliance is deformed is used as the determination result of whether the nth orthodontic appliance is deformed.

12. The device according to claim 11, characterized in that The deformation includes any one or more of the following: expansion of the arch, contraction of the arch.

13. The device according to claim 11, characterized in that Also includes: The second determination module is used to determine the deformation detection result of the multi-step orthodontic appliance based on the determination result of whether the i-th orthodontic appliance is deformed.

14. The device according to claim 10, characterized in that The first acquisition module is specifically used to: The image acquisition system is used to acquire images of the n orthodontic appliances respectively in a top-view image acquisition manner to obtain the images to be detected of the n orthodontic appliances.

15. The device according to claim 10, characterized in that The feature fusion module is specifically used for: Based on the attention mechanism, the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance are fused to obtain a fused feature.

16. The device according to claim 15, characterized in that The feature fusion module includes: a feature fusion unit, used to fuse the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain an intermediate fusion feature; A spatial attention module, used to obtain spatial attention weight values ​​corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance; A weighting module is used to weight the intermediate fusion feature based on the spatial attention weight values ​​corresponding to the features of the image to be detected of the i-1th appliance, the features of the image to be detected of the i-th appliance, and the features of the image to be detected of the i+1th appliance to obtain the fusion feature.

17. The device according to any one of claims 10 to 16, characterized in that: The feature extraction module, the feature fusion module and the classification module are implemented by a pre-trained convolutional neural network.

18. The device according to claim 17, characterized in that Also includes: A second acquisition module is used to acquire a training set, wherein the training set includes sample images of a plurality of samples, each sample including: m-2 groups of subsamples from m subsamples corresponding to m correction steps, where m is an integer greater than 2; wherein each group of subsamples includes a k-th subsample, a k-1-th subsample, a k-th subsample, and a k+1-th subsample, and the value of k is any integer in [2, m-1] in sequence; in each group of subsamples, when a subsample in each group of subsamples is deformed, the k-th subsample is annotated as a positive sample with annotation information indicating the presence of deformation; The training module is used to input the k-th subsample included in each group of subsamples in the training set into the k-1-th subsample, the k-th subsample and the k+1-th subsample into the convolutional neural network to be trained, and the convolutional neural network to be trained outputs the predicted classification result of whether the k-th subsample is deformed; and according to the labeling information and the predicted classification result corresponding to each group of subsamples, the convolutional neural network to be trained is trained until the preset training completion condition is met to obtain the convolutional neural network.

19. An electronic device, characterized in that: The device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images