Model training methods, devices, terminals, and storage media based on transfer learning
By training and generating a target AI model through transfer learning, the problem of poor image processing performance of general deep learning models in special oral scanning scenarios is solved, and accurate identification and processing of special oral scanning scenarios are achieved.
Patent Information
- Application Number
- CN202510637400.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-16
- Publication Date
- 2026-11-17
AI Technical Summary
General-purpose deep learning models struggle to meet image processing requirements in specific oral cavity scanning scenarios, often misidentifying areas that should be preserved as noisy areas, resulting in poor image processing performance.
Using transfer learning, user terminals collect and label oral cavity scan sample images, send them to the model service platform for training, and generate target AI models to adapt to specific oral cavity scan scenarios.
It improves the image processing performance of the model in special oral scanning scenarios, enabling accurate identification and preservation of important areas and avoiding accidental deletion.
Smart Images

Figure CN123115449A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to model training methods, apparatus, terminals and storage media based on transfer learning. Background Technology
[0002] In the field of dentistry, 3D reconstruction technology plays an important role. In some scenarios, the images of the acquired 3D objects contain noise. To remove noise, in the intraoral 3D scanning process, existing technologies can identify the tooth and gingival regions in a single frame of data using a general-purpose deep learning model (hereinafter referred to as a general deep learning model) and remove non-interest areas such as the buccal and lingual sides, oral mirrors, and medical gloves.
[0003] While general-purpose deep learning models perform well in routine oral scanning scenarios, they often fall short of meeting all requirements when faced with diverse diagnostic and treatment scenarios. For example, when using general-purpose deep learning models to process images in special oral scanning scenarios (such as edentulous jaws, immediate implantation, and dental plaque staining), they often misidentify some areas that should be preserved as noise areas, resulting in poor image processing performance in special oral scanning scenarios. Summary of the Invention
[0004] This application provides a model training method, apparatus, terminal, and storage medium based on transfer learning, which can improve the image processing performance of the model in special oral scanning scenarios.
[0005] In a first aspect, embodiments of this application provide a model training method based on transfer learning, the method being applied to a user terminal, the method comprising:
[0006] Send a model search command for the target oral cavity scanning scenario to the model service platform, and receive the model search results returned by the model service platform based on the model search command;
[0007] When the model search result indicates that there is no target AI model corresponding to the target oral cavity scanning scene in the AI model library of the model service platform, multiple frames of oral cavity scanning sample images under the target oral cavity scanning scene are obtained;
[0008] Multiple frames of the oral cavity scan sample images are sent to the model service platform, so that the model service platform can train a pre-trained general deep learning model based on the multiple frames of the oral cavity scan sample images to obtain the target AI model;
[0009] Receive the target AI model sent by the model service platform.
[0010] Secondly, embodiments of this application provide another model training method based on transfer learning, the method being applied to a model service platform, the method comprising:
[0011] Receive model search instructions sent by the user terminal for the target oral cavity scanning scenario;
[0012] Determine whether a target AI model corresponding to the target oral cavity scanning scenario exists in the preset AI model library, generate and send the model search results to the user terminal;
[0013] When the model search result indicates that the target AI model does not exist in the AI model library of the model service platform, the system receives multiple frames of oral cavity scan sample images sent by the user terminal.
[0014] The target AI model is obtained by training a pre-trained general deep learning model based on multiple frames of oral cavity scan sample images.
[0015] The target AI model is sent to the user terminal.
[0016] Thirdly, embodiments of this application also provide a first model training apparatus based on transfer learning, wherein the model training apparatus based on transfer learning is deployed on a user terminal, and the model training apparatus based on transfer learning includes a transceiver unit and a processing unit, wherein the first processing unit is used to control the transceiver operations of the first transceiver unit, wherein:
[0017] The transceiver unit is configured to send a model search instruction for a target oral cavity scanning scene to the model service platform, and receive the model search results returned by the model service platform based on the model search instruction; when the model search results indicate that there is no target AI model corresponding to the target oral cavity scanning scene in the AI model library of the model service platform, the unit acquires multiple frames of oral cavity scanning sample images under the target oral cavity scanning scene; sends the multiple frames of oral cavity scanning sample images to the model service platform, so that the model service platform trains a pre-trained general deep learning model based on the multiple frames of oral cavity scanning sample images to obtain the target AI model; and receives the target AI model sent by the model service platform.
[0018] Fourthly, embodiments of this application also provide a second model training apparatus based on transfer learning. This second model training apparatus is deployed on a model service platform and includes a second transceiver unit and a second processing unit, wherein:
[0019] The second transceiver unit is used to receive model search instructions for the target oral cavity scanning scenario sent by the user terminal;
[0020] The second processing unit is used to determine whether there is a target AI model corresponding to the target oral cavity scanning scenario in the preset AI model library, and to generate and send the model search results to the user terminal through the second transceiver unit;
[0021] The second transceiver unit is further configured to receive multiple frames of oral cavity scan sample images sent by the user terminal when the model search result indicates that the target AI model does not exist in the AI model library of the model service platform;
[0022] The processing unit is further configured to train a pre-trained general deep learning model based on multiple frames of oral cavity scan sample images to obtain the target AI model.
[0023] The second transceiver unit is also used to send the target AI model to the user terminal.
[0024] Fifthly, embodiments of this application also provide a user terminal, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method corresponding to the first aspect described above.
[0025] Sixthly, embodiments of this application also provide a model service platform, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method corresponding to the second aspect above.
[0026] In a seventh aspect, embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, can implement the methods corresponding to the first or second aspect described above.
[0027] This application provides a model training method, apparatus, terminal, and storage medium based on transfer learning. The method includes: a user terminal sending a model search instruction for a target oral cavity scanning scene to a model service platform, and receiving model search results returned by the model service platform based on the model search instruction; when the model search results indicate that no target AI model corresponding to the target oral cavity scanning scene exists in the AI model library of the model service platform, acquiring multiple frames of oral cavity scanning sample images under the target oral cavity scanning scene; sending the multiple frames of oral cavity scanning sample images to the model service platform, so that the model service platform trains a pre-trained general deep learning model based on the multiple frames of oral cavity scanning sample images to obtain the target AI model; and receiving the target AI model sent by the model service platform. In this embodiment, when the user does not find the corresponding AI model for the target oral cavity scanning scene in the AI model library, the user can collect and annotate oral cavity scanning sample images for the target oral cavity scanning scene in real time through the user terminal, and then send the oral cavity scanning sample images to the model service platform through transfer learning. The model service platform can then further train the general deep learning model based on the oral cavity scanning sample images to obtain the target AI model corresponding to the target oral cavity scanning scene. The obtained target AI model can accurately identify the images scanned in the target oral cavity scanning scene, thereby improving the image processing effect of the model in the special oral cavity scanning scene (target oral cavity scanning scene). Attached Figure Description
[0028] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0029] Figure 1 A schematic flowchart illustrating the model training method based on transfer learning provided in this application embodiment;
[0030] Figure 2 A schematic diagram of a sub-process of the model training method based on transfer learning provided in the embodiments of this application;
[0031] Figure 3 Another sub-process diagram of the model training method based on transfer learning provided in the embodiments of this application;
[0032] Figure 4 Another sub-process diagram of the model training method based on transfer learning provided in the embodiments of this application;
[0033] Figure 5A schematic diagram of an image backprojection scenario in the model training method based on transfer learning provided in the embodiments of this application;
[0034] Figure 6 Another sub-process diagram of the model training method based on transfer learning provided in the embodiments of this application;
[0035] Figure 7 A schematic flowchart of a model training method based on transfer learning provided in another embodiment of this application;
[0036] Figure 8 A schematic block diagram of a first model training apparatus based on transfer learning provided in an embodiment of this application;
[0037] Figure 9 A schematic block diagram of a second model training apparatus based on transfer learning provided in an embodiment of this application;
[0038] Figure 10 A schematic block diagram of a user terminal provided in an embodiment of this application;
[0039] Figure 11 A schematic block diagram of the model service platform provided in the embodiments of this application. Detailed Implementation
[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0041] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0042] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0043] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0044] This application provides a model training method, apparatus, terminal, and storage medium based on transfer learning.
[0045] The execution entity of the transfer learning-based model training method can be the transfer learning-based model training device provided in the embodiments of this application, or a user terminal or model service platform that integrates the transfer learning-based model training device. The transfer learning-based model training device can be implemented in hardware or software.
[0046] In this embodiment, before scanning the oral cavity, the user first searches the AI model library of the model service platform to see if a target AI model corresponding to the current target oral cavity scanning scene exists. If it exists, the target AI model is directly obtained from the AI model library, and the oral cavity in the target oral cavity scanning scene is scanned and reconstructed in 3D based on the target AI model. If it does not exist, multiple frames of oral cavity scanning images are acquired in the target oral cavity scanning scene, and the acquired multiple frames of oral cavity scanning images are labeled to obtain multiple frames of oral cavity scanning sample images. Based on transfer learning, the multiple frames of oral cavity scanning sample images are sent to the model service platform, so that the model service platform can train a pre-trained general deep learning model based on the multiple frames of oral cavity scanning sample images to obtain a target AI model that can be used to process scan images in the target oral cavity scanning scene. It can be seen that this embodiment can realize customized scene model training through the model service platform. The customized scene AI model obtained by training can improve the image processing effect of the model in special oral cavity scanning scenes.
[0047] In this embodiment, the target oral scanning scenarios include edentulous jaw muscle static line scanning scenario, immediate scanning scenario, and dental plaque stained oral scanning scenario.
[0048] Specifically, 1. In the scenario of scanning the static lines of the edentulous jaw muscles, due to the absence of the rigid body (teeth) region, some buccal data needs to be incorporated for stitching. However, these auxiliary regions are deleted in general deep learning models. The target AI model trained through this embodiment can not only remove noise (such as unnecessary buccal and lingual areas, oral mirrors, medical gloves, etc.) from the scanned images when processing them, but also retain these auxiliary scenes; 2. In the scenario of immediate scanning, the gums of patients who have just undergone surgery will appear swollen and white or have bloodstains. However, these swollen and white or bloodstain-stained gum areas are deleted in general deep learning models, leading to the accidental deletion of gum areas. The target AI model trained through this embodiment can retain images of swollen and white or bloodstain-stained gum areas when processing scanned images, avoiding the accidental deletion of gum areas; 3. In the scenario of dental plaque staining oral scanning, in order to better identify dental plaque on the teeth, a staining agent needs to be applied to the patient's teeth to make them appear red. However, these stained areas are mistakenly deleted in general deep learning models, but the target AI model trained through this embodiment can accurately identify and preserve the stained areas of the teeth when processing scanned images.
[0049] The following section provides a detailed explanation of the transfer learning-based model training method provided in this application, using the user terminal as the execution subject. Figure 1 This is a flowchart illustrating the model training method based on transfer learning provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps S110-S140.
[0050] S110. Send a model search instruction for the target oral cavity scanning scenario to the model service platform, and receive the model search results returned by the model service platform based on the model search instruction.
[0051] Specifically, the model search instruction carries scene keywords or target scene images corresponding to the target oral cavity scanning scene. The model service platform searches for AI models suitable for the target oral cavity scanning scene in the AI model library using scene keywords or target scene images.
[0052] For example, a user (doctor or technician) logs into the model service platform using a user terminal, enters scene keywords or uploads a target scene image on the model search interface provided by the model service platform, and clicks the "upload" control to send a model search command to the model service platform.
[0053] The model service platform's AI model library stores multiple AI models corresponding to different oral scanning scenarios. Each AI model carries different descriptive information, including model keywords and / or model scene images. When a model search command includes target scene keywords, the model service platform performs keyword matching between these keywords and the model keywords of each AI model in the library. If a matching keyword is found, the AI model corresponding to that keyword is identified as the target AI model. If no matching keyword is found, it is determined that no target AI model corresponding to the target oral scanning scenario exists in the AI model library. When a model search command includes a target scene image, the model service platform calculates image similarity between the target scene image and the model scene images of each AI model in the AI model library. If a model scene image with a similarity greater than a preset similarity threshold exists, the AI model corresponding to that scene image is identified as the target AI model. If no model scene image with a similarity greater than the preset similarity threshold exists, it is determined that no target AI model corresponding to the target oral scanning scenario exists in the AI model library.
[0054] S120. When the model search result indicates that there is no target AI model corresponding to the target oral cavity scanning scene in the AI model library of the model service platform, acquire multiple frames of oral cavity scanning sample images under the target oral cavity scanning scene.
[0055] In this embodiment, when no target AI model corresponding to the target oral cavity scanning scene is found in the AI model library, the user can use transfer learning to train an AI model corresponding to the target oral cavity scanning scene through the model service platform. First, it is necessary to obtain multiple frames of oral cavity scanning sample images under the target oral cavity scanning scene.
[0056] In some embodiments, such as Figure 2 As shown, oral cavity scan sample images can be obtained through the following steps:
[0057] S121. Acquire multiple frames of target oral cavity scanning images in the target oral cavity scanning scene;
[0058] S122. Based on preset image annotation rules, the target oral cavity scan images of each frame are annotated to obtain multiple frames of oral cavity scan sample images containing annotated regions.
[0059] Specifically, multiple frames of target oral cavity scanning images are acquired using a scanner (such as an intraoral scanner or a facial scanner) in the target oral cavity scanning scene.
[0060] In this process, the candidate oral cavity scan images are directly obtained through the scanner. These candidate images contain duplicates or images with excessively high similarity. These highly similar images are not helpful for training. To improve training speed, this embodiment requires filtering the obtained candidate oral cavity scan images. Specifically, as follows: Figure 3 As shown, acquiring multiple frames of target oral cavity scan images in the target oral cavity scan scene includes:
[0061] S1211. Obtain multiple candidate oral cavity scanning images in the target oral cavity scanning scene;
[0062] S1212. Obtain the scanning pose corresponding to each frame of the candidate oral cavity scanning image, and / or extract the contour features corresponding to each frame of the candidate oral cavity scanning image through a preset feature extraction model.
[0063] S1213. Determine the pose similarity between any two adjacent candidate oral cavity scan images based on the scan pose, and / or determine the feature similarity between any two adjacent candidate oral cavity scan images based on the contour features.
[0064] S1214. Based on the pose similarity and / or the feature similarity, filter the candidate oral cavity scan images of multiple frames to obtain the target oral cavity scan images of multiple frames.
[0065] Specifically, if the pose similarity between two adjacent candidate oral cavity scan images is greater than a preset pose similarity (e.g., 97% similarity), or the feature similarity between two adjacent candidate oral cavity scan images is greater than a preset feature similarity (e.g., 95% similarity), then only one of the two images is retained. Specifically, the clarity of the two images can be obtained, and the one with higher clarity is selected. The candidate oral cavity scan image obtained after filtering is the target oral cavity scan image.
[0066] When the acquired candidate oral cavity scan images include facial scan data, in order to protect customer privacy, the eyes in the facial scan data must first be desensitized.
[0067] In this embodiment, two data annotation methods are provided, as follows:
[0068] The first annotation method involves directly annotating the two-dimensional data. Please refer to [link / reference] for details. Figure 4 Step S122 includes the following steps:
[0069] a1221. Based on a preset pre-segmentation model, each frame of the target oral cavity scan image is pre-segmented to obtain multiple frames of pre-segmented oral cavity scan images;
[0070] a1222: Receive the annotation operation from the user for each frame of the pre-segmented oral cavity scan image to determine the annotation region corresponding to each frame of the pre-segmented oral cavity scan image, and obtain multiple frames of oral cavity scan sample images.
[0071] In this embodiment, the pre-segmentation model can be a large SAM-Med2D model. The SAM-Med2D model can segment the mask regions of all objects in the target oral cavity scan image. Users can directly annotate the identified mask regions, for example, selecting all annotated regions to be retained during data processing. These annotated regions include the gingival region and the tooth region. This embodiment allows users to easily and efficiently edit and annotate oral cavity image data.
[0072] The second annotation method, the 3D mesh annotation method, involves first annotating a 3D mesh, and then backprojecting the result onto a 2D image. Backprojection is the process of mapping the information of the 3D mesh patches back to a 2D image.
[0073] Please refer to details. Figure 5 as well as Figure 6 Step S122 includes the following steps:
[0074] b1221. Generate an original three-dimensional model of the oral cavity based on multiple frames of the target oral cavity scan images;
[0075] The original oral cavity 3D model was generated from an oral cavity scan image that had not undergone intelligent processing (such as removal of the buccal and lingual sides, oral mirror, and medical gloves).
[0076] b1222. Receive annotation operations from the user on the original oral cavity 3D model to determine the annotation area of the original oral cavity 3D model;
[0077] b1223. Perform a back projection operation on the labeled area of the original oral cavity 3D model to project the labeled area onto each of the target oral cavity scan images to obtain multiple frames of the oral cavity scan sample images.
[0078] Specifically, step b1223 includes:
[0079] The target 3D coordinates of each pixel in the labeled area in the world coordinate system are determined. Based on a preset coordinate transformation algorithm, the target 2D coordinates of each target 3D coordinate on each target oral scan image are determined according to the projection matrix corresponding to each target oral scan image. The coordinate transformation algorithm is: x = PX; where x is the target 2D coordinate, X is the target 3D coordinate, P is the projection matrix of the corresponding camera when scanning the target oral scan image, P = K[R|t], K is the intrinsic parameter matrix of the camera, R is the rotation matrix, and t is the translation vector. For each target oral scan image, the oral scan sample image is determined according to the target 2D coordinates in the target oral scan image.
[0080] Information entropy is used to measure the uncertainty of information content. The higher the information entropy, the higher the uncertainty in the deep learning model's recognition of the frame of data.
[0081] In some embodiments, since high information entropy indicates high uncertainty in model recognition, such frame data should be added to the training set to improve the model's recognition ability. In this case, to ensure that less data covers more scenarios and improve model training speed, after annotating each frame of the target oral cavity scan image based on preset image annotation rules to obtain multiple frames of oral cavity scan sample images containing annotated regions, the method further includes:
[0082] The oral cavity scan sample images are manually screened to obtain multiple first scan sample images; the general deep learning model is obtained from the model service platform; the first information entropy corresponding to each oral cavity scan sample image is determined by the general deep learning model; the first scan sample images in the multiple first scan sample images whose first information entropy is higher than a first preset information entropy threshold are filtered to obtain multiple second scan sample images.
[0083] At this time, sending multiple frames of the oral cavity scan sample images to the model service platform includes:
[0084] Multiple frames of the second scanned sample images are sent to the model service platform. At this time, the model service platform trains a general deep learning model based on the second scanned sample images to obtain the target AI model.
[0085] As can be seen, this embodiment, after manual screening, can further utilize information entropy to filter samples while ensuring the quality of sample annotation, thereby ensuring that less data covers more scenarios and improving the model training speed.
[0086] S130. Send multiple frames of the oral cavity scan sample images to the model service platform, so that the model service platform can train the pre-trained general deep learning model based on the multiple frames of the oral cavity scan sample images to obtain the target AI model.
[0087] In some embodiments, the user terminal can directly send multiple frames of oral scan sample images that have been labeled and have not been manually screened to the model service platform. In this case, the model service platform adds all the received multiple frames of oral scan sample images to the initial training of the general deep learning model. After obtaining the initially trained deep learning model, the platform obtains the second information entropy of each frame of oral scan sample image through the initially trained deep learning model. Since the oral scan sample images have not been manually screened and confirmed, there may be labeling quality issues. In this case, the model service platform will filter out oral scan sample images whose second information entropy is lower than the second preset information entropy threshold to ensure the quality of the samples. Then, the filtered oral scan sample images are used to perform deep training on the general deep learning model to obtain the target AI model.
[0088] The model service platform will also adjust the structure and parameters of general deep learning models based on user instructions, such as increasing the number of hidden layers, adjusting neuron configuration, and optimizing hyperparameters like algorithms and learning rates, to adapt to diverse scenarios in dentistry. For example, in the edentulous jaw muscle static line scenario, facing the problem of unclear textures, by adjusting the model structure and parameters, the model will utilize auxiliary data such as depth information to improve recognition accuracy; while in the dental plaque scenario, the model will specifically enhance its ability to recognize stained areas. Through transfer learning, fine-tuning neural networks, and data augmentation techniques, the model can quickly adapt to specific tasks, ensuring high accuracy and efficiency in practical applications and meeting the unique needs of different scenarios in the field of dentistry.
[0089] Furthermore, after training the target AI model in this embodiment, in order to meet the real-time requirements of the AI algorithm, the target AI model also needs to be compressed. Specifically, the model is compressed according to the weight ratio of each channel in the target AI model parameters. It can be seen that this embodiment, through customized model training, provides more customized and lightweight models for different scenarios, thereby improving the efficiency and real-time response capability of the algorithm.
[0090] Furthermore, in this embodiment, multiple frames of the oral cavity scan sample images are sent to the model service platform through a data anonymization upload mechanism to protect patient privacy.
[0091] S140. Receive the target AI model sent by the model service platform.
[0092] In this embodiment, after the model service platform trains the target AI model, it pushes the target AI model to the user terminal. The user then evaluates the target AI model to determine whether it meets the requirements. Specifically, based on the target AI model, oral scan test images in the target oral scan scenario are obtained; the user's evaluation results for the oral scan test images are received; if the evaluation result is that the test fails, the process returns to the step of obtaining multiple frames of target oral scan images in the target oral scan scenario; if the evaluation result is that the test passes, the target AI model is deployed.
[0093] For example, after obtaining the target AI model, the user uses the target AI model to perform image denoising on the original scan image obtained in the target oral cavity scan scenario to obtain an oral cavity scan test image. Then, the user observes the denoising effect of the oral cavity scan test image and inputs the evaluation result. If the evaluation result is that the test fails, the model needs to be trained further. Otherwise, the target AI model is deployed locally, and the target AI model is used to perform image processing in the target oral cavity scan scenario. The processed image is then used to perform 3D reconstruction of the oral cavity model.
[0094] Furthermore, if the evaluation result indicates that the test passed, the method further includes:
[0095] The evaluation results are sent to the model service platform, so that the model service platform stores the target AI model in the AI model library based on the evaluation results; when an authorization operation for the target AI model is received from a user, an authorization instruction for the target AI model is sent to the model service platform, so that the model service platform grants the user indicated by the authorization instruction access to the target AI model in the AI model library based on the authorization instruction.
[0096] In this embodiment, after the user evaluates the model, models that meet the requirements can be stored in the AI model library and can be made available to other users (either specified users or all users) through authorization; for models that do not meet the requirements, more high-quality labeled data can be uploaded for retraining.
[0097] Once the user terminal obtains the target AI model, it can process the oral data in the target oral scanning scenario based on the target AI model. Specifically, it acquires multiple frames of the current oral scanning images in the target oral scanning scenario; it performs three-dimensional reconstruction processing on the multiple frames of the current oral scanning images based on the target AI model to obtain the target three-dimensional model, and the target three-dimensional model retains the scanning data corresponding to the labeled area.
[0098] The aforementioned authorization process also includes verifying model accuracy, confirming model application scenarios, and confirming information related to future commercialization. After authorization is granted to other doctors and technicians (users specified in the authorization instructions), users can independently split and iterate their own AI models to meet specific professional needs and treatment scenarios. Furthermore, multiple users can pool resources and data to jointly train a single AI model to improve its generalization ability and accuracy. However, a clear decision-making framework must be established subsequently to jointly evaluate model performance and application effects, guide subsequent optimization and iteration, and ensure that the AI model can continuously meet clinical needs and maintain technological leadership.
[0099] In summary, in this embodiment, when a user does not find an AI model corresponding to the target oral cavity scanning scene in the AI model library, the user can collect and annotate oral cavity scanning sample images of the target oral cavity scanning scene in real time through the user terminal. Then, the oral cavity scanning sample images are sent to the model service platform through transfer learning, so that the model service platform can further train the general deep learning model based on the oral cavity scanning sample images to obtain the target AI model corresponding to the target oral cavity scanning scene. The obtained target AI model can accurately identify the images scanned in the target oral cavity scanning scene, thereby improving the image processing effect of the model in the special oral cavity scanning scene (target oral cavity scanning scene).
[0100] The following section provides a detailed explanation of the transfer learning-based model training method provided in this application, using a model service platform as the execution entity. Figure 7 This is a flowchart illustrating the model training method based on transfer learning provided in an embodiment of this application. Figure 7 As shown, the method includes the following steps S210-S250.
[0101] S210: Receive a model search command sent by the user terminal for the target oral cavity scanning scenario.
[0102] Specifically, the model search instruction carries scene keywords corresponding to the target oral cavity scanning scene or a target scene image corresponding to the target oral cavity scanning scene.
[0103] For example, a user (doctor or technician) logs into the model service platform using a user terminal, enters scene keywords or uploads a target scene image on the model search interface provided by the model service platform, and clicks the "upload" control to send a model search command to the model service platform.
[0104] S220. Determine whether there is a target AI model in the preset AI model library that corresponds to the target oral cavity scanning scenario, generate and send the model search results to the user terminal.
[0105] The model service platform searches for AI models suitable for the target oral cavity scanning scenario in the AI model library by using scene keywords or target scene images.
[0106] In some embodiments, the AI model library stores multiple AI models, each carrying different descriptive information, including model keywords and model scene images; determining whether a target AI model corresponding to the target oral cavity scan scene exists in the preset AI model library, generating and sending model search results to the user terminal includes:
[0107] When the model search instruction carries target scene keywords, the target scene keywords are matched with the model keywords of each AI model to obtain matching results; when the model search instruction carries target scene images, the target scene images are compared with the model scene images of each AI model to obtain similarity calculation results; the model search results are determined based on the matching results or the similarity calculation results; and the model search results are sent to the user terminal.
[0108] Specifically, the model service platform's AI model library stores multiple AI models corresponding to different oral scanning scenarios. Different AI models carry different descriptive information, including model keywords and / or model scene images. When the model search command carries target scene keywords, the model service platform performs keyword matching between these scene keywords and the model keywords of each AI model in the AI model library. If a matching keyword is found, the AI model corresponding to that keyword is identified as the target AI model. If no matching keyword is found, it is determined that there is no target AI model in the AI model library corresponding to the target oral scanning scenario. When the model search command carries a target scene image, the model service platform performs image similarity calculations between the target scene image and the model scene images of each AI model in the AI model library. If a model scene image with a similarity greater than a preset similarity threshold exists, the AI model corresponding to that model scene image is identified as the target AI model. If no model scene image with a similarity greater than the preset similarity threshold exists, it is determined that there is no target AI model in the AI model library corresponding to the target oral scanning scenario.
[0109] S230. When the model search result indicates that the target AI model does not exist in the AI model library of the model service platform, receive multiple frames of oral cavity scan sample images sent by the user terminal.
[0110] In this embodiment, when the model search results show that the target AI model does not exist in the AI model library of the model service platform, the user can train a target AI model for the target oral scan scenario pair through the model service platform. If the target AI model exists, the retrieved target AI model is directly returned to the user terminal.
[0111] After receiving the multiple frames of oral cavity scan sample images sent by the user terminal, the method further includes: if the multiple frames of oral cavity scan sample images carry a confidentiality identifier, creating a dedicated sample storage space corresponding to the target AI model, and storing the multiple frames of oral cavity scan sample images in the dedicated sample storage space to ensure the accuracy of the model and the confidentiality of the samples; if the multiple frames of oral cavity scan sample images do not carry a confidentiality identifier, storing the multiple frames of oral cavity scan sample images in the public sample storage space of the AI model library to save storage space.
[0112] S240. The pre-trained general deep learning model is trained based on multiple frames of oral cavity scan sample images to obtain the target AI model.
[0113] In this embodiment, training the model using a pre-trained general deep learning model can improve training speed and accuracy.
[0114] In some embodiments, the user terminal can directly send multiple frames of oral scan sample images that have been labeled and have not been manually screened to the model service platform. In this case, the model service platform adds all the received multiple frames of oral scan sample images to the initial training of the general deep learning model. After obtaining the initially trained deep learning model, the platform obtains the second information entropy of each frame of oral scan sample image through the initially trained deep learning model. Since the oral scan sample images have not been manually screened and confirmed, there may be labeling quality issues. In this case, the model service platform will filter out oral scan sample images whose second information entropy is lower than the second preset information entropy threshold to ensure the quality of the samples. Then, the filtered oral scan sample images are used to perform deep training on the general deep learning model to obtain the target AI model.
[0115] The model service platform will also adjust the structure and parameters of general deep learning models based on user instructions, such as increasing the number of hidden layers, adjusting neuron configuration, and optimizing hyperparameters like algorithms and learning rates, to adapt to diverse scenarios in dentistry. For example, in the edentulous jaw muscle static line scenario, facing the problem of unclear textures, by adjusting the model structure and parameters, the model will utilize auxiliary data such as depth information to improve recognition accuracy; while in the dental plaque scenario, the model will specifically enhance its ability to recognize stained areas. Through transfer learning, fine-tuning neural networks, and data augmentation techniques, the model can quickly adapt to specific tasks, ensuring high accuracy and efficiency in practical applications and meeting the unique needs of different scenarios in the field of dentistry.
[0116] Furthermore, after training the target AI model in this embodiment, in order to meet the real-time requirements of the AI algorithm, the target AI model also needs to be compressed. Specifically, the model is compressed according to the weight ratio of each channel in the target AI model parameters. It can be seen that this embodiment, through customized model training, provides more customized and lightweight models for different scenarios, thereby improving the efficiency and real-time response capability of the algorithm.
[0117] S250, Send the target AI model to the user terminal.
[0118] In this embodiment, once the target AI model is trained, the target AI model is returned to the user terminal.
[0119] Furthermore, after sending the target AI model to the user terminal, the method further includes: receiving an evaluation result sent by the user terminal indicating that the target AI model has passed the test; and in response to the evaluation result, storing the target AI model in the AI model library.
[0120] Furthermore, after storing the target AI model in the AI model library, the method further includes: receiving an authorization instruction for the target AI model sent by the user terminal; and granting access to the target AI model to the user indicated by the authorization instruction.
[0121] In this embodiment, after the user evaluates the model, the model service platform stores the model that meets the requirements in the AI model library, and can grant access to other users (either specified users or all users) for use; for the model that does not meet the requirements, more high-quality labeled data can be uploaded for retraining.
[0122] Figure 8 This is a schematic block diagram of a first model training device 800 based on transfer learning provided in an embodiment of this application. Figure 8As shown, corresponding to the above-described transfer learning-based model training method, this application also provides a first transfer learning-based model training apparatus 800. This first transfer learning-based model training apparatus 800 includes units for executing the above-described transfer learning-based model training method. The first transfer learning-based model training apparatus is deployed on a user terminal. The first transfer learning-based model training apparatus includes a first transceiver unit 801 and a first processing unit 802. The first processing unit 802 is used to control the transceiver operations of the first transceiver unit 801.
[0123] The first transceiver unit 801 is configured to send a model search instruction for a target oral cavity scanning scene to the model service platform, and receive the model search results returned by the model service platform based on the model search instruction; when the model search results indicate that there is no target AI model corresponding to the target oral cavity scanning scene in the AI model library of the model service platform, it acquires multiple frames of oral cavity scanning sample images under the target oral cavity scanning scene; sends the multiple frames of oral cavity scanning sample images to the model service platform, so that the model service platform trains a pre-trained general deep learning model based on the multiple frames of oral cavity scanning sample images to obtain the target AI model; and receives the target AI model sent by the model service platform.
[0124] In some embodiments, when the first processing unit 802 performs the step of acquiring multiple frames of oral scan sample images in the target oral scan scene, it is specifically used for:
[0125] Acquire multiple frames of target oral cavity scanning images in the target oral cavity scanning scene; perform annotation processing on each frame of the target oral cavity scanning images based on preset image annotation rules to obtain multiple frames of oral cavity scanning sample images containing the annotated regions.
[0126] In some embodiments, when the first processing unit 802 performs the step of annotating each frame of the target oral cavity scan image based on a preset image annotation rule to obtain multiple frames of oral cavity scan sample images containing annotated regions, it is specifically used for:
[0127] Based on a preset pre-segmentation model, each frame of the target oral cavity scan image is pre-segmented to obtain multiple frames of pre-segmented oral cavity scan images; the user's annotation operation for each frame of the pre-segmented oral cavity scan image is received to determine the annotation region corresponding to each frame of the pre-segmented oral cavity scan image, thereby obtaining multiple frames of oral cavity scan sample images.
[0128] In some embodiments, when the first processing unit 802 performs the step of annotating each frame of the target oral cavity scan image based on a preset image annotation rule to obtain multiple frames of oral cavity scan sample images containing annotated regions, it is specifically used for:
[0129] An original 3D oral cavity model is generated based on multiple frames of the target oral cavity scan images; the user's annotation operation on the original 3D oral cavity model is received to determine the annotation area of the original 3D oral cavity model; the annotation area of the original 3D oral cavity model is back-projected onto each of the target oral cavity scan images to obtain multiple frames of oral cavity scan sample images.
[0130] In some embodiments, when the first processing unit 802 performs the step of back-projecting the annotated region of the original oral cavity three-dimensional model to project the annotated region onto each of the target oral cavity scan images to obtain multiple frames of the oral cavity scan sample images, it is specifically used for:
[0131] The target 3D coordinates of each pixel in the labeled area in the world coordinate system are determined. Based on a preset coordinate transformation algorithm, the target 2D coordinates of each target 3D coordinate on each target oral scan image are determined according to the projection matrix corresponding to each target oral scan image. The coordinate transformation algorithm is: x = PX; where x is the target 2D coordinate, X is the target 3D coordinate, P is the projection matrix of the corresponding camera when scanning the target oral scan image, P = K[R|t], K is the intrinsic parameter matrix of the camera, R is the rotation matrix, and t is the translation vector. For each target oral scan image, the oral scan sample image is determined according to the target 2D coordinates in the target oral scan image.
[0132] In some embodiments, when the first transceiver unit 801 performs the step of acquiring multiple frames of target oral cavity scan images in the target oral cavity scan scene, it is specifically used for:
[0133] The process involves: acquiring multiple candidate oral cavity scan images in the target oral cavity scanning scene; acquiring the scanning pose corresponding to each candidate oral cavity scan image and / or extracting the contour features corresponding to each candidate oral cavity scan image using a preset feature extraction model; using the first processing unit 802 to determine the pose similarity between any two adjacent candidate oral cavity scan images based on the scanning pose and / or to determine the feature similarity between any two adjacent candidate oral cavity scan images based on the contour features; and filtering the multiple candidate oral cavity scan images based on the pose similarity and / or the feature similarity to obtain multiple target oral cavity scan images.
[0134] In some embodiments, after performing the step of annotating each frame of the target oral cavity scan image based on a preset image annotation rule to obtain multiple frames of oral cavity scan sample images containing the annotated regions, the first processing unit 802 is further configured to:
[0135] The oral cavity scan sample images are manually screened to obtain multiple first scan sample images; the general deep learning model is obtained from the model service platform; the first information entropy corresponding to each oral cavity scan sample image is determined by the general deep learning model; the first scan sample images in the multiple first scan sample images whose first information entropy is higher than a first preset information entropy threshold are filtered to obtain multiple second scan sample images.
[0136] At this time, when the first transceiver unit 801 performs the step of sending multiple frames of the oral cavity scan sample images to the model service platform, it is specifically used for:
[0137] Multiple frames of the second scanned sample images are sent to the model service platform.
[0138] In some embodiments, after the first transceiver unit 801 performs the step of receiving the target AI model sent by the model service platform, the first processing unit 802 is further configured to:
[0139] Based on the target AI model, obtain oral scan test images in the target oral scan scenario; use the first transceiver unit 801 to receive the user's evaluation results for the oral scan test images; if the evaluation result is that the test fails, return to the step of obtaining multiple frames of target oral scan images in the target oral scan scenario; if the evaluation result is that the test passes, then deploy the target AI model.
[0140] In some embodiments, if the evaluation result is that the test passes, the first transceiver unit 801 is further configured to:
[0141] The evaluation results are sent to the model service platform, so that the model service platform stores the target AI model in the AI model library based on the evaluation results; when an authorization operation for the target AI model is received from a user, an authorization instruction for the target AI model is sent to the model service platform, so that the model service platform grants the user indicated by the authorization instruction access to the target AI model in the AI model library based on the authorization instruction.
[0142] In some embodiments, after performing the step of receiving the target AI model sent by the model service platform, the first transceiver unit 801 is further configured to:
[0143] The first processing unit 802 performs three-dimensional reconstruction processing on the multiple frames of the current oral cavity scanning images based on the target AI model to obtain a target three-dimensional model. The target three-dimensional model retains the scanning data corresponding to the labeled area.
[0144] In summary, in this embodiment, when a user does not find an AI model corresponding to the target oral cavity scanning scene in the AI model library, the user can use the first transfer learning-based model training device 800 to collect and label oral cavity scanning sample images in real time. Then, the oral cavity scanning sample images are sent to the model service platform through transfer learning, so that the model service platform can further train the general deep learning model based on the oral cavity scanning sample images to obtain a target AI model corresponding to the target oral cavity scanning scene. The obtained target AI model can accurately identify the images scanned in the target oral cavity scanning scene, thereby improving the image processing effect of the model in the special oral cavity scanning scene (target oral cavity scanning scene).
[0145] Figure 9 This is a schematic block diagram of a second model training device 900 based on transfer learning provided in an embodiment of this application. Figure 9 As shown, corresponding to the above-described transfer learning-based model training method, this application also provides a second transfer learning-based model training apparatus 900. This second transfer learning-based model training apparatus 900 includes units for executing the above-described transfer learning-based model training method. The second transfer learning-based model training apparatus is deployed on a model service platform. The second transfer learning-based model training apparatus includes a second transceiver unit 901 and a second processing unit 902, wherein:
[0146] The second transceiver unit 901 is used to receive a model search command sent by the user terminal for the target oral cavity scanning scenario;
[0147] The second processing unit 902 is used to determine whether there is a target AI model corresponding to the target oral cavity scanning scenario in the preset AI model library, and generate and send the model search results to the user terminal through the second transceiver unit 901;
[0148] The second transceiver unit 901 is further configured to receive multiple frames of oral cavity scan sample images sent by the user terminal when the model search result indicates that the target AI model does not exist in the AI model library of the model service platform;
[0149] The second processing unit 902 is further configured to train a pre-trained general deep learning model based on multiple frames of oral cavity scan sample images to obtain the target AI model;
[0150] The second transceiver unit 901 is also used to send the target AI model to the user terminal.
[0151] In some embodiments, the AI model library stores multiple AI models, each carrying different descriptive information, including model keywords and model scene images; when the second processing unit 902 performs the step of determining whether a target AI model corresponding to the target oral scan scene exists in the preset AI model library, and generating and sending the model search result to the user terminal through the second transceiver unit 901, it is specifically used for:
[0152] When the model search instruction carries target scene keywords, the target scene keywords are matched with the model keywords of each AI model to obtain matching results; when the model search instruction carries target scene images, the target scene images are compared with the model scene images of each AI model to obtain similarity calculation results; the model search results are determined based on the matching results or the similarity calculation results; and the model search results are sent to the user terminal through the second transceiver unit 901.
[0153] In some embodiments, after the second transceiver unit 901 performs the step of sending the target AI model to the user terminal, the second processing unit 902 is further configured to:
[0154] The second transceiver unit 901 receives the evaluation result sent by the user terminal indicating that the target AI model has passed the test; in response to the evaluation result, the target AI model is stored in the AI model library.
[0155] In some embodiments, after performing the step of storing the target AI model into the AI model library, the second processing unit 902 is further configured to:
[0156] The second transceiver unit 901 receives the authorization instruction for the target AI model sent by the user terminal; and grants the user access to the target AI model to the user indicated by the authorization instruction.
[0157] In some embodiments, after the second transceiver unit 901 performs the step of receiving the multi-frame oral cavity scan sample images sent by the user terminal, the second processing unit 902 is further configured to:
[0158] If multiple frames of the oral cavity scan sample images carry a confidentiality identifier, a dedicated sample storage space corresponding to the target AI model is created, and the multiple frames of the oral cavity scan sample images are stored in the dedicated sample storage space; if the multiple frames of the oral cavity scan sample images do not carry a confidentiality identifier, the multiple frames of the oral cavity scan sample images are stored in the public sample storage space of the AI model library.
[0159] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the first transfer learning-based model training device, the second transfer learning-based model training device, and each unit can be referred to the corresponding descriptions in the foregoing method embodiments. For the sake of convenience and brevity, they will not be repeated here.
[0160] The aforementioned first transfer learning-based model training device 800 can be implemented as a computer program, which can be used in, for example... Figure 10 It runs on the user terminal shown.
[0161] Please see Figure 10 , Figure 10 This is a schematic block diagram of a user terminal provided in an embodiment of this application. (See also...) Figure 10 The user terminal 1000 includes a processor 1002, a memory, and a network interface 1005 connected via a system bus 1001. The memory may include a non-volatile storage medium 1003 and internal memory 1004.
[0162] The non-volatile storage medium 1003 may store an operating system 10031 and a computer program 10032. The computer program 10032 includes program instructions that, when executed, cause the processor 1002 to perform a model training method based on transfer learning.
[0163] The processor 1002 provides computing and control capabilities to support the operation of the entire user terminal 1000.
[0164] The internal memory 1004 provides an environment for the execution of the computer program 10032 in the non-volatile storage medium 1003. When the computer program 10032 is executed by the processor 1002, the processor 1002 can execute a model training method based on transfer learning.
[0165] This network interface 1005 is used for network communication with other devices. Those skilled in the art will understand that... Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the user terminal 1000 to which the present application is applied. The specific user terminal 1000 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0166] The processor 1002 is used to run a computer program 10032 stored in the memory to perform the following steps:
[0167] Send a model search command for the target oral cavity scanning scenario to the model service platform, and receive the model search results returned by the model service platform based on the model search command;
[0168] When the model search result indicates that there is no target AI model corresponding to the target oral cavity scanning scene in the AI model library of the model service platform, multiple frames of oral cavity scanning sample images under the target oral cavity scanning scene are obtained;
[0169] Multiple frames of the oral cavity scan sample images are sent to the model service platform, so that the model service platform can train a pre-trained general deep learning model based on the multiple frames of the oral cavity scan sample images to obtain the target AI model;
[0170] Receive the target AI model sent by the model service platform.
[0171] The aforementioned second transfer learning-based model training device 900 can be implemented as a computer program, which can, for example... Figure 11 The model is running on the service platform shown.
[0172] Please see Figure 11 , Figure 11 This is a schematic block diagram of a model service platform provided in an embodiment of this application. (See also...) Figure 11 The model service platform 1100 includes a processor 1102, a memory, and a network interface 1105 connected via a system bus 1101. The memory may include a non-volatile storage medium 1103 and internal memory 1104.
[0173] The non-volatile storage medium 1103 may store an operating system 11031 and a computer program 11032. The computer program 11032 includes program instructions that, when executed, cause the processor 1102 to perform a model training method based on transfer learning.
[0174] The processor 1102 is used to provide computing and control capabilities to support the operation of the entire model service platform 1100.
[0175] The internal memory 1104 provides an environment for the execution of the computer program 11032 in the non-volatile storage medium 1103. When the computer program 11032 is executed by the processor 1102, the processor 1102 can execute a model training method based on transfer learning.
[0176] This network interface 1105 is used for network communication with other devices. Those skilled in the art will understand that... Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the model service platform 1100 to which the present application is applied. The specific model service platform 1100 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0177] The processor 1102 is used to run a computer program 11032 stored in the memory to perform the following steps:
[0178] Receive model search instructions sent by the user terminal for the target oral cavity scanning scenario;
[0179] Determine whether a target AI model corresponding to the target oral cavity scanning scenario exists in the preset AI model library, generate and send the model search results to the user terminal;
[0180] When the model search result indicates that the target AI model does not exist in the AI model library of the model service platform, the system receives multiple frames of oral cavity scan sample images sent by the user terminal.
[0181] The target AI model is obtained by training a pre-trained general deep learning model based on multiple frames of oral cavity scan sample images.
[0182] The target AI model is sent to the user terminal.
[0183] It should be understood that in the embodiments of this application, the processor 1002 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.
[0184] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.
[0185] Therefore, this application also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the following steps:
[0186] Send a model search command for the target oral cavity scanning scenario to the model service platform, and receive the model search results returned by the model service platform based on the model search command;
[0187] When the model search result indicates that there is no target AI model corresponding to the target oral cavity scanning scene in the AI model library of the model service platform, multiple frames of oral cavity scanning sample images under the target oral cavity scanning scene are obtained;
[0188] Multiple frames of the oral cavity scan sample images are sent to the model service platform, so that the model service platform can train a pre-trained general deep learning model based on the multiple frames of the oral cavity scan sample images to obtain the target AI model;
[0189] Receive the target AI model sent by the model service platform.
[0190] Alternatively, it can receive model search instructions sent by the user terminal for the target oral cavity scanning scenario;
[0191] Determine whether a target AI model corresponding to the target oral cavity scanning scenario exists in the preset AI model library, generate and send the model search results to the user terminal;
[0192] When the model search result indicates that the target AI model does not exist in the AI model library of the model service platform, the system receives multiple frames of oral cavity scan sample images sent by the user terminal.
[0193] The target AI model is obtained by training a pre-trained general deep learning model based on multiple frames of oral cavity scan sample images.
[0194] The target AI model is sent to the user terminal.
[0195] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.
[0196] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.
[0197] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.
[0198] The steps in the methods of this application embodiment can be adjusted, merged, or deleted according to actual needs. The units in the apparatus of this application embodiment can be merged, divided, or deleted according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0199] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a user terminal (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.
[0200] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A model training method based on transfer learning, characterized in that, The method is applied to a user terminal, and the method includes: Send a model search command for the target oral cavity scanning scenario to the model service platform, and receive the model search results returned by the model service platform based on the model search command; When the model search result indicates that there is no target AI model corresponding to the target oral cavity scanning scene in the AI model library of the model service platform, multiple frames of oral cavity scanning sample images under the target oral cavity scanning scene are obtained; Multiple frames of the oral cavity scan sample images are sent to the model service platform, so that the model service platform can train a pre-trained general deep learning model based on the multiple frames of the oral cavity scan sample images to obtain the target AI model; Receive the target AI model sent by the model service platform.
2. The method according to claim 1, characterized in that, The acquisition of multiple frames of oral cavity scan sample images in the target oral cavity scan scene includes: Acquire multiple frames of target oral cavity scanning images in the target oral cavity scanning scene; Based on preset image annotation rules, each frame of the target oral cavity scan image is annotated to obtain multiple frames of oral cavity scan sample images containing the annotated regions.
3. The method according to claim 2, characterized in that, The target oral cavity scan image is annotated in each frame based on a preset image annotation rule to obtain multiple frames of oral cavity scan sample images containing the annotated regions, including: Based on a preset pre-segmentation model, each frame of the target oral cavity scan image is pre-segmented to obtain multiple frames of pre-segmented oral cavity scan images. The system receives annotation operations from the user for each frame of the pre-segmented oral cavity scan image to determine the annotation regions corresponding to each frame of the pre-segmented oral cavity scan image, thereby obtaining multiple frames of oral cavity scan sample images.
4. The method according to claim 2, characterized in that, The target oral cavity scan image is annotated in each frame based on a preset image annotation rule to obtain multiple frames of oral cavity scan sample images containing the annotated regions, including: A raw three-dimensional model of the oral cavity is generated based on multiple frames of the target oral cavity scan images; Receive annotation operations from the user on the original oral cavity 3D model to determine the annotation area of the original oral cavity 3D model; A back-projection operation is performed on the labeled areas of the original oral cavity 3D model to project the labeled areas onto each of the target oral cavity scan images, thereby obtaining multiple frames of the oral cavity scan sample images.
5. The method according to claim 4, characterized in that, The back-projection operation on the labeled areas of the original oral cavity 3D model, to project the labeled areas onto each of the target oral cavity scan images, to obtain multiple frames of the oral cavity scan sample images, includes: Determine the target 3D coordinates of each pixel in the labeled area in the world coordinate system; Based on a preset coordinate transformation algorithm, according to the three-dimensional coordinates of each target and the projection matrix corresponding to each target oral cavity scan image, the two-dimensional coordinates of each target on each oral cavity scan image are determined. The coordinate transformation algorithm is as follows: x = PX; Where x is the two-dimensional coordinate of the target, X is the three-dimensional coordinate of the target, P is the projection matrix of the camera when scanning the target oral cavity scan image, P=K[R|t], K is the intrinsic parameter matrix of the camera, R is the rotation matrix, and t is the translation vector; For each of the target oral cavity scan images, the oral cavity scan sample image is determined based on the target two-dimensional coordinates in the target oral cavity scan image.
6. The method according to claim 2, characterized in that, The acquisition of multiple frames of target oral cavity scan images in the target oral cavity scan scene includes: Acquire multiple candidate oral cavity scan images in the target oral cavity scan scene; Obtain the scanning pose corresponding to each frame of the candidate oral cavity scan image, and / or extract the contour features corresponding to each frame of the candidate oral cavity scan image through a preset feature extraction model; The similarity of pose between any two adjacent candidate oral cavity scan images is determined based on the scan pose, and / or the similarity of features between any two adjacent candidate oral cavity scan images is determined based on the contour features; Based on the pose similarity and / or the feature similarity, multiple frames of the candidate oral cavity scan images are filtered to obtain multiple frames of the target oral cavity scan images.
7. The method according to claim 2, characterized in that, After annotating each frame of the target oral cavity scan image based on preset image annotation rules to obtain multiple frames of oral cavity scan sample images containing the annotated regions, the method further includes: The multiple frames of oral cavity scan sample images are manually screened to obtain multiple frames of first scan sample images; Obtain the general deep learning model from the model service platform; The first information entropy corresponding to each of the oral cavity scan sample images is determined by the general deep learning model. Filter the first scan sample images in multiple frames of the first scan sample images in which the first information entropy is higher than the first preset information entropy threshold to obtain multiple frames of the second scan sample images. Sending multiple frames of the oral cavity scan sample images to the model service platform includes: Multiple frames of the second scanned sample images are sent to the model service platform.
8. The method according to claim 1, characterized in that, After receiving the target AI model sent by the model service platform, the method further includes: Based on the target AI model, obtain oral scan test images in the target oral scan scenario; Receive user evaluation results for the oral cavity scan test images; If the evaluation result is that the test fails, return to the step of obtaining multiple frames of oral scan sample images in the target oral scan scene; If the evaluation result indicates that the test is passed, then the target AI model is deployed.
9. The method according to claim 8, characterized in that, If the evaluation result is that the test passes, the method further includes: The evaluation results are sent to the model service platform so that the model service platform stores the target AI model in the AI model library based on the evaluation results; When a user grants an authorization to the target AI model, an authorization instruction for the target AI model is sent to the model service platform, so that the model service platform grants the user indicated by the authorization instruction access to the target AI model in the AI model library.
10. The method according to claim 2, characterized in that, After receiving the target AI model sent by the model service platform, the method further includes: Acquire multiple frames of current oral cavity scan images in the target oral cavity scan scene; Based on the target AI model, three-dimensional reconstruction processing is performed on multiple frames of the current oral cavity scan images to obtain a target three-dimensional model, which retains the scan data corresponding to the labeled area.
11. A model training method based on transfer learning, characterized in that, The method is applied to a model service platform, and the method includes: Receive model search instructions sent by the user terminal for the target oral cavity scanning scenario; Determine whether a target AI model corresponding to the target oral cavity scanning scenario exists in the preset AI model library, generate and send the model search results to the user terminal; When the model search result indicates that the target AI model does not exist in the AI model library of the model service platform, the system receives multiple frames of oral cavity scan sample images sent by the user terminal. The target AI model is obtained by training a pre-trained general deep learning model based on multiple frames of oral cavity scan sample images. The target AI model is sent to the user terminal.
12. The method according to claim 11, characterized in that, The AI model library stores multiple AI models, each carrying different descriptive information, including model keywords and model scene images. The step of determining whether a target AI model corresponding to the target oral cavity scan scene exists in the preset AI model library, generating and sending model search results to the user terminal, includes: When the model search instruction carries target scene keywords, the target scene keywords are matched with the model keywords of each AI model to obtain matching results; When the model search instruction carries a target scene image, the target scene image is compared with the model scene image of each AI model to obtain the similarity calculation result. The model search results are determined based on the matching results or the similarity calculation results; The model search results are sent to the user terminal.
13. The method according to claim 11, characterized in that, After sending the target AI model to the user terminal, the method further includes: Receive the evaluation result sent by the user terminal, indicating that the target AI model has passed the test; In response to the evaluation results, the target AI model is stored in the AI model library.
14. The method according to claim 13, characterized in that, After storing the target AI model into the AI model library, the method further includes: Receive the authorization instruction for the target AI model sent by the user terminal; Grant access to the target AI model to the user specified in the authorization instruction.
15. The method according to claim 11, characterized in that, After receiving the multi-frame oral cavity scan sample images sent by the user terminal, the method further includes: If multiple frames of the oral cavity scan sample images carry a confidentiality identifier, a dedicated sample storage space corresponding to the target AI model is created, and the multiple frames of the oral cavity scan sample images are stored in the dedicated sample storage space. If the multiple frames of the oral cavity scan sample images do not carry a confidentiality identifier, the multiple frames of the oral cavity scan sample images will be stored in the common sample storage space of the AI model library.
16. A first model training device based on transfer learning, characterized in that, The first transfer learning-based model training device is deployed on a user terminal. The first transfer learning-based model training device includes a first transceiver unit and a first processing unit. The first processing unit is used to control the transceiver operations of the first transceiver unit, wherein: The first transceiver unit is configured to send a model search instruction for a target oral cavity scanning scene to the model service platform, and receive the model search results returned by the model service platform based on the model search instruction; when the model search results indicate that there is no target AI model corresponding to the target oral cavity scanning scene in the AI model library of the model service platform, the unit acquires multiple frames of oral cavity scanning sample images under the target oral cavity scanning scene; sends the multiple frames of oral cavity scanning sample images to the model service platform, so that the model service platform trains a pre-trained general deep learning model based on the multiple frames of oral cavity scanning sample images to obtain the target AI model; and receives the target AI model sent by the model service platform.
17. A second model training device based on transfer learning, characterized in that, The second transfer learning-based model training device is deployed on the model service platform. The second transfer learning-based model training device includes a second transceiver unit and a second processing unit, wherein: The second transceiver unit is used to receive model search instructions for the target oral cavity scanning scenario sent by the user terminal; The second processing unit is used to determine whether there is a target AI model corresponding to the target oral cavity scanning scenario in the preset AI model library, and to generate and send the model search results to the user terminal through the second transceiver unit; The second transceiver unit is further configured to receive multiple frames of oral cavity scan sample images sent by the user terminal when the model search result indicates that the target AI model does not exist in the AI model library of the model service platform; The processing unit is further configured to train a pre-trained general deep learning model based on multiple frames of oral cavity scan sample images to obtain the target AI model. The second transceiver unit is also used to send the target AI model to the user terminal.
18. A user terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the model training method based on transfer learning as described in any one of claims 1-10.
19. A model service platform, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the model training method based on transfer learning as described in any one of claims 11-15.
20. A storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, cause the processor to perform the model training method based on transfer learning as described in any one of claims 1-10 or 11-15.
