Method, system and device for acquiring microscopic images of oral environment without blind spots

Abstract

The present invention discloses a method, system and equipment for acquiring a microscopic image of an oral environment without blind spots, and relates to the field of image processing technology; the system comprises a collection end that collects a pre-processed image of a to-be-detected area in real time, and sends the pre-processed image to an image processing end, so that the image processing end identifies an abnormal image area in the pre-processed image, and sends a collection instruction to the collection end, so that the collection end collects the image of the abnormal area as a target abnormal image; by collecting the pre-processed images of various areas in the oral cavity in real time, it is ensured that every detail can be captured, the abnormal area is quickly identified, and the collection instruction is immediately sent to the collection end, which greatly shortens the detection time and improves the detection efficiency, and also ensures that the doctor can obtain the oral condition of the patient in real time, which provides help for the formulation of a treatment plan.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a method, system and device for acquiring a microscopic image of an oral environment without blind spots. Background Art

[0002] With the continuous progress of society and the improvement of people's health awareness, oral health has gradually changed from a "minor illness" in the past to a health focus that has received much attention today. More and more people have begun to realize that the oral cavity is not only the starting point of the digestive system, but also a "barometer" of physical health. Therefore, when there is discomfort or problem in the oral cavity, they no longer choose to ignore it or deal with it on their own, but actively go to the hospital to seek professional diagnosis and treatment.

[0003] In the current dental diagnosis process, doctors usually rely on traditional inspection tools such as oral mirrors and probes that rely on naked eye observation and manual operation to examine the patient's oral condition. Although this traditional diagnostic method can detect oral problems to a certain extent, due to the complexity and irregularity of the internal structure of the oral cavity, some areas such as the gaps between teeth, deep in the mouth, or the root of the tongue may be difficult to observe directly, which may lead to some potential lesions or abnormalities being missed, thereby affecting the comprehensiveness and accuracy of the diagnosis. Identifying abnormalities in the oral cavity is highly dependent on the doctor's professional skills and experience, which increases the subjectivity and uncertainty of the diagnosis and may lead to misdiagnosis or missed diagnosis, resulting in low detection efficiency. Summary of the invention

[0004] The purpose of the present invention is to solve the above-mentioned problem and to propose a method, system and device for acquiring a microscopic image of the oral environment without blind spots.

[0005] In the first aspect of the present invention, a system for acquiring microscopic images of an oral environment without blind spots is first proposed, wherein the system comprises an acquisition end and an image processing end; wherein:

[0006] The acquisition end is used to acquire a pre-processed image of the area to be detected in real time and send the pre-processed image to the image processing end; the area to be detected is the oral cavity of the target user; the acquisition area of ​​the pre-processed image is smaller than the area to be detected; the pre-processed image is an image captured by moving the acquisition window with the first step length and with the first preset period;

[0007] The image processing end is used to identify the abnormal image area in the pre-processed image, and if there is an abnormal image area, send a collection instruction to the collection end according to the abnormal area;

[0008] The acquisition end is further used to acquire an image of the abnormal area as a target abnormal image according to the acquisition instruction;

[0009] The image processing end is further used to perform image integration on all collected pre-processed images to obtain an overall oral image of the target user, and to mark abnormal areas in the overall oral image according to all target abnormal images to obtain a final image.

[0010] In a second aspect of the present invention, a method for acquiring a microscopic image of an oral environment without blind spots is proposed, wherein the method is applied to an image processing end and comprises:

[0011] Receive a preprocessed image of the area to be detected collected by the collection end in real time, and identify an abnormal image area in the preprocessed image; the area to be detected is the oral cavity of the target user; the collection area of ​​the preprocessed image is smaller than the area to be detected; the preprocessed image is an image captured by moving the collection window with the first step length and with a first preset period;

[0012] If there is an abnormal image area, a collection instruction is sent to the collection end according to the abnormal area, so that the collection end collects the image of the abnormal area as the target abnormal image according to the collection instruction;

[0013] Performing image integration on all collected pre-processed images to obtain an overall oral image of the target user;

[0014] All target abnormal images are acquired to mark abnormal areas in the overall oral cavity image to obtain a final image.

[0015] Optionally, receiving a preprocessed image of the area to be detected collected in real time by the collection end, and identifying an abnormal image area in the preprocessed image includes:

[0016] Performing HLS color space transformation and XYZ color space transformation on the preprocessed image to obtain a first color image and a second color image;

[0017] Performing anomaly detection on the first color image and the second color image respectively by using the YOLO algorithm;

[0018] If both the first color image and the second color image have image anomalies, superimposing the abnormal areas in the first color image and the second color image to obtain a target abnormal area;

[0019] The abnormal region in the preprocessed image is marked according to the target abnormal region to obtain the image abnormal region.

[0020] Optionally, the acquisition instruction includes a first coordinate, a second step size moving acquisition window and a second preset period; the first coordinate is the coordinate of the abnormal image area; the second step size moving acquisition window is smaller than the first step size moving acquisition window; and the second preset period is smaller than the first preset period.

[0021] Optionally, performing image integration on all collected pre-processed images to obtain an overall oral image of the target user includes:

[0022] For each preprocessed image, a feature point set on the preprocessed image is obtained by a preset algorithm;

[0023] The feature point sets in the adjacent pre-processed images are matched by the FLANN algorithm;

[0024] If the matching success rate of all feature points in the feature point set is greater than a preset threshold, the overlapping areas in the adjacent pre-processed images are fused according to the successfully matched feature points to obtain an initial local oral cavity image;

[0025] The overlapping areas of all initial partial oral images are superimposed to obtain the overall oral image.

[0026] Optionally, performing feature point matching on feature point sets in adjacent pre-processed images by using the FLANN algorithm specifically includes:

[0027] For each feature point, the adjacent feature points of the feature point are obtained to obtain an adjacent feature set, and the feature points in the adjacent feature set are sorted in ascending order based on the distances between the feature point and all adjacent feature points to obtain a target adjacent feature set;

[0028] Taking the feature point as a vertex, sequentially calculating the angles formed by two adjacent feature points in the target adjacent feature set to obtain a target angle feature set;

[0029] Constructing a FLANN index according to the number of the target adjacent feature sets, the distance between each adjacent feature point in the target adjacent feature set and the feature point, and the target angle feature set;

[0030] For each feature point, the FLANN algorithm is used to perform feature point matching; the feature point matching is to first determine the matching point that meets the requirements through the number of adjacent feature sets, and then match the target adjacent feature set and the target angle feature set.

[0031] In a third aspect of the present invention, a method for acquiring a microscopic image of an oral environment without blind spots is proposed, wherein the method is applied to a collection end and comprises:

[0032] The pre-processed image of the area to be detected is collected in real time, and the pre-processed image is sent to the image processing end; so that the image processing end identifies the abnormal image area in the pre-processed image, and if there is an abnormal image area, a collection instruction is sent to the collection end according to the abnormal area; the area to be detected is the oral cavity of the target user; the collection area of ​​the pre-processed image is smaller than the area to be detected; the pre-processed image is an image captured by moving the collection window with the first step length and with the first preset period;

[0033] According to the acquisition instruction, the image of the abnormal area is acquired as the target abnormal image; so that the image processing end integrates all the acquired pre-processed images to obtain the overall oral image of the target user, and the abnormal area in the overall oral image is marked according to all the target abnormal images to obtain the final image.

[0034] Optionally, the acquisition instruction includes a first coordinate, a second step length moving acquisition window, and a second preset period; the first coordinate is the coordinate of the abnormal image area; the second step length moving acquisition window is smaller than the first step length moving acquisition window; the second preset period is smaller than the first preset period; and acquiring an image of the abnormal area as a target abnormal image according to the acquisition instruction includes:

[0035] Determine the second coordinate of the first coordinate in the current acquisition window, adjust the focal length of the camera according to the second coordinate, move the acquisition window with a second step length and a second preset period to acquire an image of the target abnormal area to obtain a target abnormal image set; the target abnormal area is the position corresponding to the image abnormal area in the area to be detected;

[0036] When the image captured by moving the acquisition window with the second step size and the second preset period does not contain the abnormal area, the image of the area to be detected is captured again with the first step size and the first preset period until the target mark appears in the captured image, and the image capture is terminated; the target mark is set for the first capture of the area to be detected.

[0037] The acquisition end, when the image acquired by moving the acquisition window with the second step size and the second preset period does not contain the abnormal area, re-acquires the image of the area to be detected by moving the acquisition window with the first step size and the first preset period until the area to be detected reaches the termination condition.

[0038] In a fourth aspect of the present invention, an electronic device is provided, comprising:

[0039] A memory for storing executable instructions;

[0040] The processor is used to implement the method for acquiring a microscopic image of an oral environment without blind spots as described in any one of claims 2 to 6 when executing the executable instructions stored in the memory.

[0041] In a fifth aspect of the present invention, an electronic device is provided, comprising:

[0042] A memory for storing executable instructions;

[0043] The processor is used to implement the method for acquiring a microscopic image of an oral environment without blind spots as described in any one of claims 7 to 8 when executing the executable instructions stored in the memory.

[0044] Beneficial effects of the present invention:

[0045] The present invention proposes a blind-angle-free acquisition system based on oral environment microscopic images, the system comprising an acquisition end and an image processing end; wherein: the acquisition end is used to acquire preprocessed images of a to-be-detected area in real time and send the preprocessed images to the image processing end; the to-be-detected area is the oral cavity of a target user; the acquisition area of ​​the preprocessed image is smaller than the to-be-detected area; the preprocessed image is an image captured by moving the acquisition window with a first step length and with a first preset period; the image processing end is used to identify image abnormality areas in the preprocessed image, and if there are image abnormality areas, an acquisition instruction is sent to the acquisition end according to the abnormal area; the acquisition end is also used to acquire images of the abnormal area as a target abnormal image according to the acquisition instruction; the image processing end is also used to integrate all acquired preprocessed images to obtain an overall oral image of the target user, and mark the abnormal areas in the overall oral image according to all target abnormal images to obtain a final image. By collecting pre-processed images of various areas in the oral cavity in real time, every detail can be captured. The pre-processed images are taken at a specific step size and period, which effectively avoids omissions and realizes no-dead-angle detection of the oral environment. The pre-processed images can also be analyzed in real time, abnormal areas can be quickly identified, and collection instructions can be immediately sent to the collection end, which greatly shortens the detection time and improves the detection efficiency. At the same time, it also ensures that the doctor can obtain the patient's oral condition in real time, which provides assistance for the formulation of treatment plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The present invention will be further described below in conjunction with the accompanying drawings.

[0047] Figure 1 A flowchart of a method for acquiring a microscopic image of an oral environment without blind spots is provided for an embodiment of the present invention;

[0048] Figure 2 A flowchart of another method for acquiring a microscopic image of an oral environment without blind spots is provided for an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of the structure of an electronic device is provided for an embodiment of the present invention;

[0050] Figure 4 A structural schematic diagram of another electronic device is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0051] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0052] The embodiment of the present invention provides a system for acquiring microscopic images of the oral environment without blind spots, the system comprising an acquisition end and an image processing end; wherein:

[0053] The acquisition end is used to acquire the pre-processed image of the area to be detected in real time and send the pre-processed image to the image processing end; the area to be detected is the oral cavity of the target user; the acquisition area of ​​the pre-processed image is smaller than the area to be detected; the pre-processed image is an image captured by moving the acquisition window with the first step length and with the first preset period;

[0054] The image processing end is used to identify abnormal image areas in the preprocessed image, and if there are abnormal image areas, send a collection instruction to the collection end according to the abnormal areas;

[0055] The acquisition end is also used to acquire an image of the abnormal area as a target abnormal image according to the acquisition instruction;

[0056] The image processing end is also used to integrate all the collected pre-processed images to obtain the overall oral image of the target user, and to mark the abnormal areas in the overall oral image according to all the target abnormal images to obtain the final image.

[0057] The system for acquiring oral environment microscopic images without blind spots provided by the embodiment of the present invention can capture preprocessed images of various areas in the oral cavity in real time to ensure that every detail can be captured. The preprocessed images are taken with a specific step size and period, which effectively avoids omissions and realizes blind spot detection of the oral environment. The preprocessed images can also be analyzed in real time to quickly identify abnormal areas and immediately send acquisition instructions to the acquisition end, which greatly shortens the detection time and improves the detection efficiency. It also ensures that the doctor can immediately obtain the patient's oral condition, which provides assistance for the formulation of treatment plans.

[0058] The embodiment of the present invention provides a method for obtaining a microscopic image of the oral environment without blind spots. Figure 1 , Figure 1 A flowchart of a method for acquiring a microscopic image of an oral environment without blind spots provided by an embodiment of the present invention. The method is applied to an image processing end and includes:

[0059] S101, receiving a pre-processed image of a region to be detected collected by a collection end in real time, and identifying an abnormal image region in the pre-processed image.

[0060] S102: If there is an abnormal image area, a collection instruction is sent to the collection end according to the abnormal area, so that the collection end collects the image of the abnormal area as the target abnormal image according to the collection instruction.

[0061] S103, performing image integration on all collected pre-processed images to obtain an overall oral image of the target user.

[0062] S104, acquiring all target abnormal images and marking abnormal areas in the overall oral cavity image to obtain a final image.

[0063] The area to be detected is the oral cavity of the target user; the acquisition area of ​​the preprocessed image is smaller than the area to be detected; and the preprocessed image is an image captured by moving the acquisition window with the first step length and with the first preset period.

[0064] Based on the method for acquiring oral environment microscopic images without blind spots provided by the embodiment of the present invention, pre-processed images of various areas in the oral cavity are collected in real time to ensure that every detail can be captured. The pre-processed images are taken with specific step sizes and cycles to effectively avoid omissions and achieve blind spot detection of the oral environment. The pre-processed images can also be analyzed in real time to quickly identify abnormal areas and immediately send acquisition instructions to the acquisition end, which greatly shortens the detection time and improves the detection efficiency. At the same time, it also ensures that the doctor can immediately obtain the patient's oral condition, which provides assistance for the formulation of treatment plans.

[0065] In one implementation, the first step length and the first preset period are set by the user according to actual conditions.

[0066] In one implementation, through real-time acquisition at the acquisition end and precise recognition at the image processing end, microscopic images of the oral environment can be acquired without blind spots. Whether it is teeth, oral mucosa or other oral structures, comprehensive and detailed observations can be obtained, which helps doctors or researchers to fully understand the patient's oral condition.

[0067] In one implementation, the recognition and analysis of preprocessed images can timely detect abnormal areas in the oral cavity, including potential lesions, inflammation and other problems. This early detection is of great significance for the prevention and treatment of oral diseases, and can effectively avoid the deterioration of the disease and relieve the pain of patients.

[0068] In one embodiment, receiving a preprocessed image of the area to be detected collected by the collection end in real time, and identifying an abnormal image area in the preprocessed image includes:

[0069] Performing HLS color space transformation and XYZ color space transformation on the preprocessed image to obtain a first color image and a second color image;

[0070] Performing anomaly detection on the first color image and the second color image respectively by using the YOLO algorithm;

[0071] If both the first color image and the second color image have image anomalies, superimposing the abnormal regions in the first color image and the second color image to obtain a target abnormal region;

[0072] The abnormal area in the preprocessed image is marked according to the target abnormal area to obtain the image abnormal area.

[0073] In one implementation, by transforming in different color spaces (HLS and XYZ), this method can extract feature information in the image from multiple angles. The HLS color space focuses on changes in hue, brightness, and saturation, while the XYZ color space focuses on the basic components of color. Combining the transformations of these two color spaces can more comprehensively capture subtle differences in oral images, thereby more accurately identifying abnormal areas.

[0074] In one implementation, the YOLO algorithm is used to perform anomaly detection on images in the HLS and XYZ color spaces respectively. This collaborative working method of multiple algorithms can effectively reduce the misjudgment or missed detection that may be caused by a single algorithm. When images in both color spaces are abnormal, they are judged as true abnormal areas, which enhances the stability and reliability of the entire detection process.

[0075] In one implementation, the target abnormal area is obtained by superimposing the abnormal areas in the HLS and XYZ color spaces. This method can more accurately locate the position and range of the abnormal area. Subsequently, the preprocessed image is marked according to the target abnormal area, so that doctors or researchers can intuitively see which areas have abnormalities, providing a strong basis for subsequent diagnosis and treatment.

[0076] In one embodiment, the acquisition instruction includes a first coordinate, a second step length moving acquisition window and a second preset period; the first coordinate is the coordinate of the abnormal image area; the second step length moving acquisition window is smaller than the first step length moving acquisition window; the second preset period is smaller than the first preset period.

[0077] In one embodiment, performing image integration on all collected pre-processed images to obtain an overall oral image of the target user includes:

[0078] For each preprocessed image, a feature point set on the preprocessed image is obtained by a preset algorithm;

[0079] The feature point sets in the adjacent pre-processed images are matched by the FLANN algorithm;

[0080] If the matching success rate of all feature points in the feature point set is greater than a preset threshold, the overlapping areas in the adjacent pre-processed images are fused according to the successfully matched feature points to obtain an initial local oral cavity image;

[0081] The overlapping areas of all initial partial oral images are superimposed to obtain the overall oral image.

[0082] In one implementation, a set of feature points on each preprocessed image is obtained through a preset algorithm, which can accurately extract key information points in the image. Subsequently, the feature points in adjacent images are matched using the FLANN algorithm to ensure that the correct corresponding points can be found when the images are stitched together. This method significantly improves the accuracy of image stitching and avoids image misalignment or distortion caused by mismatching.

[0083] In one implementation, when the matching success rate of all feature points in the feature point set is greater than a preset threshold, the overlapping areas in adjacent images are fused according to the successfully matched feature points to obtain an initial local oral image. This fusion method based on feature point matching can achieve seamless stitching between images, making the overall oral image more visually coherent and natural.

[0084] In one implementation, feature point matching and image fusion methods can not only ensure accurate image stitching, but also retain the detailed information in the image to the greatest extent. This is especially important for microscopic images of the oral environment, because detailed information is often the key basis for doctors to make diagnosis and treatment.

[0085] In one embodiment, performing feature point matching on feature point sets in adjacent pre-processed images by using the FLANN algorithm specifically includes:

[0086] For each feature point, the adjacent feature points of the feature point are obtained to obtain an adjacent feature set, and the distances between the feature point and all adjacent feature points are used to sort the feature points in the adjacent feature set in ascending order to obtain a target adjacent feature set;

[0087] Taking the feature point as the vertex, the angles formed by two adjacent feature points in the target adjacent feature set are calculated in sequence to obtain the target angle feature set;

[0088] Construct a FLANN index based on the number of target adjacent feature sets, the distance between each adjacent feature point and the feature point in the target adjacent feature set, and the target angle feature set;

[0089] For each feature point, the FLANN algorithm is used to perform feature point matching; the feature point matching is to first determine the matching points according to the number of adjacent feature sets, and then match them according to the target adjacent feature set and the target angle feature set.

[0090] In one implementation, for each feature point, its adjacent feature points are first obtained to form an adjacent feature set, and then the feature points in the adjacent feature set are sorted in ascending order according to the distance between the feature point and all adjacent feature points, thereby obtaining the target adjacent feature set. The advantage of this is that points that are closer to the current feature point can be given priority, thereby improving the accuracy of matching.

[0091] In one implementation, the current feature point is used as the vertex to calculate the angle formed by two adjacent feature points in the target adjacent feature set to form a target angle feature set. The angle information is crucial for describing the geometric structure around the feature point, and can further enrich the feature description and improve the robustness of the matching.

[0092] In one implementation, by combining distance and angle information to construct a target adjacent feature set and a target angle feature set, the geometric structure of the feature point and its surroundings can be more comprehensively described, thereby improving the accuracy of matching.

[0093] Based on the same inventive concept, the present invention also provides a method for obtaining a microscopic image of an oral environment without blind spots. Figure 2 , Figure 2 Another method for acquiring a microscopic image of an oral environment without blind spots provided by an embodiment of the present invention, the method is applied to a collection end and includes:

[0094] S201, collect the pre-processed image of the area to be detected in real time, and send the pre-processed image to the image processing end; so that the image processing end can identify the image abnormal area in the pre-processed image, and if there is an image abnormal area, send a collection instruction to the collection end according to the abnormal area.

[0095] S202, according to the acquisition instruction, an image of the abnormal area is acquired as a target abnormal image; so that the image processing end integrates all the acquired pre-processed images to obtain the overall oral image of the target user, and annotates the abnormal area in the overall oral image according to all the target abnormal images to obtain the final image.

[0096] The area to be detected is the oral cavity of the target user; the acquisition area of ​​the preprocessed image is smaller than the area to be detected; and the preprocessed image is an image captured by moving the acquisition window with the first step length and with the first preset period.

[0097] Based on the method for acquiring oral environment microscopic images without blind spots provided by the embodiment of the present invention, pre-processed images of various areas in the oral cavity are collected in real time to ensure that every detail can be captured. The pre-processed images are taken with specific step sizes and cycles, which effectively avoids omissions and realizes blind spot detection of the oral environment. The pre-processed images can also be analyzed in real time, abnormal areas can be quickly identified, and collection instructions can be immediately sent to the collection end, which greatly shortens the detection time and improves the detection efficiency. At the same time, it also ensures that the doctor can obtain the patient's oral condition in real time, which provides assistance for the formulation of treatment plans.

[0098] In one implementation, the pre-processed image of the area to be detected is collected in real time and immediately sent to the image processing end for abnormal area identification. This method can achieve instant feedback. Once the image processing end identifies the abnormal area, it can quickly send a collection instruction to the collection end to ensure that the abnormal area is accurately located and further collected in detail. This real-time performance is crucial for the early detection and timely treatment of oral diseases.

[0099] In one implementation, since the acquisition area of ​​the preprocessed image is smaller than the area to be detected and is acquired with a certain step size and period, this method can reduce the number of unnecessary image acquisitions while ensuring full coverage, which not only improves acquisition efficiency, but also optimizes resource utilization and reduces storage and processing costs.

[0100] In one implementation, automated image processing and abnormal area recognition can greatly reduce the time doctors spend on manual inspection and analysis of images, thereby improving diagnostic efficiency. At the same time, because the image processing end uses advanced algorithms and technologies to identify abnormal areas, its accuracy has also been significantly improved, helping doctors make more accurate diagnostic and treatment decisions.

[0101] In one embodiment, the acquisition instruction includes a first coordinate, a second step length moving acquisition window, and a second preset period; the first coordinate is the coordinate of the abnormal image area; the second step length moving acquisition window is smaller than the first step length moving acquisition window; the second preset period is smaller than the first preset period; and acquiring an image of the abnormal area as a target abnormal image according to the acquisition instruction includes:

[0102] Determine the second coordinate of the first coordinate in the current acquisition window, adjust the focal length of the camera according to the second coordinate, move the acquisition window with a second step length and a second preset period to acquire an image of the target abnormal area to obtain a target abnormal image set; the target abnormal area is the position corresponding to the image abnormal area in the area to be detected;

[0103] When the image acquired with the second step size moving the acquisition window and the second preset period does not contain the abnormal area, the image of the area to be detected is acquired again with the first step size and the first preset period until the target mark appears in the acquired image, and the image acquisition is terminated; the target mark is set for the first acquisition of the area to be detected.

[0104] At the acquisition end, when the image acquired by moving the acquisition window with the second step size and the second preset period does not contain the abnormal area, the image of the area to be detected is acquired again by moving the acquisition window with the first step size and the first preset period until the area to be detected reaches the termination condition.

[0105] In one implementation, by determining the coordinates of the abnormal area of ​​the image (i.e., the first coordinate) and adjusting the focal length of the camera according to the coordinates, it is possible to ensure that the camera is accurately aimed at the abnormal area for acquisition. This focusing method improves the accuracy of acquisition and helps to obtain clearer and more detailed images of the abnormal area.

[0106] In one implementation, using a second step size to move the acquisition window and a second preset period to acquire images of the target abnormal area helps to capture more detailed information. At the same time, since high-frequency acquisition is only performed in the abnormal area, indiscriminate intensive acquisition in the entire area to be detected is avoided, thereby saving resources and improving acquisition efficiency.

[0107] In one implementation, when the image acquired by moving the acquisition window with the second step size and the second preset period no longer contains the abnormal area, the system can automatically re-acquire the image of the area to be detected with the first step size and the first preset period. This mechanism ensures continuous tracking of the abnormal area and re-acquisition when necessary to avoid missed detection or false detection.

[0108] The embodiment of the present invention further provides an electronic device, such as Figure 3 As shown, it includes a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304.

[0109] Memory 303, used for storing computer programs;

[0110] The processor 301 is used to execute the program stored in the memory 303 to implement the following steps:

[0111] Receive the pre-processed image of the area to be detected collected by the acquisition end in real time, and identify the image abnormal area in the pre-processed image;

[0112] If there is an abnormal image area, a collection instruction is sent to the collection end according to the abnormal area, so that the collection end collects the image of the abnormal area as the target abnormal image according to the collection instruction;

[0113] Perform image integration on all collected pre-processed images to obtain the overall oral image of the target user;

[0114] Obtain all target abnormal images and mark the abnormal areas in the overall oral image to obtain the final image;

[0115] The area to be detected is the oral cavity of the target user; the acquisition area of ​​the preprocessed image is smaller than the area to be detected; and the preprocessed image is an image captured by moving the acquisition window with the first step length and with the first preset period.

[0116] The embodiment of the present invention also provides another electronic device, such as Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0117] Memory 403, used for storing computer programs;

[0118] The processor 401 is used to execute the program stored in the memory 403 to implement the following steps:

[0119] The pre-processed image of the area to be detected is collected in real time and sent to the image processing end; so that the image processing end can identify the abnormal image area in the pre-processed image, and if there is an abnormal image area, a collection instruction is sent to the collection end according to the abnormal area.

[0120] According to the acquisition instruction, the image of the abnormal area is acquired as the target abnormal image; so that the image processing end integrates all the acquired pre-processed images to obtain the overall oral image of the target user, and the abnormal area in the overall oral image is marked according to all the target abnormal images to obtain the final image.

[0121] The area to be detected is the oral cavity of the target user; the acquisition area of ​​the preprocessed image is smaller than the area to be detected; and the preprocessed image is an image captured by moving the acquisition window with the first step length and with the first preset period.

[0122] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0123] The communication interface is used for communication between the above electronic device and other devices.

[0124] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0125] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0126] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0127] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device and storage medium, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0128] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A system for acquiring oral environment microscopic images without blind spots, characterized in that: The system includes an acquisition end and an image processing end; wherein: The acquisition end is used to acquire a pre-processed image of the area to be detected in real time and send the pre-processed image to the image processing end; the area to be detected is the oral cavity of the target user; the acquisition area of ​​the pre-processed image is smaller than the area to be detected; the pre-processed image is an image captured by moving the acquisition window with the first step length and with the first preset period; The image processing end is used to identify the abnormal image area in the pre-processed image, and if there is an abnormal image area, send a collection instruction to the collection end according to the abnormal area; The acquisition end is further used to acquire an image of the abnormal area as a target abnormal image according to the acquisition instruction; The image processing end is further used to perform image integration on all collected pre-processed images to obtain an overall oral image of the target user, and to mark abnormal areas in the overall oral image according to all target abnormal images to obtain a final image; Identifying an abnormal image region in the preprocessed image includes: Performing HLS color space transformation and XYZ color space transformation on the preprocessed image to obtain a first color image and a second color image; Performing anomaly detection on the first color image and the second color image respectively by using the YOLO algorithm; If both the first color image and the second color image have image anomalies, superimposing the abnormal areas in the first color image and the second color image to obtain a target abnormal area; The abnormal region in the preprocessed image is marked according to the target abnormal region to obtain the image abnormal region.

2. A method for acquiring oral environment microscopic images without blind spots, characterized in that: The method applied to the system for acquiring oral environment microscopic images without blind spots as described in claim 1, wherein the method is applied to the image processing end and comprises: Receive a preprocessed image of the area to be detected collected by the collection end in real time, and identify an abnormal image area in the preprocessed image; the area to be detected is the oral cavity of the target user; the collection area of ​​the preprocessed image is smaller than the area to be detected; the preprocessed image is an image captured by moving the collection window with the first step length and with a first preset period; If there is an abnormal area in the image, a collection instruction is sent to the collection end according to the abnormal area; So that the acquisition end acquires the image of the abnormal area as the target abnormal image according to the acquisition instruction; Performing image integration on all collected pre-processed images to obtain an overall oral image of the target user; Acquire all target abnormal images and mark the abnormal areas in the overall oral cavity image to obtain a final image; Receiving a preprocessed image of the area to be detected collected in real time by the collection end, and identifying an abnormal image area in the preprocessed image includes: Performing HLS color space transformation and XYZ color space transformation on the preprocessed image to obtain a first color image and a second color image; Performing anomaly detection on the first color image and the second color image respectively by using the YOLO algorithm; If both the first color image and the second color image have image anomalies, superimposing the abnormal areas in the first color image and the second color image to obtain a target abnormal area; The abnormal region in the preprocessed image is marked according to the target abnormal region to obtain the image abnormal region.

3. The method for acquiring oral environment microscopic images without blind spots according to claim 2, characterized in that: The acquisition instruction includes a first coordinate, a second step-length moving acquisition window and a second preset period; the first coordinate is the coordinate of the abnormal image area; the second step-length moving acquisition window is smaller than the first step-length moving acquisition window; the second preset period is smaller than the first preset period.

4. The method for acquiring oral environment microscopic images without blind spots according to claim 2, characterized in that: The image integration of all collected pre-processed images to obtain the overall oral image of the target user includes: For each preprocessed image, a feature point set on the preprocessed image is obtained by a preset algorithm; The feature point sets in the adjacent pre-processed images are matched by the FLANN algorithm; If the matching success rate of all feature points in the feature point set is greater than a preset threshold, the overlapping areas in the adjacent pre-processed images are fused according to the successfully matched feature points to obtain an initial local oral cavity image; The overlapping areas of all initial partial oral images are superimposed to obtain the overall oral image.

5. The method for acquiring oral environment microscopic images without blind spots according to claim 4, characterized in that: The feature point matching of the feature point set in the adjacent pre-processed image by the FLANN algorithm specifically includes: For each feature point, the adjacent feature points of the feature point are obtained to obtain an adjacent feature set, and the feature points in the adjacent feature set are sorted in ascending order based on the distances between the feature point and all adjacent feature points to obtain a target adjacent feature set; Taking the feature point as a vertex, sequentially calculating the angles formed by two adjacent feature points in the target adjacent feature set to obtain a target angle feature set; Constructing a FLANN index according to the number of the target adjacent feature sets, the distance between each adjacent feature point in the target adjacent feature set and the feature point, and the target angle feature set; For each feature point, the FLANN algorithm is used to perform feature point matching; the feature point matching is to first determine the matching point that meets the requirements through the number of adjacent feature sets, and then match the target adjacent feature set and the target angle feature set.

6. A method for acquiring oral environment microscopic images without blind spots, characterized in that: The method applied to the system for acquiring oral environment microscopic images without blind spots as described in claim 1, wherein the method is applied to the acquisition end and comprises: The pre-processed image of the area to be detected is collected in real time, and the pre-processed image is sent to the image processing end; so that the image processing end identifies the abnormal image area in the pre-processed image, and if there is an abnormal image area, a collection instruction is sent to the collection end according to the abnormal area; the area to be detected is the oral cavity of the target user; the collection area of ​​the pre-processed image is smaller than the area to be detected; the pre-processed image is an image captured by moving the collection window with the first step length and with the first preset period; According to the acquisition instruction, an image of the abnormal area is acquired as a target abnormal image; so that the image processing end integrates all the acquired pre-processed images to obtain an overall oral image of the target user, and annotates the abnormal area in the overall oral image according to all the target abnormal images to obtain a final image; The image processing end identifying the abnormal image area in the pre-processed image includes: Performing HLS color space transformation and XYZ color space transformation on the preprocessed image to obtain a first color image and a second color image; Performing anomaly detection on the first color image and the second color image respectively by using the YOLO algorithm; If both the first color image and the second color image have image anomalies, superimposing the abnormal areas in the first color image and the second color image to obtain a target abnormal area; The abnormal region in the preprocessed image is marked according to the target abnormal region to obtain the image abnormal region.

7. The method for acquiring oral environment microscopic images without blind spots according to claim 6, characterized in that: The acquisition instruction includes a first coordinate, a second step length moving acquisition window and a second preset period; the first coordinate is the coordinate of the abnormal area of ​​the image; the second step length moving acquisition window is smaller than the first step length moving acquisition window; the second preset period is smaller than the first preset period; and acquiring an image of the abnormal area as a target abnormal image according to the acquisition instruction includes: Determine a second coordinate of the first coordinate in the current acquisition window, adjust the focal length of the camera according to the second coordinate, move the acquisition window with a second step length and a second preset period to acquire an image of the target abnormal area to obtain a target abnormal image set; The target abnormal area is the position corresponding to the abnormal image area in the area to be detected; When the image acquired by moving the acquisition window with the second step length and the second preset period does not contain the abnormal area, the image of the to-be-detected area is acquired again with the first step length and the first preset period until the target mark appears in the acquired image, and the image acquisition is terminated; The target mark is set for the first acquisition of the area to be detected; The acquisition end, when the image acquired by moving the acquisition window with the second step size and the second preset period does not contain the abnormal area, re-acquires the image of the area to be detected by moving the acquisition window with the first step size and the first preset period until the area to be detected reaches the termination condition.

8. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the method for acquiring a microscopic image of an oral environment without blind spots as described in any one of claims 2 to 5 when executing the executable instructions stored in the memory.

9. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the method for acquiring a microscopic image of an oral environment without blind spots as described in any one of claims 6 to 7 when executing the executable instructions stored in the memory.

Images