An image processing method, apparatus, device, and storage medium

Abstract

The present disclosure provides an image processing method, device and equipment and storage medium, relates to the technical field of image processing, in particular to the technical field of computer vision. The specific implementation scheme is: obtaining a plurality of to-be-processed images; identifying teeth in the to-be-processed images to obtain an identification result; obtaining tooth center point coordinates in the to-be-processed images according to the identification result; registering tooth arrangement in the to-be-processed images according to the tooth center point coordinates to obtain a registered image; and splicing a plurality of registered images to obtain an image processing result.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, in particular to an image processing method and device, equipment and a storage medium in the technical field of computer vision. BACKGROUND

[0002] In recent years, analysis and research on dental endoscopes and dental images captured by the dental endoscopes have become a research hotspot in dental medical imaging, and dental endoscopes have been rapidly popularized and widely applied in clinical and daily life. Compared with the computed tomography (CT) imaging technology commonly used in dentistry, the dental endoscope has the following advantages:

[0003] (1) No radioactivity, continuous and multiple image acquisition is possible;

[0004] (2) Good real-time performance, easy to operate, and can be used as a home oral health device.

[0005] However, due to differences in shooting techniques of operators and other reasons, the images captured by the dental endoscope are quite different and not standardized. In addition, due to the limitation of the collection distance, the images captured by the dental endoscope cannot cover all the teeth, so it is necessary to splice multiple dental endoscope images to obtain a panoramic image of the oral cavity. In the prior art, the dental endoscope images are generally standardized and spliced manually by human. SUMMARY

[0006] The present disclosure provides an image processing method, device, equipment and storage medium with higher efficiency.

[0007] According to an aspect of the present disclosure, an image processing method is provided, comprising: acquiring a plurality of images to be processed; identifying the teeth in the images to be processed to obtain an identification result; obtaining the tooth center point coordinates in the images to be processed according to the identification result; registering the tooth arrangement in the images to be processed according to the tooth center point coordinates to obtain a registration image; and splicing a plurality of registration images to obtain an image processing result.

[0008] According to another aspect of the present disclosure, an image processing device is provided, comprising: an acquisition module configured to acquire a plurality of images to be processed; an identification module configured to identify the teeth in the images to be processed to obtain an identification result; a calculation module configured to obtain the tooth center point coordinates in the images to be processed according to the identification result; a registration module configured to register the tooth arrangement in the images to be processed according to the tooth center point coordinates to obtain a registration image; and a splicing module configured to splice a plurality of registration images to obtain an image processing result.

[0009] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the present disclosure.

[0010] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to perform the method of the present disclosure.

[0011] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method of the present disclosure.

[0012] According to another aspect of the present disclosure, an intraoral camera system is provided, comprising: an image acquisition device, at least one processor connected with the image acquisition device; and a memory communicatively connected with the at least one processor; wherein the image acquisition device is configured to acquire a plurality of images to be processed; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the present disclosure.

[0013] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings are used to better understand the present scheme, and do not limit the present disclosure. Among them:

[0015] Figure 1 is a flowchart of an image processing method according to the first embodiment of the present disclosure;

[0016] Figure 2 is a first scene diagram of an image processing method according to the first embodiment of the present disclosure;

[0017] Figure 3 is a second scene diagram of an image processing method according to the first embodiment of the present disclosure;

[0018] Figure 4 is a third scene diagram of an image processing method according to the first embodiment of the present disclosure;

[0019] Figure 5 is a flowchart of an image processing method according to the fourth embodiment of the present disclosure;

[0020] Figure 6 is a scene schematic diagram of an image processing method according to a fourth embodiment of the present disclosure;

[0021] Figure 7 is a first scene schematic diagram of an image processing method according to a fifth embodiment of the present disclosure;

[0022] Figure 8 is a second scene schematic diagram of an image processing method according to the fifth embodiment of the present disclosure;

[0023] Figure 9 is a first scene schematic diagram of an image processing method according to a sixth embodiment of the present disclosure;

[0024] Figure 10 is a second scene schematic diagram of an image processing method according to the sixth embodiment of the present disclosure;

[0025] Figure 11 is an application flow schematic diagram of an image processing method according to the present disclosure;

[0026] Figure 12 is a structure schematic diagram of an image processing device according to a seventh embodiment of the present disclosure;

[0027] Figure 13 is a block diagram of an electronic device for implementing an image processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, and should be considered as merely exemplary. Thus, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0029] In recent years, oral endoscopes have been rapidly popularized and widely used in clinical and daily life, but due to differences in shooting techniques of the operators and other reasons, the images captured by the oral endoscopes differ greatly and are not standardized enough. In addition, due to the limitation of the collection distance, the images captured by the oral endoscopes cannot cover all the teeth, which requires the splicing of multiple oral endoscope images. However, the similarity between the oral endoscope images is very high, and the existing image splicing methods cannot accurately splice multiple oral endoscope images.

[0030] Figure 1 is a flow schematic diagram of an image processing method according to a first embodiment of the present disclosure, as Figure 1As shown, the method mainly includes:

[0031] In step S101, a plurality of to-be-processed images are acquired.

[0032] In this embodiment, a plurality of to-be-processed images need to be acquired first. The to-be-processed images can be intraoral camera images. The intraoral camera images can clearly and intuitively reflect the state of the inside of the patient's mouth. However, when the intraoral camera is used to collect images of the inside of the patient's mouth, the intraoral camera images cannot cover all the teeth because the collection distance is very close. Generally, only 2 to 3 teeth can be covered. In order to observe all the teeth in the patient's mouth, a plurality of intraoral camera images of the inside of the same patient's mouth, i.e., a plurality of to-be-processed images, need to be collected. Specifically, scaling, cropping, and other operations can be performed on the plurality of to-be-processed images to make the sizes of the plurality of to-be-processed images consistent.

[0033] In step S102, the teeth in the to-be-processed images are identified to obtain an identification result.

[0034] In step S103, the center point coordinates of the teeth in the to-be-processed images are obtained according to the identification result.

[0035] In this embodiment, after the plurality of to-be-processed images are acquired, the center point coordinates of the teeth in the to-be-processed images need to be calculated. Specifically, the teeth in the to-be-processed images can be identified according to a deep learning model first to obtain an identification result, and then the center point coordinates of the teeth in the to-be-processed images are calculated according to the identification result.

[0036] In an implementable manner, a training sample set can be acquired first. The training sample set includes sample intraoral camera images that have been labeled with teeth. Then, the deep learning model is trained using the training sample set, and the teeth in the to-be-processed images are identified according to the trained deep learning model to obtain an identification result.

[0037] In an implementable manner, the deep learning model can be a target detection model, such as a Faster-RCNN model, a YOLO model, etc. The target detection model regards each tooth as a target, and each tooth is framed in the form of an external rectangle. Therefore, the center point coordinates of the external rectangle of the tooth in the identification result can be calculated directly, and the center point coordinates of the external rectangle of the tooth can be taken as the center point coordinates of the tooth.

[0038] Figure 2 is a first scene schematic diagram of an image processing method according to the first embodiment of the present disclosure, as Figure 2As shown, taking the top-left vertex of the image to be processed as the origin, the recognition result shows that the bounding rectangle of a certain tooth in the image to be processed is rectangle ABCD, with coordinates (x, y, width, height), where (x, y) are the coordinates of point A, width is the width of the bounding rectangle, and height is the height of the bounding rectangle. Then, the coordinates of the center point E of the bounding rectangle are... The coordinates of the center point E of the circumscribed rectangle can be used as the coordinates of the center point of the tooth.

[0039] In one implementation, the deep learning model can be an image segmentation model, such as the level set method or watershed algorithm in unsupervised segmentation methods; the fully convolutional network (FCN) model or UNet model in semantic segmentation models; or the Mask-RCNN model in instance segmentation models. The image segmentation model can extract the dental arch contours from the image to be processed, and then perform an erosion operation on the connected dental arch regions, that is, shrink the tooth boundaries inward to obtain several discontinuous tooth regions. Afterward, the coordinates of the center point of each tooth region can be calculated, and the coordinates of the center point of the tooth region can be used as the center point coordinates of that tooth. For example, Figure 3 This is a schematic diagram of a second scene according to an image processing method based on the first embodiment of this disclosure, such as... Figure 3 As shown, the image segmentation model divides the teeth in the image to be processed into multiple unconnected tooth regions, where point F is the coordinate of the center point of a tooth in the image to be processed. Specifically, the centroid coordinates of each tooth region can be calculated using existing methods, and the centroid coordinates of the tooth can be used as the coordinates of the center point of the tooth region; alternatively, a circumscribed rectangle can be added to each tooth region, and the coordinates of the center point of the circumscribed rectangle can be used as the coordinates of the center point of the tooth region.

[0040] Step S104: Based on the coordinates of the tooth center point, register the tooth arrangement in the image to be processed to obtain a registered image.

[0041] In this embodiment, after obtaining the coordinates of the tooth center points in the image to be processed, it is also necessary to register the tooth arrangement in the image to be processed based on the tooth center point coordinates to obtain a registered image. Specifically, due to differences in device posture during image acquisition, the orientation of the line connecting the tooth center points in each image to be processed may be inconsistent, that is, the tooth arrangement may be inconsistent. Therefore, before stitching, it is necessary to adjust the tooth arrangement to be arranged along the horizontal line of the center of the image to be processed, that is, to register the tooth arrangement in the image to be processed.

[0042] In an embodiment, for the straight-line dental arrangement image of the posterior teeth, canine teeth region, etc., the arrangement of the center point coordinates of the teeth in the image is approximately a straight line, but there is an angle and an intercept between the straight line and the center horizontal line of the image to be processed. Therefore, the position of the image to be processed as a whole needs to be transformed, for example, the image to be processed as a whole is rotated and translated with the center point of the image to be processed as the center, so that the straight line of the dental arrangement of the image to be processed coincides with the center horizontal line of the image to be processed, and a registration image is obtained.

[0043] In an embodiment, for the curved dental arrangement of the front teeth, due to the significant difference between the distance of the teeth and the intraoral endoscope, the arrangement of the center point coordinates of the teeth in the image is approximately a parabola. The distance of the points on the parabola to the center horizontal line of the image to be processed can be used to move the pixels in the image to be processed, so that the pixels corresponding to the center point coordinates of the teeth in the image to be processed are moved to the center horizontal line of the image to be processed, and a registration image is obtained.

[0044] In step S105, the registration images are spliced to obtain an image processing result.

[0045] In this embodiment, after the dental arrangement in the image to be processed is registered to obtain a registration image, the registration images need to be spliced to obtain an image processing result. Specifically, the dental arrangement of the registration images is arranged along the center horizontal line of the image to be processed, so that the splicing of the registration images can be completed by directly translating the registration images in the horizontal direction.

[0046] In an embodiment, when multiple images to be processed inside the patient's mouth are collected, the spatial positions of each image to be processed inside the mouth can be labeled. When the registration images are spliced, the multiple registration images can be spliced according to the labeled spatial positions. Figure 4 is a third scene diagram of an image processing method according to the first embodiment of the present disclosure, Figure 4 is a panoramic image of the patient's mouth obtained after the multiple registration images are spliced, that is, an image processing result.

[0047] In the first embodiment of the present disclosure, a plurality of to-be-processed images are first acquired, then teeth in the to-be-processed images are recognized to obtain a recognition result, and according to the recognition result, tooth center point coordinates in the to-be-processed images are obtained, and finally, according to the tooth center point coordinates, the arrangement of the dentition in the to-be-processed images is registered to obtain a registered image, and the plurality of registered images are spliced to obtain an image processing result. In this embodiment, according to the tooth center point coordinates, the arrangement of the dentition in the to-be-processed images is adjusted to a form of arranging along the center horizontal line of the to-be-processed image, which is equivalent to normalizing the to-be-processed image, and splicing the registered image obtained after normalization can complete accurate splicing of the to-be-processed image, solving the problems of consuming manpower and time and low accuracy in normalizing and splicing the intraoral endoscope image manually.

[0048] In the second embodiment of the present disclosure, the recognition result includes position information of a tooth circumscribed rectangle in the to-be-processed image, and according to the recognition result, the tooth center point coordinates in the to-be-processed image are obtained, including: according to the position information of the tooth circumscribed rectangle in the to-be-processed image, the tooth center point coordinates in the to-be-processed image are calculated.

[0049] In this embodiment, whether the deep learning model is a target detection model or an image segmentation model, the position information of each tooth circumscribed rectangle can be obtained according to the recognition result, for example, the position information can be (x, y, width, height), where (x, y) is the coordinate of the upper left corner of the circumscribed rectangle, width is the width of the circumscribed rectangle, and height is the height of the circumscribed rectangle. Therefore, according to the position information of each tooth circumscribed rectangle, the center point coordinates of each tooth circumscribed rectangle can be calculated, and the center point coordinates of the tooth circumscribed rectangle can be taken as the tooth center point coordinates. Specifically, the center point coordinates of the circumscribed rectangle can be

[0050] In the third embodiment of the present disclosure, step S104 mainly includes:

[0051] According to the tooth center point coordinates, the arrangement of the dentition in the to-be-processed image is linearly fitted to obtain a first fitting result, and according to the first fitting result, the arrangement of the dentition in the to-be-processed image is registered to obtain a registered image.

[0052] In this embodiment, after the tooth center point coordinates are obtained, the arrangement of the dentition in the to-be-processed image is linearly fitted according to the tooth center point coordinates to obtain a first fitting result, and then the arrangement of the dentition in the to-be-processed image is registered according to the first fitting result. Specifically, the first fitting result fitted from the tooth center point coordinates can reflect the arrangement state of the dentition in the to-be-processed image, so the arrangement of the dentition in the to-be-processed image can be registered according to the first fitting result, that is, the arrangement of the dentition is adjusted to a form of arranging along the center horizontal line of the to-be-processed image.

[0053] In an implementable manner, different linear fitting methods can be used for the linear fitting of the tooth arrangement in the to-be-processed image according to different tooth regions in the to-be-processed image. For example, for a linear tooth arrangement image of a posterior tooth region or a canine region, the arrangement of the center point coordinates of the teeth in the image is approximately a straight line, and thus a linear equation y = f(x) = kx + b can be obtained by using the least square method, where k represents the slope of the straight line, and b represents the intercept of the straight line. Specifically, the linear equation can be fitted according to the following formula: where y i is the guessed true value, L is the total error, and the parameters k and b are solved when L is the smallest. For a curved tooth arrangement such as a front tooth arrangement, the arrangement of the center point coordinates of the teeth in the image is approximately a parabola, and thus a parabolic equation y = f(x) = ax 2 + bx + c can be fitted by using the least square method, where a, b, and c are the quadratic coefficients of the parabolic equation.

[0054] In the third embodiment of the present disclosure, the first fitting result obtained according to the center point coordinates of the teeth can reflect the arrangement state of the tooth arrangement in the to-be-processed image, and the tooth arrangement in the to-be-processed image can be registered according to the first fitting result, so as to facilitate the subsequent splicing of the registered image.

[0055] Figure 5 is a flowchart of an image processing method according to the fourth embodiment of the present disclosure, as shown in Figure 5 the linear fitting includes linear fitting, and the tooth arrangement in the to-be-processed image is registered according to the first fitting result to obtain a registered image, including:

[0056] In step S201, an angle between the straight line corresponding to the tooth arrangement and the center horizontal line of the to-be-processed image is calculated according to the first fitting result to obtain a first calculation result.

[0057] In step S202, a position transformation is performed on the to-be-processed image according to the first calculation result to obtain an initial image.

[0058] In the present embodiment, if the to-be-processed image is a linear tooth arrangement image of a posterior tooth region or a canine region, the linear fitting is linear fitting, and the angle between the straight line corresponding to the tooth arrangement and the center horizontal line of the to-be-processed image can be calculated according to the first fitting result to obtain a first calculation result, and then a position transformation is performed on the to-be-processed image according to the first calculation result to obtain an initial image. Specifically, the position transformation can be rotation.

[0059] In an implementation, if the first fitting result is y = f(x) = kx + b, the angle between the straight line and the horizontal line of the center of the image to be processed can be calculated according to the following formula: where k is the slope of the straight line, and then the image to be processed is rotated by-angle degrees with the center point of the image to be processed as the center, so that the fitting straight line is parallel to the horizontal line of the center of the image to be processed, where-angle represents the operation of rotating the image to be processed, and is always opposite to the angle of the current dentition arrangement of the image to be processed.

[0060] In step S203, linear fitting is performed on the dentition arrangement in the initial image to obtain a second fitting result.

[0061] In step S204, position transformation is performed on the initial image according to the second fitting result to obtain a registration image.

[0062] In the embodiment, after the position transformation is performed on the image to be processed, although the fitting straight line is parallel to the horizontal line of the center of the image to be processed, the fitting straight line does not coincide with the horizontal line of the center of the image to be processed, so linear fitting is further performed on the dentition arrangement in the initial image to obtain a second fitting result, and then position transformation is performed on the initial image according to the second fitting result to obtain a registration image. Specifically, when the position transformation is performed on the image to be processed, the coordinates of the tooth center points in the image to be processed are also subjected to the same position transformation, and then linear fitting is performed on the dentition arrangement in the initial image according to the coordinates of the tooth center points subjected to the position transformation, so as to obtain the second fitting result.

[0063] In an implementation, the second fitting result is similar to the first fitting result, but the slope k' = 0 and the intercept is b', and the initial image can be translated along the vertical direction according to the intercept b' so that the fitting straight line coincides with the horizontal line of the center of the image to be processed, thereby obtaining the registration image. Figure 6 is a scene schematic diagram of an image processing method according to the fourth embodiment of the present disclosure, as shown in Figure 6 6a is an image to be processed, and 6b is a registration image corresponding to 6a.

[0064] In the fifth embodiment of the present disclosure, the linear fitting includes curve fitting, the registration of the dentition arrangement in the image to be processed according to the first fitting result includes: calculating the vertical distance from the point on the curve corresponding to the dentition arrangement to the horizontal line of the center of the image to be processed according to the first fitting result to obtain a second calculation result; and performing position transformation on the pixels in the image to be processed according to the second calculation result to obtain a registration image.

[0065] In this embodiment, if the image to be processed is a curved dental arch image such as the incisor row, then the linear fitting is replaced by curve fitting, and the first fitting result is a parabolic equation. Based on the first fitting result, the vertical distance from the point on the curve corresponding to the dental arch arrangement to the horizontal line of the center of the image to be processed is calculated to obtain the second calculation result. Then, based on the second calculation result, the position transformation of the pixels in the image to be processed is performed to obtain the registered image.

[0066] In one possible implementation, if the shape of the image to be processed is (W, H), that is, the width of the image to be processed is W and the height is H, then its center horizontal line can be represented as... And the first fitting result is y = f(x) = ax 2 +bx+c, for any point on the parabola, the vertical distance y from the point on the curve corresponding to the tooth arrangement to the horizontal line of the center of the image to be processed can be calculated using the following formula. diff (x): Then the pixel at coordinates (x, y) in the image to be processed is moved to (x, y+y). diff (x)) at.

[0067] Figure 7 This is a schematic diagram of a first scene according to a fifth embodiment of the present disclosure of an image processing method, as shown below. Figure 7 As shown, if the shape of the image to be processed is (W, H), the first fitting result of the image to be processed is curve b, and the horizontal center line of the image to be processed is straight line a. For a point G on curve b, whose coordinates are (x, y), it needs to be moved vertically to straight line a, that is, point G needs to be moved to point J. The vertical distance between point G and straight line a is... Then the coordinates of point J are (x, y+GJ), which is equivalent to moving point G upward by the length of line segment GJ. For all points in the image to be processed with a horizontal coordinate of x, perform the same movement operation as point G. Figure 8 This is a schematic diagram of a second scene according to a fifth embodiment of the present disclosure of an image processing method, as shown below. Figure 8 As shown, 8a is the image to be processed, and 8b is the registration image corresponding to 8a.

[0068] In the fourth and fifth embodiments of this disclosure, based on the first fitting result, straight tooth rows and curved tooth rows in the image to be processed are registered in different ways to improve the registration accuracy and facilitate subsequent stitching of the registered images.

[0069] In the sixth embodiment of this disclosure, step S105 mainly includes: calculating the overlap of spatially adjacent registered images; and stitching multiple registered images together according to the overlap to obtain an image processing result.

[0070] In the embodiment, firstly, the overlap amount of the spatially adjacent registered images needs to be calculated, and then the registered images are stitched according to the overlap amount, so as to obtain the image processing result, wherein the overlap amount represents the relative translation amount between the spatially adjacent registered images.

[0071] In an implementable manner, when the plurality of to-be-processed images inside the oral cavity of the patient are collected, the spatial positions of the to-be-processed images inside the oral cavity can be labeled, that is, the spatial positions of the plurality of to-be-processed images are known, and the spatial positions of the plurality of registered images are also known, so that the other registered image spatially adjacent to each registered image can be known.

[0072] In an implementable manner, the overlap amount can be calculated according to the following formula: move x =e A +e B , wherein move x is the overlap amount, e A and e B are distances between the stitching side and the stitching side boundary frame of the registered image on the center horizontal line of the registered image. Figure 9 is a first scene schematic diagram of an image processing method according to the sixth embodiment of the present disclosure, as shown in Figure 9 , the center horizontal line of the registered image A is a straight line c, if the registered image A left side needs to be stitched with the spatially adjacent registered image B, the intersection point of the straight line c and the stitching side boundary frame is K, the intersection point of the straight line c and the stitching side is L, and e A is the length of the line segment KL, and e B corresponding to the registered image B is calculated in the same way, so that the overlap amount move x can be calculated, then the registered image A can be moved to the left by move x , or the registered image B can be moved to the right by move x , so as to complete the stitching of the registered image A and the registered image B, and after the stitching of all the registered images is completed, the panoramic image of the oral cavity of the patient can be obtained, that is, the image processing result.

[0073] Figure 10 is a second scene schematic diagram of an image processing method according to the sixth embodiment of the present disclosure, as shown in Figure 10 , 10a is two spatially adjacent to-be-processed images, after the two spatially adjacent to-be-processed images in 10a are registered, two spatially adjacent registered images are obtained, and after the two spatially adjacent registered images are stitched, 10b can be obtained.

[0074] In the sixth embodiment of the present disclosure, the plurality of normalized registered images are spliced according to the overlap amount, and the accurate splicing of the to-be-processed images can be completed, and the problem that the existing image splicing method cannot accurately splice the plurality of intraoral camera images due to the high similarity between the intraoral camera images is solved.

[0075] Figure 11 is an application flow diagram of an image processing method according to the present disclosure, as shown in Figure 11 , wherein 11a is a flow diagram of normalizing the intraoral camera images, and 11b is a flow diagram of splicing the normalized intraoral camera images. As shown in 11a, when normalizing the intraoral camera images, first, input a plurality of intraoral camera images inside the oral cavity of the same patient as to-be-processed images, then extract the tooth center points of the intraoral camera images to obtain the tooth center point coordinates, and register the tooth arrangement in the intraoral camera according to the tooth center point coordinates. The registration process includes equation fitting according to the tooth center point coordinates, i.e. linear fitting, then adjusting the intraoral camera images according to the fitting results, thereby obtaining the normalized intraoral camera images, which are the registered images in step S104, and outputting the normalized intraoral camera images. As shown in 11b, when splicing the normalized intraoral camera images, first, input a plurality of adjacent normalized intraoral camera images inside the oral cavity of the same patient, then read in two adjacent normalized intraoral camera images, and calculate the overlap amount move x between the two adjacent normalized intraoral camera images read in, and splice the two adjacent normalized intraoral camera images read in according to the overlap amount move x , to obtain the splicing result. Then, read in the next adjacent image, and splice the next adjacent image with the splicing result, until all the input normalized intraoral camera images are spliced to obtain a panoramic splicing image, i.e. the image processing result, and output the image processing result. It should be emphasized that Figure 11 the application flow of the image processing method shown in can be completed by an intraoral camera system provided by the present disclosure.

[0076] Figure 12 is a structural diagram of an image processing device according to the seventh embodiment of the present disclosure, as shown in Figure 12 , the device mainly includes:

[0077] The acquisition module 10 is used to acquire multiple images to be processed; the recognition module 11 is used to recognize the teeth in the images to be processed and obtain the recognition result; the calculation module 12 is used to obtain the coordinates of the center point of the teeth in the images to be processed based on the recognition result; the registration module 13 is used to register the tooth row arrangement in the images to be processed based on the coordinates of the center point of the teeth and obtain the registered image; the stitching module 14 is used to stitch the multiple registered images to obtain the image processing result.

[0078] In one embodiment, the calculation module 12 is further configured to calculate the coordinates of the center point of the tooth in the image to be processed based on the position information of the bounding rectangle of the tooth in the image to be processed.

[0079] In one embodiment, the registration module 13 mainly includes: a fitting submodule, used to perform linear fitting on the tooth row arrangement in the image to be processed based on the tooth center point coordinates to obtain a first fitting result; and a registration submodule, used to register the tooth row arrangement in the image to be processed based on the first fitting result to obtain a registered image.

[0080] In one embodiment, linear fitting includes straight line fitting, and the registration submodule mainly includes: a first calculation unit, used to calculate the angle between the straight line corresponding to the dental arch arrangement and the horizontal line of the center of the image to be processed based on the first fitting result, to obtain a first calculation result; a first transformation unit, used to perform position transformation on the image to be processed based on the first calculation result, to obtain an initial image; a fitting unit, used to perform linear fitting on the dental arch arrangement in the initial image, to obtain a second fitting result; and a second transformation unit, used to perform position transformation on the initial image based on the second fitting result, to obtain a registered image.

[0081] In one embodiment, linear fitting includes curve fitting, and the registration submodule mainly includes: a second calculation unit, used to calculate the vertical distance from the point on the curve corresponding to the tooth arrangement to the horizontal line of the center of the image to be processed according to the first fitting result, and obtain a second calculation result; and a third transformation unit, used to perform position transformation on the pixels in the image to be processed according to the second calculation result, and obtain a registered image.

[0082] In one embodiment, the stitching module 14 mainly includes: a calculation submodule for calculating the overlap of spatially adjacent registered images; and a stitching submodule for stitching multiple registered images according to the overlap to obtain an image processing result.

[0083] In one possible implementation, the calculation submodule calculates the overlap amount according to the following formula: move x =e A +e B , among which, move x e is the amount of overlap. A and eB respectively, are distances between a stitching side of the registration image and a stitching side boundary box on a horizontal line of the center of the registration image.

[0084] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information are in line with relevant laws and regulations and do not violate public order and good customs.

[0085] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0086] Figure 13 A schematic block diagram of an example electronic device 1300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0087] As shown in Figure 13 The electronic device 1300 includes a computing unit 1301 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 1302 or a computer program loaded from a storage unit 1308 into a random access memory (RAM) 1303. Various programs and data required for the operation of the electronic device 1300 can also be stored in the RAM 1303. The computing unit 1301, the ROM 1302, and the RAM 1303 are connected to each other through a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0088] Various components in the electronic device 1300 are connected to the I / O interface 1305, including an input unit 1306, such as a keyboard, a mouse, and the like; an output unit 1307, such as various types of displays, speakers, and the like; a storage unit 1308, such as a magnetic disk, an optical disk, and the like; and a communication unit 1309, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 1309 allows the electronic device 1300 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0089] The computing unit 1301 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like. The computing unit 1301 performs various methods and processes described above, such as an image processing method. For example, in some embodiments, an image processing method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1308. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1300 via the ROM 1302 and / or the communication unit 1309. When the computer program is loaded onto the RAM 1303 and executed by the computing unit 1301, one or more steps of an image processing method described above can be performed. Alternatively, in other embodiments, the computing unit 1301 can be configured to perform an image processing method by any other suitable means, such as by means of firmware.

[0090] The disclosure embodiments also provide an intraoral camera system, wherein the intraoral camera system comprises: an image acquisition device, at least one processor connected to the image acquisition device; and a memory in communication connection with the at least one processor; wherein the image acquisition device is configured to acquire a plurality of images to be processed; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform an image processing method.

[0091] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0092] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0093] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0094] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0095] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0096] The computer system can include clients and servers. This relationship can be. The servers are typically remote from the clients with the interactions between them occurring over a communication network. The relationship between a client and a server is one of client-server. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0097] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in series, or executed in different orders, as long as the desired results of the technical solutions of the present disclosure are achieved, and the present disclosure is not limited herein.

[0098] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. An image processing method, comprising: Acquire multiple images to be processed; The teeth in the image to be processed are identified, and the identification results are obtained; Based on the recognition results, the coordinates of the center point of the tooth in the image to be processed are obtained; Based on the coordinates of the tooth center point, the tooth arrangement in the image to be processed is registered to obtain a registered image; The multiple registered images are stitched together to obtain the image processing result; The step of stitching together multiple registered images to obtain an image processing result includes: Calculate the overlap of the spatially adjacent registered images; Based on the overlap amount, multiple registered images are stitched together to obtain the image processing result; The overlap amount is calculated according to the following formula: move x =and A +e B , Among them, move x e is the amount of overlap. A and e B These are the distances between the stitching edge and the stitching side bounding box of the registered image, respectively, on the horizontal line at the center of the registered image.

2. The method according to claim 1, wherein, The recognition result includes the position information of the circumscribed rectangle of the teeth in the image to be processed. Obtaining the coordinates of the center point of the teeth in the image to be processed based on the recognition result includes: Based on the position information of the circumscribed rectangle of the teeth in the image to be processed, the coordinates of the center point of the teeth in the image to be processed are calculated.

3. The method according to claim 1, wherein, The step of registering the tooth arrangement in the image to be processed based on the coordinates of the tooth center point to obtain a registered image includes: Based on the coordinates of the tooth center point, a linear fit is performed on the tooth row arrangement in the image to be processed to obtain a first fitting result; Based on the first fitting result, the tooth arrangement in the image to be processed is registered to obtain the registered image.

4. The method according to claim 3, wherein, The linear fitting includes straight line fitting, and the registration of the tooth arrangement in the image to be processed based on the first fitting result to obtain the registered image includes: Based on the first fitting result, the angle between the straight line corresponding to the tooth row arrangement and the horizontal line at the center of the image to be processed is calculated to obtain the first calculation result; Based on the first calculation result, the image to be processed is transformed to obtain an initial image; A second fitting result is obtained by linearly fitting the tooth arrangement in the initial image; Based on the second fitting result, the initial image is transformed to obtain the registered image.

5. The method according to claim 3, wherein, The linear fitting includes curve fitting, and the registration of the dentition arrangement in the image to be processed based on the first fitting result to obtain the registered image includes: Based on the first fitting result, the vertical distance from the point on the curve corresponding to the tooth row arrangement to the horizontal line of the center of the image to be processed is calculated to obtain the second calculation result; Based on the second calculation result, the position transformation of the pixels in the image to be processed is performed to obtain the registration image.

6. An image processing apparatus, comprising: The acquisition module is used to acquire multiple images to be processed; The recognition module is used to identify teeth in the image to be processed and obtain recognition results; The calculation module is used to obtain the coordinates of the center point of the tooth in the image to be processed based on the recognition result; The registration module is used to register the tooth arrangement in the image to be processed according to the coordinates of the tooth center point to obtain a registered image; The stitching module is used to stitch together multiple registered images to obtain the image processing result; The splicing module includes: A calculation submodule is used to calculate the overlap of spatially adjacent registered images; The stitching submodule is used to stitch together multiple registered images according to the overlap amount to obtain the image processing result; The calculation submodule is also used to calculate the overlap amount according to the following formula: move x =e A +e B , among which, move x e is the amount of overlap. A and e B These are the distances between the stitching edge and the stitching side bounding box of the registered image, respectively, on the horizontal line at the center of the registered image.

7. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

9. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

10. An oral endoscope system, wherein, The oral endoscope system includes: an image acquisition device, at least one processor connected to the image acquisition device; and A memory communicatively connected to at least one of the processors; wherein, The image acquisition device is used to acquire multiple images to be processed; The memory stores instructions executable by at least one of the processors to enable the at least one processor to perform the method of any one of claims 1-5.

Images