Method and computer program product for batch synthesis of three-dimensional photos of patients in medical institutions

Abstract

This patent relates to a method and computer program product for batch synthesis of 3D images of patients in medical institutions. The method involves: acquiring a data record table; performing a synthesis step based on the data of the first patient in the data record table; inputting the data of a single patient recorded in the data record table into a 3D processing program; acquiring a set of images corresponding to the entered patient data; inputting the acquired set of images into the 3D data processing program; starting the processing process of the 3D data processing program to perform synthesis on the images in the set; acquiring the synthesis result output by the 3D data processing program and saving the result in a preset location; repeating the synthesis step for the data of the second patient in the data record table until all patient data in the data record table has been processed. This patent provides an automated image synthesis method capable of batch synthesis of 3D images of multiple patients.

Description

Technical Field

[0001] This patent relates to the field of medical image processing technology, and in particular to a method for batch synthesis of three-dimensional photographs of patients in medical institutions and a computer program product. Background Technology

[0002] Existing 3D data processing programs can synthesize 2D photographs of the same patient, combining 2D photographs from a photograph set into a 3D photograph.

[0003] In the medical field, 3D imaging technology has gradually become an indispensable tool in diagnosis, treatment, and scientific research. Especially in the management and application of patient photographs, 3D images (or stereoscopic images) can provide more detailed and comprehensive patient information, which is of great significance for doctors' diagnosis, surgical planning, and treatment effectiveness evaluation. Medical staff can operate on computers and use 3D processing programs to synthesize images.

[0004] In existing technologies, medical staff need to manually perform an operation each time they process a patient's photo: selecting the patient's 2D image and clicking the 3D composite button to create a 3D composite image. However, in practice, processing a large number of 3D patient photos requires medical staff to repeat this operation for each patient's photo set, which consumes a significant amount of their time and effort, making the process complex and time-consuming.

[0005] Therefore, it is particularly important to develop an efficient and accurate method for batch synthesis of 3D photos and a corresponding computer program product. Summary of the Invention

[0006] To solve, or at least partially solve, the aforementioned technical problems, this patent provides a method for batch synthesis of three-dimensional images of patients in medical institutions, comprising:

[0007] Retrieve the data record table;

[0008] Based on the data of the first patient in the data record table, perform the synthesis step, which includes:

[0009] Enter the data of a single patient recorded in the data record table into the three-dimensional processing program;

[0010] Based on the entered patient data, obtain the photo set corresponding to the entered patients;

[0011] The acquired photo set is entered into a 3D data processing program;

[0012] Start the processing of the 3D data processing program, enabling it to perform a composite operation on the photos in the photo set;

[0013] Acquire the synthesis results output by the 3D data processing program and save the synthesis results in a preset location;

[0014] For the data of the second patient in the data record table, repeat the synthesis step until all patients' data in the data record table have been processed.

[0015] Optionally, the step of obtaining a photo set corresponding to the entered patient data includes:

[0016] Retrieve the patient ID from the entered patient data;

[0017] The memory is indexed by number to find the photo set corresponding to the number;

[0018] The steps for inputting the acquired photo set into the 3D data processing program include:

[0019] The photos in the acquired photo set are sequentially entered into the 3D data processing program;

[0020] The process of initiating the 3D data processing program includes the following steps:

[0021] Select photos from the entered photo set and click the "Glue Images" button to start the processing.

[0022] Optionally, after the steps of acquiring the composite result output by the 3D data processing program and saving the composite result in a preset location,

[0023] If the synthesis result can be obtained, add a synthesis success mark to the corresponding patient data in the data record table;

[0024] If the synthesis result cannot be obtained, a synthesis failure mark is added to the corresponding patient data in the data record table.

[0025] Optionally, the photo collection may include at least:

[0026] A frontal photograph of the patient;

[0027] First lateral photograph of the patient;

[0028] The patient's second lateral view photograph, which, along with the first lateral view photograph, shows the patient's left and right lateral views, respectively.

[0029] Optionally, the step of obtaining a photo set corresponding to the entered patient data further includes:

[0030] Determine whether the frontal and second-side photos in the photo set contain defect marking areas;

[0031] If the determination is yes, then the frontal photo and the second side photo are corrected based on the first side photo.

[0032] Optionally, the photo collection may include at least:

[0033] First lateral photograph of the patient;

[0034] The step of obtaining a photo set corresponding to the entered patients based on the entered patient data also includes:

[0035] Determine whether a frontal photo and a second side photo of the patient can be obtained. The second side photo and the first side photo are photos of the left and right sides of the patient, respectively.

[0036] If the determination is yes, then obtain the patient's frontal photo and second side photo;

[0037] If the determination is no, the photo set is expanded based on the photos currently contained in the photo set until the 3D processing program can synthesize the photo set;

[0038] After the steps of acquiring the composite result output by the 3D data processing program and saving the composite result in a preset location, the process also includes:

[0039] Add auxiliary generation tags to the corresponding patients in the data record table to clarify that the patients' synthesis results are computer-aided synthesis results.

[0040] Optionally, the photo collection also includes:

[0041] The patient's second lateral photograph;

[0042] The steps to expand your photo collection include:

[0043] A frontal photograph of the patient is generated based on the first and second side photographs.

[0044] Optionally, the second side view photo is generated by mirroring the first side view photo.

[0045] Optionally, the photo collection also includes:

[0046] A frontal photograph of the patient;

[0047] The steps to expand your photo collection include:

[0048] Mirror the first side view photo to obtain a template for the second side view photo;

[0049] The difference between the second region and the first region of the frontal photo is obtained. The second region is the region corresponding to the second side photo, and the first region is the region corresponding to the first side photo.

[0050] Based on the template for correcting the distinguishing sub-regions, a second side view photo is generated, and a frontal photo after eliminating the distinguishing sub-regions is regenerated.

[0051] This patent also provides a calculator program product that stores a computer program. When the computer program is executed by a processor, it can implement the steps of the aforementioned method for batch synthesis of three-dimensional images of patients in medical institutions.

[0052] Compared with existing technologies, the technical advantages of this patent are as follows:

[0053] 1. The data recording table of this embodiment can record data information of multiple patients and obtain the patient's number through the patient's data information, so that the three-dimensional processing program can find the storage location of the patient's corresponding photo set by using the patient's number as an index.

[0054] 2. By using the processing of a 3D processing program to synthesize the patient's corresponding photo set, multiple 2D images in the photo set can be combined into a 3D image, which is simple for users to operate.

[0055] 3. In addition, after completing the compositing operation for one patient, the 3D processing program continues to perform the compositing operation for the next patient's photo set in the data record table until all patients in the data record table have completed the photo compositing operation, thus realizing the batch processing of multiple patient photo sets.

[0056] 4. The three-dimensional processing program can generate three-dimensional photos of several customers in a batch in a data record table with one click. Compared with traditional three-dimensional modeling methods, such as using modeling software such as 3DMax and AutoCAD or three-dimensional scanners, the method of batch synthesizing three-dimensional images from two-dimensional images is highly automated, lower in cost, faster, and greatly saves labor costs.

[0057] 5. The method features an automatic compensation algorithm for photos in the photo set. When photos in the set are insufficient to generate a complete 3D image due to defects, the existing photos in the set are expanded through methods such as mirroring and fusion until a complete 3D image can be synthesized. Through automatic repair and compensation, even if there are missing or incomplete photos in the photo set, they can still be automatically supplemented and expanded, improving the efficiency of batch synthesis and reducing generation errors. Attached Figure Description

[0058] To more clearly illustrate the embodiments of this patent, the relevant drawings will be briefly described below. It should be understood that the drawings described below are only for illustrating some embodiments of this patent, and those skilled in the art can obtain many other technical features and connections not mentioned herein based on these drawings.

[0059] Figure 1This is a flowchart of a method for batch synthesis of three-dimensional images of patients in a medical institution, according to an embodiment of this patent.

[0060] Figure 2 This is a flowchart of a method for performing a synthesis step based on patient data in a data record table, according to an embodiment of this patent.

[0061] Figure 3 This is a flowchart of a method for inputting data of a single patient into a three-dimensional processing program according to an embodiment of this patent.

[0062] Figure 4 This is a flowchart illustrating a specific method for obtaining the synthesis result output by a three-dimensional data processing program and saving the synthesis result in a preset location according to an embodiment of this patent.

[0063] Figure 5 This is a flowchart illustrating a specific method for determining whether a frontal or second side view photograph of a patient has been obtained, according to one embodiment of this patent. Detailed Implementation

[0064] The patent will now be described in detail with reference to the accompanying drawings.

[0065] Implementation Method 1

[0066] In existing technologies, acquiring and processing a large number of 3D patient photographs is a complex and time-consuming process, especially in the medical field. Synthesizing a patient's 2D image into a 3D image typically requires manual operation by the user. The user needs to select the patient's photo set, navigate to the corresponding photo set folder, select the photos, and click the synthesize button to create a 3D image of a single patient. When processing photos of multiple patients, the user needs to repeat the synthesis operation sequentially, resulting in low efficiency and high labor costs.

[0067] In view of this, the first embodiment of this patent proposes a method for batch synthesis of three-dimensional images of patients in medical institutions, see [link to relevant documentation]. Figure 1 and Figure 2 As shown, the synthesis method includes:

[0068] Retrieve the data record table;

[0069] Based on the data of the first patient in the data record table, perform the synthesis step, which includes:

[0070] Enter the data of a single patient recorded in the data record table into the three-dimensional processing program;

[0071] Based on the entered patient data, obtain the photo set corresponding to the entered patients;

[0072] The acquired photo set is entered into a 3D data processing program;

[0073] Start the processing of the 3D data processing program, enabling it to perform a composite operation on the photos in the photo set;

[0074] Acquire the synthesis results output by the 3D data processing program and save the synthesis results in a preset location;

[0075] For the data of the second patient in the data record table, repeat the synthesis step until all patients' data in the data record table have been processed.

[0076] In a specific scenario, a computer is equipped with the 3D processing program of this embodiment. This 3D processing program can generally execute the following steps in sequence:

[0077] S1. Obtain the data record table. This data record table records information about multiple patients. This information serves as an index, allowing the 3D processing program to locate the storage location of each patient's image set based on the index.

[0078] S2. Perform the compositing step based on the patient data in the data record table. In the compositing step, the 3D processing program first enters the data of a single patient recorded in the record table, then finds the storage location of the corresponding photo set based on the entered patient data, enters the photo set, performs the compositing operation, and outputs the compositing result.

[0079] S3. Repeat the image compositing step for all patients in sequence. After processing the data of the first patient in the data record table, repeat the compositing step for subsequent patients in sequence until all patients' data in the data record table has been processed.

[0080] Step S2 above can be further subdivided into steps S21 to S26 as follows:

[0081] S21. Input the data of a single patient recorded in the data record table into the 3D processing program. The data record table records information data of multiple patients. The 3D processing program first inputs the data of one patient and then searches for the data.

[0082] S22. Based on the entered patient data, obtain the photo set corresponding to the entered patient. The data entered in S21 serves as an index for the 3D processing program to locate the storage location of the patient's corresponding photo set.

[0083] S23. Input the acquired photo set into the 3D data processing program. After retrieving the storage location of the photo set based on the patient's data, input the patient's photo set into the 3D processing program.

[0084] S24. Start the processing process of the 3D data processing program, enabling the program to composite the photos in the photo set. The 3D processing program starts its processing process after the photo set is entered; executing this process allows for the compositing of the photos in the photo set.

[0085] S25. Obtain the composite result output by the 3D data processing program and save the composite result in a preset location. After the processing is completed, the photos in the photo set are composited into 3D photos, and the 3D processing program outputs the composite result, which is then saved in a preset location.

[0086] S26. Perform the compositing step based on the patient's data in the data record table. After completing steps S21 to S25 above, one compositing operation for a single patient is completed. After completing the compositing operation for the previous patient, the 3D processing program enters the information data for the next patient and begins repeating steps S21 to S25.

[0087] In one operation process, the user's specific operation flow can be as follows: The user opens the 3D processing program interface, inputs the patient's identity information, such as name, date of birth, patient number, ID number, etc., and enters the storage location of the patient's corresponding photo set. If the storage location does not yet contain photos, the photo set is imported into that location using the image import function. The part to be generated as a 3D photo can be a part of the patient's head, limbs, or torso. Taking the patient's head as the target for 3D photo generation, the corresponding photo set can include 2D photos taken around the head, such as frontal, left-side, and right-side photos. The photo set consisting of the aforementioned frontal, left-side, and right-side photos is imported into the storage location for compositing.

[0088] Next, the user can select all the photos in the photo set and click the 3D composite button. The 3D processing program will then automatically composite the 2D photos in the photo set. There are two possible composite methods:

[0089] 1. Structural Self-Motion Method. This method calculates the 3D structure of a scene by analyzing multiple 2D images of the same scene. This step first collects multiple images of the same scene from different perspectives, then uses the SfM algorithm to calculate the camera's position information and the scene's 3D point cloud, and combines the point cloud and texture information to generate a 3D model, which is finally rendered into a 3D image.

[0090] 2. Neural Network-Based Approach. This method leverages deep learning in artificial intelligence to generate 3D images by inferring depth information from single or multiple 2D images. Specifically, a deep learning model (such as a convolutional neural network) is first trained to predict a depth map. Then, the trained model is used to predict the depth of the input 2D image. Finally, the depth map and the original image are combined to generate the 3D effect.

[0091] After the step of synthesizing a three-dimensional image from two-dimensional images, the three-dimensional processing program can output the synthesized result at the storage location corresponding to the photo set, thus obtaining a three-dimensional image of the patient's head.

[0092] Technical effects:

[0093] 1. The data recording table of this embodiment can record data information of multiple patients and obtain the patient's number through the patient's data information, so that the three-dimensional processing program can find the storage location of the patient's corresponding photo set by using the patient's number as an index.

[0094] 2. By using the processing of a 3D processing program to synthesize the patient's corresponding photo set, multiple 2D images in the photo set can be combined into a 3D image, which is simple for users to operate.

[0095] 3. In addition, after completing the compositing operation for one patient, the 3D processing program continues to perform the compositing operation for the next patient's photo set in the data record table until all patients in the data record table have completed the photo compositing operation, thus realizing the batch processing of multiple patient photo sets.

[0096] 4. The three-dimensional processing program can generate three-dimensional photos of several customers in a batch in a data record table with one click. Compared with traditional three-dimensional modeling methods, such as using modeling software such as 3DMax and AutoCAD or three-dimensional scanners, the method of batch synthesizing three-dimensional images from two-dimensional images is highly automated, lower in cost, faster, and greatly saves labor costs.

[0097] 5. By synthesizing two-dimensional images into three-dimensional images, stereoscopic images can be formed, allowing doctors to observe the lesions on the patient's body more intuitively and improving the accuracy of diagnosis.

[0098] 6. The established 3D images can be rotated and cut as needed, providing doctors with multi-angle views, which helps with preoperative planning and real-time guidance during surgery, reducing surgical risks.

[0099] 7. Three-dimensional synthesis can reduce reliance on traditional contrast agents and lower the X-ray dose received by patients. At the same time, precise surgical planning can also help reduce postoperative complications and speed up patient recovery.

[0100] It's worth noting that the data record table can be in tabular, database, JSON, or XML file format, or even the metadata of an image file itself. The most direct way to obtain the data record table is in formats such as JPEG or TIFF, which can be captured by a mobile phone or camera and displayed visually as images on a computer.

[0101] This patent also provides a calculator program product that stores a computer program, which, when executed by a processor, can implement the steps of the aforementioned method.

[0102] This calculator program contains the aforementioned computer program, including a 3D processing program. Running this program enables the batch synthesis of 3D images. Specifically, the program retrieves a data table, searches for patient identification numbers, finds the corresponding photo set for each patient, performs the synthesis operation, and outputs a 3D image. After sequentially performing the synthesis process on multiple patients, the batch synthesis of 2D images into 3D images is achieved.

[0103] Implementation Method 2

[0104] In the first embodiment, a method for batch synthesis of three-dimensional photographs of patients in medical institutions was proposed. The second embodiment of this patent also proposes a method for batch synthesis of three-dimensional photographs of patients in medical institutions, and this embodiment further improves upon the first embodiment. The main improvement lies in providing a method for indexing the location of the photograph set by patient ID, and specifying a concrete method for synthesizing the photographs.

[0105] Specifically, see Figure 3 As shown, the steps for obtaining the photo set corresponding to the entered patient data include:

[0106] Retrieve the patient ID from the entered patient data;

[0107] The steps of indexing the images in memory to find the corresponding image set and then inputting the acquired image set into the 3D data processing program include:

[0108] The photos in the acquired photo set are sequentially entered into the 3D data processing program;

[0109] The process of initiating the 3D data processing program includes the following steps:

[0110] Select photos from the entered photo set and click the "Glue Images" button to start the processing.

[0111] The steps for obtaining a photo set corresponding to the entered patient data include:

[0112] S221. Obtain the patient ID from the entered patient data. The data record table contains data information for multiple patients. In addition to basic information such as name, ID number, and date of birth, each patient's data information also includes a patient ID. The ID can be used as an index for the 3D processing program to search for the storage location of the patient's corresponding photo set.

[0113] S222. Index the patient's number in memory to find the photo set corresponding to that number. The 3D processing program indexes the patient's number in memory to find the photo set corresponding to that patient's number.

[0114] In other words, a number can be assigned to the storage location of the patient's corresponding photo set, and this number can be associated with the number in the patient's data. This allows the 3D processing program to retrieve and match the corresponding photo set storage location based on the patient's number, facilitating subsequent operations on the photo set.

[0115] Secondly, the steps for inputting the acquired photo set into the 3D processing program include:

[0116] S231. The photos in the acquired photo set are sequentially entered into the 3D data processing program. The photo set can contain multiple individual photos, and each individual photo can also be a composite photo composed of two or more images. Sequentially entering the photos in the photo set into the 3D processing program facilitates the program's recognition and analysis, improves processing efficiency and enhances the accuracy of image processing, and also facilitates subsequent image analysis and processing.

[0117] The process of initiating the 3D data processing program includes the following steps:

[0118] S241. Select the photos from the entered photo set and click the "Glue Images" button to start the processing. Users can select or select all photos from the entered photo set sequentially and click the "Glue Images" button to start the 3D data processing program to automatically composite the photos.

[0119] P3 This embodiment has the following technical effects:

[0120] 1. Match the patient ID in the patient data with the data at the location of the photo set in the storage, so that users can indirectly find the storage location of the corresponding photo set of the patient by using patient-related information or directly by using the patient ID.

[0121] 2. The 3D data processing program processes the photos in the photo set one by one in sequence, which helps to improve the efficiency of the 3D processing program in image processing. The images processed later can be analyzed and processed based on the results of the previous image processing, which improves the accuracy of 3D image synthesis.

[0122] 3. Click the "Glue Images" button to start the processing and begin automatically combining the photos in the input photo set. The operation is convenient and quick.

[0123] Implementation Method 3

[0124] The third embodiment of this patent also proposes a method for batch synthesis of three-dimensional images of patients in medical institutions. This embodiment is a further improvement on the first and second embodiments. The improvement lies in that, in step S25, an algorithm is added to determine and mark whether the synthesis result output by the three-dimensional data processing program is acquired.

[0125] Specifically, see Figure 4 As shown, after obtaining the composite result output by the 3D data processing program and saving the composite result in a preset location,

[0126] S251. If the synthesis result can be obtained, add a synthesis success mark to the corresponding patient data in the data record table;

[0127] S252. If the synthesis result cannot be obtained, add a synthesis failure mark to the corresponding patient data in the data record table.

[0128] The success / failure markers are used to mark the processing results of the 3D processing program on the photograph. If the fusion is successful and a 3D image is output, a success marker is added to the corresponding part of the patient data; if the fusion fails and a 3D image cannot be output, a failure marker is added to the corresponding part of the patient data.

[0129] After batch processing all patient data in the data log table, the 3D processing program can return the processing status to the user. For example, it displays a list of patients whose data processing was successful (marked as "successful") and a list of patients whose data processing failed, based on the success / failure markers in the patient data. The user can then check the photos in the corresponding photo sets for each patient on the list of patients whose data processing failed, and supplement or replace the images as needed.

[0130] By designing success / failure markers for compositing, the 3D processing program can provide feedback on the processing results to the user, allowing the user to intuitively understand which patients have been successfully synthesized. This also facilitates the user's inspection of the photos in the photo set corresponding to patients who failed to be synthesized, enabling the repair or replacement of defective photos.

[0131] Implementation Method 4

[0132] The fourth embodiment of this patent also proposes a method for batch synthesis of three-dimensional photographs of patients in medical institutions. The fourth embodiment is largely the same as the second and third embodiments, but its improvement lies in that the photograph set also includes a frontal photograph, a first side photograph, and a second side photograph of the patient, which are two-dimensional photographs obtained by taking pictures of the same part from multiple angles. It also provides a method for expanding and repairing the photographs by using existing photographs in the photograph set, so that a more complete three-dimensional image can be obtained after synthesis.

[0133] Specifically, the photo collection includes at least:

[0134] A frontal photograph of the patient;

[0135] First lateral photograph of the patient;

[0136] The patient's second lateral view photograph, which, along with the first lateral view photograph, shows the patient's left and right lateral views, respectively.

[0137] When using a patient's head as the subject of the image synthesis, to achieve better 3D image composition, the photo set can include at least three types of photos: a frontal view of the patient, a first side view of the patient, and a second side view of the patient. These three types of photos capture the patient's head from three different angles, resulting in a more complete and detailed 3D image after synthesis.

[0138] The photo set contains multiple sets of photos taken from different angles. On the one hand, this allows the 3D processing program to generate more detailed 3D images after processing multiple sets of photos. On the other hand, 2D photos taken from multiple angles over a large area can be processed by the 3D processing program to generate 3D images that cover a wider area of ​​the patient and medical staff and provide a more complete image.

[0139] In an optional embodiment, when the hand is the object to be generated, in order to fully restore the hand, the photo set may include at least a photo of the palm, a photo of the back of the hand, a photo of the first side of the hand, and a photo of the second side of the hand, so as to fully and completely restore the model of the patient's hand.

[0140] Some photos in the photo collection may be damaged. In this case, the photos can be repaired using relevant algorithms in the 3D processing program.

[0141] Optionally, see Figure 3 As shown, in the batch synthesis method, the step of obtaining the photo set corresponding to the entered patient based on the entered patient data also includes:

[0142] S223. Determine whether the frontal and second side photos in the photo set contain defect marking areas;

[0143] S224. If the determination is yes, then the frontal photo and the second side photo are corrected according to the first side photo.

[0144] First, let's explain the concept of "defect marking areas": In some cases, two-dimensional photographs may reveal physiological defects in certain parts of a patient's body, such as cleft lip, facial asymmetry, facial flaws, or burns on the body surface. In such cases, to restore the original condition of the body or to demonstrate the postoperative recovery effect to the patient, medical staff can mark the physiological defect areas in some of the two-dimensional photographs and then repair these marked areas using a three-dimensional synthesis program based on other photographs in the same set.

[0145] In this case, the portion of a two-dimensional image marked as a defect area at a certain angle can be compensated and repaired using a two-dimensional image from another angle. The following compensation and repair methods are possible:

[0146] 1. For a symmetrical part of a patient's body, mirror compensation can be performed using this symmetry. By analyzing the axis or plane of symmetry of the part, a symmetrical virtual image is generated to supplement the defects in the original image.

[0147] 2. Images from multiple perspectives are fused to compensate for information loss in a single image. Algorithms analyze the relationships between images to accurately reconstruct occluded or missing parts.

[0148] For example, for a patient with a defect on one side of their face, the defect can be repaired by mirroring a 2D photograph of the other side of the face, generating a 2D photograph of the repaired area. Then, the photographs from both sides are used to synthesize a 3D photograph, creating a 3D image without the defect. This 3D photograph can be used to demonstrate the postoperative recovery effect or to reconstruct the patient's facial condition before surgery, facilitating the doctor's observation and surgical planning.

[0149] Alternatively, taking the patient's head as an example, the specific method is as follows: A 3D processing program acquires photographs of the patient's head, and an image recognition algorithm determines whether the frontal and second side-view photographs in the photograph set contain defect-marked areas. If defect-marked areas are present, the defect-marked areas in the frontal or second side-view photographs can be repaired and corrected using image data from the first side-view photograph. If the first and second side-view photographs are symmetrical about the patient's head, the second side-view photograph can be mirrored using the first side-view photograph. If the first side-view photograph and the frontal photograph overlap, the complete image portion of the first side-view photograph can be used to compensate for the image defects in the frontal photograph.

[0150] Implementation Method 5

[0151] In some cases, the number of 2D photographs taken from various angles in a photo set is too small to form a complete 3D image. When it's not possible to supplement the photo set with external photographs, algorithms in a 3D processing program can be used to expand the photo set using existing photographs, facilitating the synthesis of a 3D image.

[0152] Accordingly, the fifth embodiment of this patent proposes a method for batch synthesis of three-dimensional photographs of patients in medical institutions. The fifth embodiment is largely the same as the third and fourth embodiments, with the main difference being that this embodiment provides a method for expanding photographs based on a photograph set.

[0153] Specifically, see Figure 3 As shown, a method for batch synthesis of 3D images of patients in a medical institution, wherein the image set includes at least:

[0154] First lateral photograph of the patient;

[0155] The step of obtaining a photo set corresponding to the entered patients based on the entered patient data also includes:

[0156] Determine whether a frontal photo and a second side photo of the patient can be obtained. The second side photo and the first side photo are photos of the left and right sides of the patient, respectively.

[0157] If the determination is yes, then obtain the patient's frontal photo and second side photo;

[0158] If the determination is no, the photo set is expanded based on the photos currently contained in the photo set until the 3D processing program can synthesize the photo set;

[0159] If the photo set contains at least a first-view photograph of the patient, the step of obtaining the photo set corresponding to the entered patient based on the entered patient data further includes:

[0160] S225. Determine whether a frontal photograph and a second side photograph of the patient can be obtained. The second side photograph and the first side photograph are photographs of the patient's left and right sides, respectively. The first side photograph and the second side photograph are defined as corresponding to the patient's left and right side photographs, respectively. In this step, the 3D processing program identifies the types of photographs in the photograph set and determines whether a frontal photograph and a second side photograph of the patient can be obtained from the photograph set.

[0161] S226. If the determination is yes, then acquire the patient's frontal and second side profile photos. When the 3D processing program detects that the photo set contains a frontal and second side profile photo, it determines this as yes and reads the frontal and second side profile photos.

[0162] S227. If the determination is no, then expand the photo set according to the photos currently contained in the photo set, until...

[0163] The 3D processing program can synthesize photo sets. When the program detects that the photo set lacks frontal and second-side photos, it expands the photo set using image augmentation algorithms based on the existing photos until the program can synthesize the photo set.

[0164] In optional embodiments, situations may arise where photos in the photo set are missing, or where the areas shown in the photos cannot be synthesized into the desired image due to physiological defects. For example, when the second side view and frontal view are missing, or the areas shown have physiological defects, the first side view can be used to expand and generate a frontal view and a second side view. Then, a three-dimensional image can be synthesized based on the original first side view, the frontal view generated from the first side view, and the second side view to obtain a complete and clear three-dimensional image, which is beneficial for medical personnel to observe. In specific situations, such as when showing a patient the postoperative repair status of a certain area, a three-dimensional image synthesized using an area that was originally without defects can present a more satisfactory result than a three-dimensional image synthesized using photos showing defective areas.

[0165] It's worth noting that common image augmentation algorithms include Keras-based image augmentation. Specifically, it can be Keras-based image augmentation. Keras is a high-level neural network API that allows for the simple and fast construction and training of various types of neural networks, including deep learning models. In Keras-based image augmentation, Keras provides the Image Data Generator class, which can perform transformations on images, including but not limited to: rotating the image, randomly translating the image left or right, randomly translating the image up or down, randomly flipping the image horizontally, and performing scaling and cropping operations.

[0166] Image augmentation algorithms also include deep learning-based image generation algorithms. These algorithms analyze the texture, color, and structural information of the original image and then generate new pixels based on this information, thereby achieving image augmentation. Deep learning-based image generation algorithms have advantages such as efficiency, accuracy, and flexibility.

[0167] See Figure 4 As shown, in this embodiment, after the step of obtaining the synthesis result output by the 3D data processing program and saving the synthesis result in a preset location, the method further includes:

[0168] S253. Add auxiliary generation tags to the corresponding patients in the data record table to confirm that the composite result for the patient is a computer-aided composite result. If the generated 3D image is a composite of a 2D image augmented by an algorithm, add auxiliary generation tags to the generated result. After the 3D image is output, the user can check whether the 3D image has auxiliary generation tags through the result feedback list or the patient's information data for further comparison and verification. If the generated 3D image is not significantly different from the real situation, it can be used directly. If the generated 3D image differs somewhat from the real situation, photos can be added to the photo set to regenerate the image.

[0169] In this step, adding auxiliary generation markers reminds the user which 3D images were generated using augmented 2D photographs. When the auxiliary generation result is not significantly different from the reality, the 3D photograph can be used directly, helping to improve the efficiency of the compositing process. When the auxiliary generation result differs significantly from the reality, corresponding photographs are added to the relevant photo set. This reduces the workload and cost of post-compositing inspection and maintenance.

[0170] Optionally, the photo set for the aforementioned batch compositing method also includes:

[0171] The patient's second lateral photograph;

[0172] The steps to expand your photo collection include:

[0173] S2271. Generate a frontal photograph of the patient based on the first and second side photographs.

[0174] The second side view photograph of the patient, compared to the first side view photograph, captures a specific part of the patient from a different angle and may overlap in coverage. Furthermore, the first and second side view photographs, after being processed by a 3D program, can generate a frontal photograph of the patient. Taking the head as an example, the first side view photograph captures the image of the patient's right side of the face, including a partial frontal view, while the second side view photograph captures the image of the patient's left side of the face, also including a partial frontal view. The 3D processing program can use the first and second side view photographs to overlay the frontal portion of the head image, thus generating a frontal photograph of the patient.

[0175] In some cases, a 2D photograph of one side of the photographed area may be severely damaged, making it impossible to accurately reproduce the patient's face. In such cases, a 2D image of the opposite side of the area can be generated by mirroring the photograph to replace the damaged one. Therefore, this embodiment proposes the following photograph generation method:

[0176] Optionally, the second side view photo is generated by mirroring the first side view photo.

[0177] For symmetrical areas, when a first side view photograph captures one side of the area, a second side view photograph can be generated by mirroring the first side view photograph to obtain an image of the other side of the photographed area. These two mirrored photographs, after being processed by a 3D program, can generate a complete 3D image of the photographed area, allowing for its reconstruction and avoiding situations where reconstruction is impossible due to photographic damage.

[0178] Implementation Method Six

[0179] The sixth embodiment of this patent also proposes a method for batch synthesis of three-dimensional photographs of patients in medical institutions. This embodiment is a further improvement on the fifth embodiment, the improvement being that it provides a method for correcting the distinguishing sub-regions by comparing a frontal photograph and a first side photograph, which can achieve the repair of surface blemishes such as moles and scars.

[0180] Specifically, see Figure 5 As shown, the photo set also includes:

[0181] A frontal photograph of the patient;

[0182] The steps to expand your photo collection include:

[0183] S2272, Mirror the first side view photo to obtain a template for the second side view photo;

[0184] S2273. Obtain the sub-region that is different from the first region of the frontal photo. The second region is the region corresponding to the second side photo, and the first region is the region corresponding to the first side photo.

[0185] S2274. Based on the template for correcting the distinguishing sub-regions, generate a second side view photograph and regenerate the frontal photograph after eliminating the distinguishing sub-regions.

[0186] A 3D processing program can divide a frontal photograph of a symmetrical area of ​​a patient into a first region located to the left of the symmetrical area and a second region located to the right of the symmetrical area. The distinguishing sub-regions are obtained by comparing the differences between the first and second regions. Specifically, the first region can be mirrored, and then the mirrored first region is compared with the second region. Regions with lower similarity are extracted and marked as distinguishing sub-regions. These sub-regions are then processed using a repair template stored in the 3D processing program to achieve facial restoration and eliminate surface blemishes.

[0187] The above methods and steps can eliminate surface scars or moles on a patient's body, resulting in a cleaner and more aesthetically pleasing 3D image. This also facilitates observation of the 3D image by surgical personnel or provides patients with predictive photos of postoperative recovery for reference.

[0188] Finally, it should be noted that those skilled in the art will understand that many technical details have been provided in the embodiments of this patent to facilitate a better understanding of the invention. However, even without these technical details and various variations and modifications based on the above embodiments, the technical solutions claimed in the claims of this patent can be substantially achieved. Therefore, in practical applications, various changes can be made to the above embodiments in form and detail without departing from the spirit and scope of this patent.

Claims

1. A method for batch synthesis of three-dimensional images of patients in a medical institution, characterized in that, include: Retrieve the data record table; Based on the data of the first patient in the data record table, perform the synthesis step, which includes: Enter the data of a single patient recorded in the data record table into the three-dimensional processing program; Based on the entered patient data, obtain the photo set corresponding to the entered patients; The acquired photo set is entered into a 3D data processing program; Start the processing of the 3D data processing program, enabling it to perform a composite operation on the photos in the photo set; Acquire the synthesis results output by the 3D data processing program and save the synthesis results in a preset location; For the data of the second patient in the data record table, repeat the synthesis step until all patients' data in the data record table have been processed.

2. The method for batch synthesis of three-dimensional images of patients in a medical institution according to claim 1, characterized in that, The steps for obtaining a photo set corresponding to the entered patient data include: Retrieve the patient ID from the entered patient data; The memory is indexed by number to find the photo set corresponding to the number; The steps for inputting the acquired photo set into the 3D data processing program include: The photos in the acquired photo set are sequentially entered into the 3D data processing program; The process of initiating the 3D data processing program includes the following steps: Select photos from the entered photo set and click the "Glue Images" button to start the processing.

3. The method for batch synthesis of three-dimensional images of patients in a medical institution according to claim 1, characterized in that, After acquiring the composite result output by the 3D data processing program and saving the composite result in a preset location, If the synthesis result can be obtained, add a synthesis success mark to the corresponding patient data in the data record table; If the synthesis result cannot be obtained, a synthesis failure mark is added to the corresponding patient data in the data record table.

4. The method for batch synthesis of three-dimensional images of patients in a medical institution according to claim 1, characterized in that, The photo collection should include at least: A frontal photograph of the patient; First lateral photograph of the patient; The patient's second lateral view photograph, which, along with the first lateral view photograph, shows the patient's left and right lateral views, respectively.

5. The method for batch synthesis of three-dimensional images of patients in a medical institution according to claim 4, characterized in that, The steps for obtaining the photo set corresponding to the entered patients, based on the entered patient data, also include: Determine whether the frontal and second-side photos in the photo set contain defect marking areas; If the determination is yes, then the frontal photo and the second side photo are corrected based on the first side photo.

6. The method for batch synthesis of three-dimensional images of patients in a medical institution according to claim 1, characterized in that, The photo collection should include at least: First lateral photograph of the patient; The step of obtaining a photo set corresponding to the entered patients based on the entered patient data also includes: Determine whether a frontal photo and a second side photo of the patient can be obtained. The second side photo and the first side photo are photos of the left and right sides of the patient, respectively. If the determination is yes, then obtain the patient's frontal photo and second side photo; If the determination is no, the photo set is expanded based on the photos currently contained in the photo set until the 3D processing program can synthesize the photo set; After the steps of acquiring the composite result output by the 3D data processing program and saving the composite result in a preset location, the process also includes: Add auxiliary generation tags to the corresponding patients in the data record table to clarify that the patients' synthesis results are computer-aided synthesis results.

7. The method for batch synthesis of three-dimensional images of patients in a medical institution according to claim 6, characterized in that, The photo collection also includes: The patient's second lateral photograph; The steps to expand your photo collection include: A frontal photograph of the patient is generated based on the first and second side photographs.

8. The method for batch synthesis of three-dimensional images of patients in a medical institution according to claim 7, characterized in that, The second side view photo was generated by mirroring the first side view photo.

9. A method for batch synthesis of three-dimensional images of patients in a medical institution according to claim 6, characterized in that, The photo collection also includes: A frontal photograph of the patient; The steps to expand your photo collection include: Mirror the first side view photo to obtain a template for the second side view photo; The difference between the second region and the first region of the frontal photo is obtained. The second region is the region corresponding to the second side photo, and the first region is the region corresponding to the first side photo. Based on the template for correcting the distinguishing sub-regions, a second side view photo is generated, and a frontal photo after eliminating the distinguishing sub-regions is regenerated.

10. A calculator program product, storing a computer program, characterized in that, When executed by a processor, a computer program is capable of implementing the steps of any one of claims 1 to 9.

Images