Guide image generating apparatus and method for breast reconstruction surgery, extended reality device

Abstract

The present disclosure relates to an apparatus and method for generating a guide image for breast reconstruction surgery, and to an XR device. An apparatus for generating a guide image for breast reconstruction surgery according to one embodiment of the present disclosure is an apparatus for generating a guide image for breast reconstruction surgery, comprising a processor and a memory operably connected to the processor and storing at least one code executed by the processor. The memory may store code that, when executed through the processor, causes the processor to acquire a 3D breast image as a result of a 3D scan of a patient's breast before breast reconstruction surgery from a 3D scanner, acquire body contour information and upper body angle information of the patient located on the operating table from an XR device, and generate a guide image by overlaying the 3D breast image on the patient's body contour information based on the patient's upper body angle information.

Description

Technology Field

[0001] The present disclosure relates to an apparatus and method for generating guide images for breast reconstruction surgery, and an XR device. Background Technology

[0002] In breast reconstruction surgery performed after a total mastectomy, the goal is to create a breast shape that is as similar as possible to the original breast or symmetrical to the opposite breast. To achieve this, photographs of the patient are taken before surgery to calculate the breast volume, and during breast reconstruction surgery, the operating table is positioned at nearly a 90-degree angle so that the patient can view the shape of the breast while standing.

[0003] The aforementioned background technology is technical information that the inventor possessed for the derivation of the present disclosure or acquired during the process of deriving the present disclosure, and it cannot be considered as prior art disclosed to the general public prior to the filing of the present disclosure. Prior art literature

[0004] Korean Patent Publication No. 2022-0096333 (July 7, 2022) The problem to be solved

[0005] One objective of the present disclosure is to provide a guide image generation device and method for breast reconstruction surgery and an XR device that can improve the accuracy of breast reconstruction surgery.

[0006] The problems that the present disclosure aims to solve are not limited to those mentioned above, and other problems and advantages of the present disclosure not mentioned can be understood from the following description and will be more clearly understood from the embodiments of the present disclosure. Furthermore, it will be understood that the problems and advantages that the present disclosure aims to solve can be realized by the means and combinations thereof set forth in the claims. means of solving the problem

[0007] A guide image generating device for breast reconstruction surgery according to one embodiment of the present disclosure is a guide image generating device for breast reconstruction surgery, comprising a processor and a memory operably connected to the processor and storing at least one code executed by the processor, wherein the memory may store code that, when executed through the processor, causes the processor to acquire a 3D breast image as a 3D scan result of the patient's breast before breast reconstruction surgery from a 3D scanner, acquire body contour information of the patient located on the operating table and upper body angle information of the patient from an XR device, and generate a guide image by overlaying the 3D breast image on the patient's body contour information based on the patient's upper body angle information.

[0008] A method for generating a guide image for breast reconstruction surgery according to one embodiment of the present disclosure is a method for generating a guide image performed by a processor of a guide image generating device for breast reconstruction surgery, and may include the steps of: obtaining a three-dimensional breast image as a result of a three-dimensional scan of a patient's breast before breast reconstruction surgery from a three-dimensional scanner; obtaining body contour information of a patient located on an operating table and upper body angle information of a patient from an XR device; and generating a guide image by overlaying the three-dimensional breast image on the patient's body contour information based on the patient's upper body angle information.

[0009] A guide image generating device for breast reconstruction surgery according to another embodiment of the present disclosure is an XR device for generating a guide image for breast reconstruction surgery, comprising a processor and a memory operably connected to the processor and storing at least one code executed by the processor, wherein the memory may store code that, when executed through the processor, causes the processor to acquire a 3D breast image as a 3D scan result of the patient's breast before breast reconstruction surgery from a 3D scanner, calculate the patient's body contour information and the patient's upper body angle information located on the operating table, and, based on the patient's upper body angle information, generate and output a guide image in which the 3D breast image is overlaid on the patient's body contour information.

[0010] In addition to this, other methods for implementing the present disclosure, other systems, and computer-readable recording media storing a computer program for executing said method may be further provided.

[0011] Other aspects, features, and advantages other than those described above will become clear from the following drawings, claims, and detailed description of the invention. Effects of the invention

[0012] According to the present disclosure, the accuracy of breast reconstruction surgery can be improved through a guide image generation device and method for breast reconstruction surgery and an XR device.

[0013] The effects of the present disclosure are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0014] FIG. 1 is an exemplary diagram of a guide image generation environment for breast reconstruction surgery according to the present disclosure. FIG. 2 is a block diagram illustrating the configuration of a guide image generating device for breast reconstruction surgery according to the present disclosure. FIG. 3 is a block diagram illustrating the configuration of an image generation management unit according to one embodiment of the guide image generation device for breast reconstruction surgery of FIG. 2. FIG. 4 is an exemplary diagram of a patient's first and second positions according to the present disclosure. FIG. 5 is an exemplary illustration of a third position of a patient according to the present disclosure. FIG. 6 is an example of a guide image according to the present disclosure. FIG. 7 is an example diagram showing the patient's body contour information and the patient's body contour extraction result overlaid according to the result of the decision of overlay permission according to the present disclosure. FIG. 8 is a block diagram illustrating the configuration of an image generation management unit according to another embodiment of the guide image generation device for breast reconstruction surgery of FIG. 2. FIG. 9 is a block diagram illustrating the configuration of a guide image generating device for breast reconstruction surgery according to another embodiment of the present disclosure. FIG. 10 is a block diagram illustrating the configuration of an augmented reality device equipped with a guide image generating device for breast reconstruction surgery according to another embodiment of the present disclosure. FIG. 11 is a flowchart illustrating a method for generating a guide image for breast reconstruction surgery according to one embodiment of the present disclosure. FIG. 12 is a flowchart illustrating a method for generating a guide image for breast reconstruction surgery according to another embodiment of the present disclosure. Specific details for implementing the invention

[0015] The advantages and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, but can be implemented in various different forms and should be understood to include all modifications, equivalents, and substitutions that fall within the spirit and scope of the present invention. The embodiments presented below are provided to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention. In describing the present invention, detailed descriptions of related known technologies are omitted if it is determined that such detailed descriptions may obscure the essence of the present invention.

[0016] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “comprising” or “having” are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. Terms such as “first,” “second,” etc., may be used to describe various components, but the components should not be limited by these terms. These terms are used solely for the purpose of distinguishing one component from another.

[0017] Additionally, in this application, “part” may be a hardware component, such as a processor or circuit, and / or a software component executed by a hardware component, such as a processor.

[0018] Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings. In describing with reference to the attached drawings, identical or corresponding components are given the same reference numerals, and redundant descriptions thereof will be omitted.

[0019] In the following embodiments, the terms first, second, etc. are used not in a restrictive sense, but for the purpose of distinguishing one component from another component.

[0020] In the following embodiments, singular expressions include plural expressions unless the context clearly indicates otherwise.

[0021] In the following embodiments, terms such as "include" or "have" mean that the features or components described in the specification are present, and do not preclude the possibility that one or more other features or components may be added.

[0022] Where an embodiment can be implemented differently, a specific process sequence may be performed differently from the order described. For example, two processes described consecutively may be performed substantially simultaneously or proceed in the reverse order of the description.

[0023] FIG. 1 is an exemplary diagram of a guide image generation environment for breast reconstruction surgery according to the present disclosure. Referring to FIG. 1, the guide image generation environment (1) for breast reconstruction surgery according to the present disclosure may include a guide image generation device (100), a 3D scanner (200), an XR device (300), and a network (400).

[0024] The guide image generating device (100) can obtain a three-dimensional breast image as a result of a three-dimensional scan of the patient's breast before breast reconstruction surgery from a three-dimensional scanner (200). In this embodiment, the three-dimensional breast image obtained by the guide image generating device (100) from the three-dimensional scanner (200) may include an image of the patient's breast or an image of the patient's body contour information including the patient's breast. The guide image generating device (100) can obtain a three-dimensional breast image for the patient's first to third postures described below from the three-dimensional scanner (200).

[0025] The guide image generating device (100) can obtain body contour information of a patient positioned on an operating table and upper body angle information of the patient from the XR device (300).

[0026] The guide image generating device (100) can generate a guide image by overlaying a 3D breast image obtained from a 3D scanner (200) on the patient's body contour information obtained from the XR device (300), based on the patient's upper body angle information obtained from the XR device (300).

[0027] The guide image generating device (100) When generating a guide image, the overlay reference point can be determined using the patient's body contour information. Additionally, the guide image generating device (100) When generating a guide image, the overlay reference point can be determined by a barcode or QR code attached to the patient's body during 3D scanning.

[0028] The guide image generating device (100) can transmit the generated guide image to the XR device (300).

[0029] In the present embodiment, the guide image generating device (100) can generate a guide image by utilizing artificial intelligence. Here, artificial intelligence (AI) is a field of computer engineering and information technology that studies methods to enable computers to perform thinking, learning, self-development, etc., which are possible with human intelligence, and can mean enabling computers to imitate human intelligent behavior.

[0030] Furthermore, artificial intelligence does not exist in isolation but is closely related, directly and indirectly, to many other fields of computer science. Particularly in the modern era, there are very active attempts to introduce AI elements into various sectors of information technology and utilize them to solve problems within those fields.

[0031] Artificial intelligence can primarily utilize technologies and methods such as machine learning, deep learning, natural language processing, and computer vision.

[0032] Machine learning is a field of artificial intelligence that encompasses research areas that empower computers to learn without explicit programming. Specifically, machine learning can be defined as a technology that studies and builds systems and algorithms capable of learning, making predictions, and improving their own performance based on empirical data. Rather than executing strictly defined static program commands, machine learning algorithms may adopt an approach of constructing specific models to derive predictions or decisions based on input data.

[0033] These machine learning algorithms may include LSTM (long short term memory networks), GRU (Gated Recurrent Unit), CNN (convolutional neural networks), Random forest, Decision tree, SVM (support vector machine), etc.

[0034] In this embodiment, the guide image generating device (100) may exist independently in the form of a server, or the guide image generating function provided by the guide image generating device (100) may be implemented in the form of an application and installed on the XR device (300).

[0035] The 3D scanner (200) can generate a 3D breast image as a result of a 3D scan of the patient's breast before breast reconstruction surgery and transmit it to a guide image generating device (100).

[0036] In this embodiment, the 3D scanner (200) may include a device that accurately captures objects or parts of the real world and reproduces them in a digital form. When the 3D scanner (200) is applied to breast reconstruction surgery, it can scan the patient's chest, analyze the results in detail, and visualize them for use in breast reconstruction surgery. The 3D scanner (200) generally uses optical or laser scanning technology and can scan the patient's chest from multiple angles using multiple camera lenses or lasers. In this process, the shape, size, contour, and skin surface of the patient's chest can be precisely captured. The data scanned by the 3D scanner (200) is converted into a 3D model through software, and this 3D model can accurately visualize and display the patient's chest in a virtual space. This can provide the advantage of allowing the doctor and the patient to simulate the actual results before surgery and establish a more sophisticated surgical plan.

[0037] When generating a three-dimensional breast image, the 3D scanner (200) can generate a three-dimensional breast image of a patient's first posture that simulates a preset upper body angle of the patient applied during breast reconstruction surgery. Here, the patient's first posture may include a posture in which the upper body is raised to a preset vertical angle (e.g., 70 degrees) based on the state in which the patient is lying on the operating table (e.g., 0 degrees).

[0038] Additionally, the 3D scanner (200) can generate a 3D breast image for a patient's second position by vertically converting the upper body angle based on the angle applied when generating the 3D breast image for the patient's first position. Here, the patient's second position may include a position in which the upper body angle is vertically converted to, for example, 30 degrees, 50 degrees, 80 degrees, etc., based on the state in which the patient is lying on the operating table (e.g., 0 degrees).

[0039] Additionally, the 3D scanner (200) can generate a 3D breast image of the patient's third position in which the patient's body, which was lying down, is tilted to the side as the operating table is seated. Here, the patient's third position may include a position in which the patient's body, which was lying down, is tilted in a first direction (e.g., +10 degrees to the left) or a second direction (e.g., +10 degrees to the right) as the operating table is seated.

[0040] The XR device (300) can generate body contour information and upper body angle information of a patient positioned on an operating table and transmit them to a guide image generating device (100).

[0041] An XR device (300) is a device that fuses the real world with virtual elements and can provide a visually expanded reality experience. Generally, an XR device (300) can use technologies such as a portable display, camera, and sensor to recognize the environment in real time and insert and display virtual objects. The types of such XR devices (300) may include smartphones and tablets, or XR glasses / headsets, XR markers / tags, XR wearables, etc.

[0042] The XR device (300) can recognize the environment of the operating room and collect environmental data using built-in sensors (not shown) and cameras (not shown) to generate body contour information of a patient positioned on the operating table. The environmental data collected by the XR device (300) may include the body of the patient positioned on the operating table captured by the built-in camera. The XR device (300) can generate body contour information by utilizing computer vision technology and image recognition algorithms to identify parts of the patient's body and extract their contours.

[0043] Additionally, the XR device (300) may use sensors such as an accelerometer and a gyroscope provided internally to generate upper body angle information of a patient positioned on an operating table. The XR device (300) may calculate the patient's upper body angle information by applying a sensor fusion algorithm, a quaternion filter, marker-based tracking, etc., to the patient's posture and movement data collected by sensors such as an accelerometer and a gyroscope provided internally.

[0044] The network (400) can serve to connect the guide image generating device (100), the 3D scanner (200), and the XR device (300). This network (400) may include wired networks such as LAN (local area network), WAN (wide area network), MAN (metropolitan area network), and ISDN (integrated service digital network), or wireless networks such as WLAN (wireless LAN), CDMA (code-division multiple access), and satellite communication, but the scope of the present invention is not limited thereto. Additionally, the network (400) can transmit and receive information using short-range communication and / or long-range communication. Here, short-range communication may include Bluetooth, RFID (radio frequency identification), IrDA (infrared data association), UWB (ultra-wideband), ZigBee, and Wi-Fi technologies, and long-range communication may include CDMA (code-division multiple access), FDMA (frequency-division multiple access), TDMA (time-division multiple access), OFDMA (orthogonal frequency-division multiple access), and SC-FDMA (single carrier frequency-division multiple access) technologies.

[0045] The network (400) may include connections of network elements such as hubs, bridges, routers, and switches. The network (400) may include one or more connected networks, such as a multi-network environment, including a public network such as the Internet and a private network such as a secure corporate private network. Access to the network (400) may be provided through one or more wired or wireless access networks.

[0046] The network (400) may include connections of network elements such as hubs, bridges, routers, and switches. The network (400) may include one or more connected networks, such as a multi-network environment, including a public network such as the Internet and a private network such as a secure corporate private network. Access to the network (400) may be provided through one or more wired or wireless access networks.

[0047] Furthermore, the network (400) can support CAN (controller area network) communication, V2I (vehicle to infrastructure) communication, V2X (vehicle to everything) communication, WAVE (wireless access in vehicular environment) communication technology, and IoT (Internet of Things) network and / or 5G communication that exchanges and processes information between distributed components such as objects.

[0048] FIG. 2 is a block diagram illustrating the configuration of a guide image generating device for breast reconstruction surgery according to the present disclosure. In the following description, parts that overlap with the description of FIG. 1 will be omitted. Referring to FIG. 2, the guide image generating device (100) may include a communication unit (110), a storage medium (120), a program storage unit (130), a database (140), an image generation management unit (150), and a control unit (160).

[0049] The communication unit (110) may provide a communication interface necessary to provide transmission and reception signals between the guide image generation device (100), the 3D scanner (200), and the XR device (300) in the form of packet data in conjunction with the network (400). Furthermore, the communication unit (110) may perform the role of receiving a predetermined information request signal from the 3D scanner (200) or the XR device (300) and may perform the role of transmitting information processed by the image generation management unit (150) to the 3D scanner (200) or the XR device (300). Here, the communication interface is a medium that performs the role of connecting the guide image generation device (100), the 3D scanner (200), and the XR device (300), and may include a path that provides a connection path so that the 3D scanner (200) or the XR device (300) can transmit and receive information after connecting to the guide image generation device (100). Additionally, the communication unit (110) may be a device including hardware and software necessary to transmit and receive signals, such as control signals or data signals, through a wired or wireless connection with another network device.

[0050] The storage medium (120) performs the function of temporarily or permanently storing data processed by the control unit (160). Here, the storage medium (120) may include magnetic storage media or flash storage media, but the scope of the present invention is not limited thereto. Such storage medium (120) may include internal memory and / or external memory, and may include volatile memory such as DRAM, SRAM, or SDRAM, non-volatile memory such as OTPROM (one time programmable ROM), PROM, EPROM, EEPROM, mask ROM, flash ROM, NAND flash memory, or NOR flash memory, flash drives such as SSD, CF (compact flash) card, SD card, Micro-SD card, Mini-SD card, Xd card, or memory stick, or storage devices such as HDD.

[0051] The program storage unit (130) is equipped with control software that performs tasks such as obtaining a three-dimensional breast image as a result of a three-dimensional scan of the patient's breast before breast reconstruction surgery from a three-dimensional scanner (200), obtaining the patient's body contour information and the patient's upper body angle information located on the operating table from an XR device (300), and generating a guide image by overlaying the three-dimensional breast image on the patient's body contour information based on the patient's upper body angle information.

[0052] The database (140) may include a management database that stores various information for generating guide images. For example, the management database may store an algorithm for calculating the contour and volume of the breast from a 3D breast image obtained from a 3D scanner (200), and an artificial intelligence algorithm for generating guide images.

[0053] Additionally, the database (140) may include a user database that stores information about patients who will undergo breast reconstruction surgery. Here, the patient information may include basic information about the user, such as the patient's name, affiliation, personal details, gender, age, contact information, email, address, and image; information about the patient's authentication (login), such as ID (or email) and password; information related to the connection, such as the country of connection, location of connection, information about the device used for connection, and the connected network environment.

[0054] The image generation management unit (150) can obtain a 3D breast image as a 3D scan result of the patient's breast before breast reconstruction surgery from a 3D scanner (200). The image generation management unit (150) can obtain body contour information of the patient positioned on the operating table and upper body angle information of the patient from an XR device (300). Based on the upper body angle information of the patient obtained from the XR device (300), the image generation management unit (150) can generate a guide image by overlaying the 3D breast image obtained from the 3D scanner (200) onto the body contour information of the patient obtained from the XR device (300).

[0055] The control unit (160) is a type of central processing unit and can control the operation of the entire guide image generation device (100) by running control software mounted in the program storage unit (130). The control unit (160) may include all types of devices capable of processing data, such as a processor. Here, 'processor' may refer to a data processing device embedded in hardware that has a physically structured circuit to perform functions expressed by code or instructions included in a program, for example. Examples of such data processing devices embedded in hardware may include microprocessors, central processing units (CPUs), processor cores, multiprocessors, ASICs (application-specific integrated circuits), and FPGAs (field programmable gate arrays), but the scope of the present invention is not limited thereto.

[0056] FIG. 3 is a block diagram illustrating the configuration of an image generation management unit among the guide image generation devices for breast reconstruction surgery of FIG. 2, and FIG. 4 to 7 are exemplary diagrams illustrating guide image generation according to the present disclosure. In the following description, parts that overlap with the description of FIG. 1 and FIG. 2 will be omitted. Referring to FIG. 3 to 7, the image generation management unit (150) may include a first acquisition unit (151), a second acquisition unit (152), and a generation unit (153).

[0057] The first acquisition unit (151) can acquire a three-dimensional breast image as a result of a three-dimensional scan of the patient's breast before breast reconstruction surgery from a three-dimensional scanner (200).

[0058] The first acquisition unit (151) can acquire a 3D breast image of a patient in a first posture that simulates a preset upper body angle of the patient applied during breast reconstruction surgery from a 3D scanner (200). Additionally, the first acquisition unit (151) can generate a 3D breast image of a patient in a second posture from the 3D scanner (200) in which the upper body angle is vertically transformed based on the first posture. Additionally, the first acquisition unit (151) can generate a 3D breast image of a patient in a third posture from the 3D scanner (200) in which the body of a patient who was lying down is tilted to the side while sitting on the operating table.

[0059] FIGS. 4 and FIGS. 5 illustrate the patient's posture applied by the 3D scanner (200) when generating a 3D breast image.

[0060] FIG. 4(a) illustrates a first position of a patient (e.g., 70 degrees) in which the upper body is raised vertically at a preset angle relative to the state in which the patient is lying on the operating table (e.g., 0 degrees). The 3D scanner (200) can generate a 3D breast image for the patient's first position and transmit it to a first acquisition unit (151). The first acquisition unit (151) can acquire a 3D breast image for the first position from the 3D scanner (200).

[0061] FIGS. 4(b) and (c) illustrate a second position of a patient (e.g., 30 degrees, 80 degrees, etc.) in which the upper body angle is vertically transformed using a standard applied when generating a three-dimensional breast image for the patient's first position. Here, the standard applied when generating the three-dimensional breast image for the first position may be the state in which the patient is lying on the operating table (e.g., 0 degrees). The three-dimensional scanner (200) can generate a three-dimensional breast image for the patient's second position and transmit it to the first acquisition unit (151). The first acquisition unit (151) can acquire the three-dimensional breast image for the second position from the three-dimensional scanner (200).

[0062] Figure 5 illustrates the third position of the patient, in which the patient's body is tilted to the side while the operating table is seated.

[0063] FIG. 5(a) illustrates a third position in which the body of a lying patient is tilted in a first direction (e.g., +10 degrees to the left) as the operating table is seated. FIG. 5(b) illustrates another third position in which the body of a lying patient is tilted in a second direction (e.g., +10 degrees to the right) as the operating table is seated. The 3D scanner (200) can generate a 3D breast image for the patient's third position and transmit it to a first acquisition unit (151). The first acquisition unit (151) can acquire a 3D breast image for the third position from the 3D scanner (200).

[0064] In an optional embodiment, the first acquisition unit (151) may calculate the contour and volume of the breast corresponding to one or more of the first, second, and third postures from a three-dimensional breast image for the first to third postures acquired from a three-dimensional scanner (200). The first acquisition unit (151) may apply thresholding, segmentation, graph cut, and probabilistic model algorithms to calculate the contour and volume of the breast. In another embodiment, the three-dimensional scanner (200) may calculate the contour and volume of the breast from a three-dimensional breast image and transmit the result to the first acquisition unit (151).

[0065] The second acquisition unit (152) can acquire body contour information of a patient positioned on an operating table and upper body angle information of a patient from the XR device (300).

[0066] The generating unit (153) can generate a guide image by overlaying a three-dimensional breast image on the patient's body contour information based on the patient's upper body angle information.

[0067] FIG. 6 illustrates a guide image according to one embodiment generated by the generation unit (153). Referring to FIG. 6, 610 represents a three-dimensional breast image of a patient obtained from a three-dimensional scanner (200), and 620 represents body contour information of a patient obtained from an XR device (300).

[0068] In this embodiment, the XR device (300) is worn by a doctor in the form of XR glasses / headset, and the doctor can perform high-accuracy breast reconstruction surgery while viewing guide images output on the XR device (300).

[0069] When generating a guide image, the generating unit (153) can generate a guide image by overlaying a three-dimensional breast image on the patient's body contour information based on the patient's body contour information.

[0070] As an optional embodiment, the generating unit (153) may generate a result of extracting the patient's body contour from a 3D breast image when generating a guide image. The generating unit (153) may generate a result of extracting the patient's body contour from a 3D breast image by utilizing computer vision technology and an image recognition algorithm. The generating unit (153) may compare the patient's body contour information obtained from the XR device (300) with the result of extracting the patient's body contour generated by the generating unit (153). The generating unit (153) may decide whether to allow or disallow an overlay based on the similarity between the patient's body contour information and the patient's body contour prediction result as a result of the comparison.

[0071] The generating unit (153) may use, for example, a comparison and evaluation algorithm to calculate the similarity between the patient's body contour information and the patient's body contour prediction result. The generating unit (153) may calculate the similarity between contour information by utilizing the distance comparison of contour points, contour similarity index, morphological operation, statistical analysis, etc., for the patient's body contour information and the patient's body contour prediction result. First, the coordinates of the contour points are compared to calculate the distance or difference between the points, and then, these distances or differences are combined to calculate a contour similarity index and generate a similarity score. Here, the similarity score can be converted into similarity.

[0072] The generation unit (153) can determine whether to grant an overlay based on the similarity between the patient's body contour information and the patient's body contour extraction result as a result of comparison being greater than or equal to a reference value (e.g., 1), and can generate a message indicating that the granting of the overlay has been determined. The generation unit (153) can transmit the message indicating that the granting of the overlay has been determined to the XR device (300).

[0073] The generating unit (153) can determine whether to disallow the overlay based on the result of comparison that the similarity between the patient's body contour information and the patient's body contour extraction result is less than a reference value, and can generate a message indicating that the disallowance of the overlay has been determined. The generating unit (153) can transmit the message indicating that the disallowance of the overlay has been determined to the XR device (300).

[0074] FIG. 7 illustrates an example of overlaying the patient's body contour information and the patient's body contour extraction result according to the decision result of the overlay permission. 710 represents the patient's body contour extraction result obtained from a 3D scanner (200), and 720 may represent the patient's body contour information.

[0075] In this embodiment, the XR device (300) is worn by a doctor in the form of XR glasses / headset, and the doctor can perform high-accuracy breast reconstruction surgery while viewing guide images output on the XR device (300).

[0076] When generating a guide image, the generating unit (153) can generate a guide image corresponding to the patient's upper body angle information and the patient's 3D breast image by using a first deep neural network model that is pre-trained to generate a guide image to be overlaid on the patient's body contour information using the patient's upper body angle information and the patient's 3D breast image. The model may be trained in a supervised learning manner by training data that takes as input a plurality of first upper body angles that are vertically transformed based on when the patient is lying down (e.g., 30 degrees, 50 degrees, 60 degrees, 70 degrees, etc.) and one or more of a plurality of second upper body angles that are horizontally transformed on the first upper body angle, and a 3D breast image corresponding to one or more of the first upper body angle and the second upper body angle, and the guide image as a label.

[0077] The generation unit (153) can train an initially configured first deep neural network model using labeled training data in a supervised learning manner. Here, the initially configured first deep neural network model is an initial model designed to be configured as a model capable of generating guide images for a patient, and the parameter values ​​are set to arbitrary initial values. As the initial model is trained through the aforementioned training data, the parameter values ​​are optimized so that it can be completed as a guide image generation model capable of accurately generating guide images for a patient.

[0078] In an optional embodiment, the generation unit (153) may generate a guide image generation model. When generating the guide image generation model, the generation unit (153) may obtain the result of acquiring the patient's 3D breast image from the 3D scanner (200), and the patient's upper body angle information and the patient's body contour information from the XR device (300). The generation unit (153) may train a first deep neural network model using a training data set in which the patient's upper body angle information and the patient's 3D breast image are inputs and the guide image is a label. Through training the deep neural network model, the generation unit (153) may generate a guide image generation model that generates a guide image to be overlaid on the patient's body contour information using the patient's upper body angle information and the patient's 3D breast image, and store it in memory (180 in FIG. 9).

[0079] The accuracy of breast reconstruction surgery can be improved through the guide image generation device for breast reconstruction surgery according to the present embodiment.

[0080] FIG. 8 is a block diagram illustrating the configuration of an image generation management unit according to another embodiment of the guide image generation device for breast reconstruction surgery of FIG. 2. In the following description, parts that overlap with the description of FIG. 1 to FIG. 7 will be omitted. Referring to FIG. 8, the image generation management unit (150) may include a second acquisition unit (152), a generation unit (153), and a third acquisition unit (154). In this embodiment, the second acquisition unit (152) and the third acquisition unit (154) may be the same as those in FIG. 3.

[0081] The third acquisition unit (154) can calculate the contour of the breast and the volume of the breast corresponding to the three-dimensional breast image for the fourth position in addition to the three-dimensional breast image for the first to third positions described above.

[0082] In this embodiment, the contour of the breast and the volume of the breast may be included in 3D geometry data for breast reconstruction. The 3D geometry data for breast reconstruction may include the contour and volume of the breast, as well as the shape of the breast structure, the size of the breast, the position of the breast, and the spatial relationship of the breast.

[0083] From FIG. 3, three-dimensional breast images for the first to third positions can be acquired before breast reconstruction surgery to produce three-dimensional geometric data for breast reconstruction. However, during actual breast reconstruction surgery, there may be cases where three-dimensional geometric data corresponding to three-dimensional breast images for other positions or omitted positions, in addition to the first to third positions, is required. For example, before surgery, three-dimensional breast images for the first to third positions at 70 degrees, 30 degrees, and 80 degrees, respectively, were acquired to produce three-dimensional geometric data for breast reconstruction, but during actual breast reconstruction surgery, three-dimensional geometric data corresponding to the three-dimensional breast image for 60 degrees may be required.

[0084] In this embodiment, the third acquisition unit (154) can generate three-dimensional geometry data for a fourth posture as a different posture or a missing posture using an artificial intelligence model.

[0085] The third acquisition unit (154) can generate three-dimensional geometric data corresponding to a three-dimensional breast image for a fourth posture using a second deep neural network model that is pre-trained to predict three-dimensional geometric data for breast reconstruction using breast-related information. Here, the second deep neural network model may be a model trained in a supervised learning manner using training data that takes breast-related information as input and includes posture information including the patient's upper body angle and three-dimensional geometric data as labels. In this embodiment, the patient's breast-related information may include one or more of a two-dimensional breast image, a three-dimensional breast image, a mammography video, depth data, and three-dimensional geometric data.

[0086] The third acquisition unit (154) can train the initially set second deep neural network model using labeled training data in a supervised learning manner. Here, the initially set second deep neural network model is an initial model designed to be configured as a model capable of generating 3D geometry data corresponding to a 3D breast image for the patient's fourth posture, and the parameter values ​​are set to arbitrary initial values. As the initial model is trained through the aforementioned training data, the parameter values ​​are optimized so that it can be completed as a guide image generation model capable of accurately generating 3D geometry data for the patient's fourth posture.

[0087] In this embodiment, the second deep neural network model may include a NeRF (neural radiance fields) model. The NeRF model uses a neural network to model the shape and structure of the breast in three dimensions and can simulate and visualize the state before and after surgery. Since the NeRF model can accurately model the shape and characteristics of the breast, it can assist in planning how to perform the surgery beforehand. The NeRF model can be used to simulate changes and manipulations of the breast in real time during surgery, which allows the surgeon to better understand the surgical process and perform it reliably. Since the NeRF model can be used to visually show the patient the expected results before and after surgery, it can be utilized for patient education. Additionally, the NeRF model can be used to evaluate the outcome of the surgery by simulating the actual results after surgery and comparing them with the results of previous surgeries.

[0088] The second acquisition unit (152) can acquire body contour information of the patient positioned on the operating table and upper body angle information of the patient from the XR device (300). The generation unit (153) can generate a guide image by overlaying three-dimensional geometry data for the fourth posture onto the patient's body contour information. Detailed information regarding the second acquisition unit (152) and the generation unit (153) is the same as that described above, so it will be omitted.

[0089] FIG. 9 is a block diagram illustrating the configuration of a guide image generating device for breast reconstruction surgery according to another embodiment of the present disclosure. In the following description, parts that overlap with the description of FIG. 1 to FIG. 8 will be omitted. Referring to FIG. 9, a guide image generating device (100) according to another embodiment may include a first processor (170) and a first memory (180).

[0090] In this embodiment, the first processor (170) can process the functions performed by the communication unit (110), storage medium (120), program storage unit (130), database (140), image generation management unit (150), and control unit (160) disclosed in FIGS. 2 and 3.

[0091] This first processor (170) can control the operation of the entire guide image generation device (100). Here, 'processor' may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by code or instructions included in a program, for example. Examples of such data processing devices embedded in hardware may include microprocessors, central processing units, processor cores, multiprocessors, ASICs, FPGAs, etc., but the scope of the present invention is not limited thereto.

[0092] The first memory (180) is operably connected to the first processor (170) and can store at least one code associated with an operation performed by the first processor (170).

[0093] Additionally, the first memory (180) may perform the function of temporarily or permanently storing data processed by the first processor (170) and may include data built into the database (140). Here, the first memory (180) may include a magnetic storage medium or a flash storage medium, but the scope of the present invention is not limited thereto. Such first memory (180) may include internal memory and / or external memory, and may include volatile memory such as DRAM, SRAM, or SDRAM, non-volatile memory such as OTPROM, PROM, EPROM, EEPROM, mask ROM, flash ROM, NAND flash memory, or NOR flash memory, flash drives such as SSD, CF card, SD card, Micro-SD card, Mini-SD card, xD card, or Memory Stick, or storage devices such as HDD.

[0094] FIG. 10 is a block diagram illustrating the configuration of an XR device equipped with a guide image generating device for breast reconstruction surgery according to another embodiment of the present disclosure. In the following description, parts that overlap with the description of FIG. 1 to FIG. 9 will be omitted. Referring to FIG. 10, the XR device (300) may include a second processor (310) and a second memory (320). The present embodiment may be in a form where the guide image generating device (100) is mounted on the XR device (300).

[0095] The second processor (310) can obtain a three-dimensional breast image as a result of a three-dimensional scan of the patient's breast from the three-dimensional scanner (200) before breast reconstruction surgery.

[0096] The second processor (310) can calculate the body contour information of the patient positioned on the operating table and the upper body angle information of the patient.

[0097] The second processor (310) can generate and output a guide image in which a three-dimensional breast image is overlaid on the patient's body contour information based on the patient's upper body angle information.

[0098] Hereinafter, the functions of the second processor (310) and the second memory (320) provided in the XR device (300) are the same as those of the first processor (170) and the first memory (180) shown in FIG. 9, so a detailed description will be omitted.

[0099] FIG. 11 is a flowchart illustrating a method for generating a guide image for breast reconstruction surgery according to one embodiment of the present disclosure. In the following description, parts that overlap with the description of FIG. 1 to FIG. 10 will be omitted. The method for generating a guide image for breast reconstruction surgery according to the present embodiment will be described under the assumption that a guide image generating device (100) performs the task at a first processor (170) with the assistance of surrounding components.

[0100] Referring to FIG. 11, in step S1110, the first processor (170) can obtain a three-dimensional breast image from the three-dimensional scanner (200) as a result of a three-dimensional scan of the patient's breast before breast reconstruction surgery. The first processor (170) can obtain a three-dimensional breast image from the three-dimensional scanner (200) for a first posture of the patient that simulates a preset upper body angle of the patient applied during breast reconstruction surgery. The first processor (170) can obtain a three-dimensional breast image for a second posture of the patient in which the upper body angle is vertically transformed using the reference applied when obtaining the three-dimensional breast image for the first posture from the three-dimensional scanner (200). The first processor (170) can obtain a three-dimensional breast image from the three-dimensional scanner (200) for a third posture of the patient in which the body of the patient, who was lying down, is tilted to the side while sitting on the operating table. The first processor (170) may also calculate the contour and volume of the breast corresponding to one or more of the first posture, second posture and third posture from the three-dimensional breast image.

[0101] In step S1120, the first processor (170) can obtain body contour information of a patient positioned on an operating table and upper body angle information of the patient from the XR device (300).

[0102] In step S1130, the first processor (170) can generate a guide image by overlaying a 3D breast image onto the patient's body contour information based on the patient's upper body angle information. The first processor (170) can generate a guide image by overlaying a 3D breast image onto the patient's body contour information based on the patient's body contour information. The first processor (170) generates a patient's body contour extraction result from the 3D breast image, compares the patient's body contour information with the patient's body contour extraction result, and can decide whether to allow or disallow the overlay based on the similarity between the patient's body contour information and the patient's body contour extraction result as a result of the comparison. The first processor (170) can decide to allow the overlay based on the similarity between the patient's body contour information and the patient's body contour extraction result as a result of the comparison being greater than or equal to a reference value, and can generate a message indicating that the overlay has been allowed and transmit it to the XR device (300). The first processor (170) can determine whether to disallow the overlay based on the result of comparison that the similarity between the patient's body contour information and the patient's body contour extraction result is less than a reference value, and can generate a message indicating that the disallowance of the overlay has been determined and transmit it to the XR device (300).

[0103] In an optional embodiment, the first processor (170) may generate a guide image corresponding to the patient's upper body angle information and the patient's three-dimensional breast image by using a first deep neural network model that is pre-trained to generate a guide image to be overlaid on the patient's body contour information using the patient's upper body angle information and the patient's three-dimensional breast image. Here, the first deep neural network model may be a model trained in a supervised learning manner by training data that takes as input a three-dimensional breast image corresponding to one or more of a plurality of first upper body angles that are vertically transformed based on the patient lying down and a plurality of second upper body angles that are horizontally transformed on the first upper body angle, and a guide image as a label.

[0104] In an optional embodiment, the first processor (170) may generate three-dimensional geometric data for a fourth posture as a different posture or a missing posture using an artificial intelligence model. From FIG. 3, three-dimensional breast images for the first to third postures may be acquired before breast reconstruction surgery to produce three-dimensional geometric data for breast reconstruction. However, during actual breast reconstruction surgery, there may be cases where three-dimensional geometric data corresponding to three-dimensional breast images for a different posture or a missing posture, other than the first to third postures, is required. Accordingly, the first processor (170) may generate three-dimensional geometric data corresponding to a three-dimensional breast image for the fourth posture using a second deep neural network model that is pre-trained to predict three-dimensional geometric data for breast reconstruction using breast-related information. Here, the second deep neural network model may be a model trained in a supervised learning manner using training data that takes breast-related information as input and includes posture information including the patient's upper body angle and three-dimensional geometric data as labels. In this embodiment, the patient's breast-related information may include one or more of a two-dimensional breast image, a three-dimensional breast image, a mammography video, depth data, and three-dimensional geometry data. The first processor (170) may obtain the patient's body contour information and the patient's upper body angle information located on the operating table from the XR device (300). The first processor (170) may generate a guide image by overlaying the three-dimensional geometry data for the fourth posture onto the patient's body contour information.

[0105] FIG. 12 is a flowchart illustrating a method for generating a guide image for breast reconstruction surgery according to another embodiment of the present disclosure. In the following description, parts that overlap with the description of FIG. 1 to FIG. 11 will be omitted. The method for generating a guide image for breast reconstruction surgery according to the present embodiment will be described assuming that the XR device (300) performs the task at the second processor (310) with the help of surrounding components.

[0106] Referring to FIG. 12, in step S1210, the second processor (310) can obtain a three-dimensional breast image as a result of a three-dimensional scan of the patient's breast before breast reconstruction surgery from the three-dimensional scanner (200). The second processor (310) can obtain a three-dimensional breast image from the three-dimensional scanner (200) for a first posture of the patient that simulates a preset upper body angle of the patient applied during breast reconstruction surgery, a second posture of the patient in which the upper body angle is vertically transformed using a standard applied when obtaining the three-dimensional breast image for the first posture, and a third posture of the patient in which the body of the patient who was lying down is tilted to the side while sitting on the operating table. The second processor (310) may also calculate the contour of the breast and the volume of the breast corresponding to one or more of the first posture, the second posture, and the third posture from the three-dimensional breast image.

[0107] In step S1220, the second processor (310) can calculate the body contour information of the patient located on the operating table and the upper body angle information of the patient. To generate the body contour information of the patient located on the operating table, the second processor (310) can recognize the environment of the operating room and collect environmental data using built-in sensors (not shown) and cameras (not shown). The environmental data collected by the second processor (310) may include the body of the patient located on the operating table captured by the built-in camera. The second processor (310) can generate body contour information by utilizing computer vision technology and image recognition algorithms to identify the patient's body parts and extract their contours. Additionally, to generate the upper body angle information of the patient located on the operating table, the second processor (310) may use internally equipped sensors such as accelerometers and gyroscopes. The second processor (310) can calculate upper body angle information of the patient by applying a sensor fusion algorithm, a quaternion filter, marker-based tracking, etc., to the patient's posture and movement data collected by sensors such as an accelerometer and a gyroscope equipped inside.

[0108] In step S1230, the second processor (310) can generate a guide image by overlaying a 3D breast image onto the patient's body contour information based on the patient's upper body angle information. The second processor (310) can generate a guide image by overlaying a 3D breast image onto the patient's body contour information based on the patient's body contour information. The second processor (310) generates a patient's body contour extraction result from the 3D breast image, compares the patient's body contour information with the patient's body contour extraction result, and determines whether to allow or disallow the overlay based on the similarity between the patient's body contour information and the patient's body contour extraction result as a result of the comparison. The second processor (310) determines whether to allow the overlay based on the similarity between the patient's body contour information and the patient's body contour extraction result as a result of the comparison being greater than or equal to a reference value, and can generate a message indicating that the overlay has been allowed and transmit it to the XR device (300). The second processor (310) can determine whether to disallow the overlay based on the result of comparison that the similarity between the patient's body contour information and the patient's body contour extraction result is less than a reference value, and can generate a message indicating that the disallowance of the overlay has been determined and transmit it to the XR device (300).

[0109] In an optional embodiment, the second processor (310) may generate a guide image corresponding to the patient's upper body angle information and the patient's three-dimensional breast image by using a first deep neural network model that is pre-trained to generate a guide image to be overlaid on the patient's body contour information using the patient's upper body angle information and the patient's three-dimensional breast image. Here, the first deep neural network model may be a model trained in a supervised learning manner by training data that takes as input a three-dimensional breast image corresponding to one or more of a plurality of first upper body angles that are vertically transformed based on when the patient is lying down and a plurality of second upper body angles that are horizontally transformed on the first upper body angle, and a guide image as a label.

[0110] In an optional embodiment, the second processor (310) may generate three-dimensional geometric data for a fourth posture as a different posture or a missing posture using an artificial intelligence model. From FIG. 3, three-dimensional breast images for the first to third postures may be acquired before breast reconstruction surgery to produce three-dimensional geometric data for breast reconstruction. However, during actual breast reconstruction surgery, there may be cases where three-dimensional geometric data corresponding to three-dimensional breast images for a different posture or a missing posture, other than the first to third postures, is required. Accordingly, the second processor (310) may generate three-dimensional geometric data corresponding to a three-dimensional breast image for the fourth posture using a second deep neural network model that is pre-trained to predict three-dimensional geometric data for breast reconstruction using breast-related information. Here, the second deep neural network model may be a model trained in a supervised learning manner using training data that takes breast-related information as input and includes posture information including the patient's upper body angle and three-dimensional geometric data as labels. In this embodiment, the patient's breast-related information may include one or more of a 2D breast image, a 3D breast image, a mammography video, depth data, and 3D geometry data. The second processor (310) may obtain the patient's body contour information and the patient's upper body angle information located on the operating table from the XR device (300). The second processor (310) may generate a guide image by overlaying the 3D geometry data for the fourth posture onto the patient's body contour information.

[0111] The embodiments according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded on a computer-readable medium. In this case, the medium may include a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and a hardware device specifically configured to store and execute program instructions, such as a ROM, RAM, or flash memory.

[0112] Meanwhile, the above-mentioned computer program may be one specifically designed and configured for the present invention, or one known and available to those skilled in the art of computer software. Examples of computer programs may include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0113] In the specification of the present invention (particularly in the claims), the use of the term "above" and similar descriptive terms may be in both singular and plural. Furthermore, where a range is described in the present invention, it is to include an invention to which individual values ​​belonging to said range are applied (unless otherwise stated), and this is equivalent to describing each individual value constituting said range in the detailed description of the invention.

[0114] Unless explicitly stated or contrary to the order of the steps constituting the method according to the present invention, said steps may be performed in a suitable order. The present invention is not necessarily limited by the order in which said steps are described. The use of all examples or exemplary terms (e.g., etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is not limited by said examples or exemplary terms unless limited by the claims. Furthermore, those skilled in the art will understand that various modifications, combinations, and changes may be made according to design conditions and factors within the scope of the claims or equivalents to which they are added.

[0115] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all scopes equivalent to or equivalently modified from the claims set forth below, as well as the claims set forth below, shall be considered to fall within the scope of the concept of the present invention. Explanation of the symbols

[0116] 100: Guide image generating device 200: 3D Scanner 300: XR device 400: Network

Claims

Claim 1 A guide image generation device for breast reconstruction surgery, comprising: a processor; and a memory operably connected to the processor and storing at least one code executed by the processor, wherein the memory stores a code that, when executed through the processor, causes the processor to acquire a 3D breast image as a 3D scan result of a patient's breast prior to breast reconstruction surgery from a 3D scanner, acquire body contour information of a patient located on an operating table and upper body angle information of the patient from an XR device, and generate a guide image by overlaying the 3D breast image on the patient's body contour information based on the patient's upper body angle information, wherein the XR device includes an XR marker, and the processor acquires the patient's upper body angle information by applying a marker-based tracking method based on the XR marker, wherein the processor acquires the patient's upper body angle information by processing data collected by an accelerometer or gyroscope sensor provided inside the XR device through a sensor fusion algorithm or a quaternion filter. Claim 2 A guide image generation device for breast reconstruction surgery, wherein the memory stores a code that causes the processor to acquire, when acquiring the three-dimensional breast image, acquire a three-dimensional breast image for a first posture of a patient simulating a preset upper body angle of the patient applied during breast reconstruction surgery from the three-dimensional scanner, acquire a three-dimensional breast image for a second posture of a patient in which the upper body angle is vertically transformed using a standard applied when acquiring the three-dimensional breast image for the first posture from the three-dimensional scanner, and acquire a three-dimensional breast image for a third posture of a patient in which the body of a patient lying down is tilted in a first direction or a second direction while the operating table is seated from the three-dimensional scanner. Claim 3 A guide image generation device for breast reconstruction surgery, wherein the memory further stores a code that causes the processor to calculate a breast contour and a breast volume corresponding to one of the first posture, the second posture and the third posture from the three-dimensional breast image after acquiring a three-dimensional breast image for the third posture. Claim 4 A guide image generation device for breast reconstruction surgery, wherein the memory stores code that causes the processor to generate a guide image by overlaying the three-dimensional breast image on the patient's body contour information based on the patient's body contour information when generating the guide image. Claim 5 A guide image generation device for breast reconstruction surgery, wherein the memory stores a code that causes the processor, when generating the guide image, to generate a patient's body contour extraction result from the three-dimensional breast image, compare the patient's body contour information with the patient's body contour extraction result, and determine whether to allow or disallow the overlay based on the similarity between the patient's body contour information and the patient's body contour extraction result as a result of the comparison. Claim 6 A guide image generation device for breast reconstruction surgery, wherein the memory stores code that causes the processor to determine whether to allow the overlay and generate a message indicating that the overlay has been allowed, based on the result of the comparison that the similarity between the patient's body contour information and the patient's body contour extraction result is greater than or equal to a reference value, and to determine whether to disallow the overlay and generate a message indicating that the overlay has been disallowed, based on the result of the comparison that the similarity between the patient's body contour information and the patient's body contour extraction result is less than a reference value. Claim 7 A guide image generation device for breast reconstruction surgery, wherein the memory stores code that causes the processor to generate a guide image corresponding to the patient's upper body angle information and the patient's three-dimensional breast image by using a deep neural network model that is pre-trained to generate a guide image to be overlaid on the patient's body contour information using the patient's upper body angle information and the patient's three-dimensional breast image when generating the guide image, and the deep neural network model is a model trained in a supervised learning manner by training data having one or more of a plurality of first upper body angles that are vertically transformed based on when the patient is lying down and one or more of a plurality of second upper body angles that are horizontally transformed on the first upper body angle, and a three-dimensional breast image corresponding to one or more of the first upper body angle and the second upper body angle as input, and the guide image as a label. Claim 8 A guide image generation device for breast reconstruction surgery, wherein the memory further stores code that causes the processor to generate three-dimensional geometric data for a fourth posture using a second deep neural network model that is pre-trained to predict three-dimensional geometric data for breast reconstruction using breast-related information before acquiring upper body angle information of the patient, and the second deep neural network model is a model trained in a supervised learning manner by training data that has posture information including the patient's upper body angle and three-dimensional geometric data as labels, and the memory stores code that causes the processor to generate a guide image in which the three-dimensional geometric data for the fourth posture is overlaid on the patient's body contour information when generating the guide image. Claim 9 A method for generating a guide image for breast reconstruction surgery performed by a processor of a guide image generating device, comprising: a step of acquiring a three-dimensional breast image as a result of a three-dimensional scan of a patient's breast before breast reconstruction surgery from a three-dimensional scanner; a step of acquiring body contour information of a patient located on an operating table and upper body angle information of the patient from an XR device; and a step of generating a guide image by overlaying the three-dimensional breast image on the patient's body contour information based on the patient's upper body angle information, wherein the step of acquiring the patient's upper body angle information comprises acquiring the patient's upper body angle information by applying a marker-based tracking method based on an XR marker included in the XR device, wherein the patient's upper body angle information is acquired by processing data collected by an accelerometer or gyroscope sensor provided inside the XR device through a sensor fusion algorithm or a quaternion filter. Claim 10 In claim 9, the step of acquiring the three-dimensional breast image comprises: acquiring a three-dimensional breast image for a first posture of a patient that simulates a preset upper body angle of the patient applied during breast reconstruction surgery from the three-dimensional scanner; acquiring a three-dimensional breast image for a second posture of a patient in which the upper body angle is vertically transformed using a reference applied when acquiring the three-dimensional breast image for the first posture from the three-dimensional scanner; and acquiring a three-dimensional breast image for a third posture of a patient in which the body of a patient who was lying down is tilted in a first direction or a second direction while the operating table is seated from the three-dimensional scanner, a method for generating a guide image for breast reconstruction surgery. Claim 11 A method for generating a guide image for breast reconstruction surgery, comprising, in addition to the step of acquiring a three-dimensional breast image for the third position, the step of calculating a breast contour and a breast volume corresponding to one or more of the first position, the second position and the third position from the three-dimensional breast image. Claim 12 A method for generating a guide image for breast reconstruction surgery, wherein the step of generating the guide image comprises the step of generating a guide image by overlaying the three-dimensional breast image on the patient's body contour information based on the patient's body contour information. Claim 13 A method for generating a guide image for breast reconstruction surgery according to claim 9, wherein the step of generating the guide image comprises: generating a result of extracting the patient's body contour from the three-dimensional breast image; comparing the patient's body contour information with the result of extracting the patient's body contour; and determining whether to allow or disallow the overlay based on the similarity between the patient's body contour information and the result of extracting the patient's body contour as a result of the comparison. Claim 14 A method for generating a guide image for breast reconstruction surgery, wherein the determining step comprises: determining permission for the overlay based on the result of the comparison that the similarity between the patient's body contour information and the result of extracting the patient's body contour is greater than or equal to a reference value, and generating a message indicating that permission for the overlay has been determined; and determining disallowance of the overlay based on the result of the comparison that the similarity between the patient's body contour information and the result of extracting the patient's body contour is less than a reference value, and generating a message indicating that disallowance of the overlay has been determined. Claim 15 A method for generating a guide image for breast reconstruction surgery according to claim 9, wherein the step of generating the guide image comprises the step of generating a guide image corresponding to the patient's upper body angle information and the patient's three-dimensional breast image by using a first deep neural network model that is pre-trained to generate a guide image to be overlaid on the patient's body contour information using the patient's upper body angle information and the patient's three-dimensional breast image, wherein the first deep neural network model is a model trained in a supervised learning manner by training data having one or more of a plurality of first upper body angles that are vertically transformed based on when the patient is lying down and one or more of a plurality of second upper body angles that are horizontally transformed on the first upper body angle, and a three-dimensional breast image corresponding to one or more of the first upper body angle and the second upper body angle as input, and the guide image as a label. Claim 16 In claim 9, prior to the step of acquiring upper body angle information of the patient, the method further comprises the step of generating three-dimensional geometric data for a fourth posture using a second deep neural network model pre-trained to predict three-dimensional geometric data for breast reconstruction using breast-related information, wherein the second deep neural network model is a model trained in a supervised learning manner by training data having posture information including the patient's upper body angle and three-dimensional geometric data as labels, and the step of generating the guide image includes the step of generating a guide image by overlaying the three-dimensional geometric data for the fourth posture onto the patient's body contour information. Claim 17 A computer-readable recording medium storing a computer program for executing any one of the methods of claims 9 through 16 using a computer. Claim 18 An XR device for generating a guide image for breast reconstruction surgery, comprising: a processor; and a memory operably connected to the processor and storing at least one code executed by the processor, wherein the memory stores code that, when executed through the processor, causes the processor to acquire a 3D breast image as a 3D scan result of a patient's breast prior to breast reconstruction surgery from a 3D scanner, calculate body contour information of the patient positioned on the operating table and upper body angle information of the patient, and generate and output a guide image by overlaying the 3D breast image on the patient's body contour information based on the patient's upper body angle information, wherein the processor calculates the patient's upper body angle information by applying a marker-based tracking method based on an XR marker, and calculates the patient's upper body angle information by processing data collected by an accelerometer or gyroscope sensor provided inside the XR device through a sensor fusion algorithm or a quaternion filter. Claim 19 In claim 18, the memory stores code that causes the processor to generate a guide image by overlaying the three-dimensional breast image on the patient's body contour information based on the patient's body contour information when generating the guide image, an XR device. Claim 20 An XR device according to claim 18, wherein the memory stores code that causes the processor, when generating the guide image, to generate a patient body contour extraction result from the three-dimensional breast image, compare the patient body contour information and the patient body contour extraction result, and determine whether to allow or disallow the overlay based on the similarity between the patient body contour information and the patient body contour extraction result as a result of the comparison.

Images