Projection device for analyzing an object of interest on or above a surface of a subject

Abstract

An apparatus comprising a detector having a detection field and configured to detect at least a portion of a subject in the detection field and to detect objects in the detection field; and a processor configured to identify a command based on the detected objects, wherein the command specifies a region on a surface of the subject; determine attributes of tissues of the subject within the specified region; and output an indication of the determined attributes.

Description

Technical Field

[0001] The present invention relates to an apparatus comprising: a detector having a detection field located in front of a subject and configured to detect an object in the detection field; and a processor configured to determine the attributes of the subject's tissues based on a detected object identification command and output an indication of the determined attributes. Background Technology

[0002] For surgical procedures, it is important to locate features within the patient's body that may be visible, for example, on medical images.

[0003] US 9510914 B2 describes providing surgical guidance and image registration, wherein three-dimensional image data associated with an object or patient is registered to topological image data obtained using a surface topology imaging device.

[0004] US 9646423 B1 describes a method for providing augmented reality in minimally invasive surgery, the method comprising: capturing preoperative image data of a patient's internal organs; capturing intraoperative image data of the internal organs with an endoscope during surgery; registering the preoperative image data and intraoperative data in real time during surgery; tracking the position and orientation of the endoscope during surgery; and augmenting the intraoperative image data captured by the endoscope in real time using rendering of at least a portion of the patient's internal organs registered with the real-time intraoperative image data from the endoscope but outside the field of view of the endoscope.

[0005] WO 2015 / 135985 A1 describes a projection system for projecting images of features on a subject. Intraoperative location markers are set on a first body surface, and a movable first detector with a line of sight is provided.

[0006] A drawback of existing technologies is the need for location markers on the subject. Additionally, existing techniques may involve calibrating the projected image with the detected image, which can be cumbersome and / or reduce the accuracy and / or quality of the projected image. Furthermore, there is a need for an improved imaging tool capable of providing relevant information about the subject. Summary of the Invention

[0007] The present invention seeks to provide an apparatus for detecting one or more objects of interest on or above a subject's surface, specifying a region on the subject's surface, and determining one or more properties of tissue within the specified region.

[0008] According to one aspect of the invention, an apparatus includes: a detector having a detection field located in front of a subject and configured to detect at least a portion of the subject in the detection field and to detect objects in the detection field; and a processor configured to: identify a command based on the detected objects, wherein the command specifies a region on the surface of the subject; determine attributes of tissues of the subject within the specified region; and output an indication of the determined attributes.

[0009] The apparatus disclosed herein can be designed to detect objects within the detector's field of view. The detected object can be external to, in front of, or on the surface of the subject. Depending on the detected object, the processor can identify internal commands specifying a region on the subject's surface; the specified region may also include tissue beneath the subject's surface, such as internal tissues and / or organs. After specifying the region on the subject's surface, the processor is configured to determine the properties of the subject's tissues within the specified region. In this context, determining the properties of the tissues refers to determining the properties of the subject's superficial and / or deep tissues. Therefore, a property is any measurable trait or characteristic that the tissues in the specified region may possess.

[0010] The processor outputs indications of the determined attributes, such as on a display or by projection onto the subject's surface, or by means of auditory signals. In this way, the determination and output of attributes can improve the physician's understanding of the subject's tissues and / or body composition or characteristics during surgery, particularly for attributes that are difficult to visually assess or are completely imperceptible. Typically, the device's ability to determine and output attributes of subject tissues within a designated area improves the precision of medical procedures by providing information to assist the practitioner in making well-founded decisions. The device is configured to detect objects outside the subject (e.g., the surgeon's hand or instruments) or objects belonging to the subject, such as anatomical features (e.g., organs) and / or physiological characteristics, as further explained below.

[0011] The apparatus as claimed in any of the preceding claims further includes a projector configured to project indications of the designated area and / or indications of determined properties of the tissue onto the subject's surface. The projector acts as a visual aid, enabling easy identification of key areas of the subject by directly projecting indications of the designated area and tissue properties onto the subject's surface. This visual overlay facilitates improved instrument handling, ensuring precise placement and manipulation. The projector reduces the need for the practitioner to divert their attention to an external screen, allowing the practitioner to visualize properties without shifting their focus away from the current surgical environment.

[0012] The object may be external to the subject, and the object may indicate points that define the shape of the designated region. The processor may be configured to: detect consecutive points indicated by the object on the surface of the subject as the object moves relative to the subject; and connect the consecutive points to determine the shape of the designated region. In this way, the subject's tissue within the designated region is precisely defined by the object indication provided by the detected external object. This improves the efficiency of the device in providing information to the user on demand. Furthermore, connecting consecutive points enables real-time reconstruction of the shape of the designated region, which helps to accurately determine the properties of the subject's tissue within the designated region.

[0013] The projector can be configured to project an indication of the shape of a designated area once points on the subject's surface have been detected. This real-time representation of the designated area enhances user understanding and communication with the device. Users, such as physicians, can interactively draw or manipulate the shape of the designated area to ensure accuracy and guide it throughout the procedure. This feature not only increases user confidence but also facilitates effective communication and collaboration between the user and the device, ultimately improving the quality and safety of the surgery.

[0014] The shape of the designated region can be closed or nearly closed, wherein determining tissue properties can include determining the properties of superficial and / or deep tissues of the subject on and / or below the region defined by the designated region. The shape of the designated region can be open, closed, or nearly closed, and they can be used for different purposes. By defining a closed or nearly closed shape, the processor determines tissue properties specific to the closed region, which the processor identifies using attributes such as area, volume, etc. An open shape can be used to enable the processor to recognize another command for alternative calculations, such as linear calculations (e.g., length, curvature, angle, etc. of the open shape) or to provide a request for advice during medical procedures; for example, an open shape can request the device to provide advice on closing or treating a wound.

[0015] The detector can be configured to detect at least a portion of a hand or instrument as an object. This enables intuitive interaction through hand gestures and postures, simplifying communication between the user and the device and making it more user-friendly. Furthermore, since users can draw without touching the subject's surface, this promotes hand-free operation and reduces the risk of cross-contamination at the operating area.

[0016] In other scenarios, the object is part of the subject's body, and the processor is further configured to determine the object's shape within a specified area. This enhances user interaction by eliminating the need for manual shape input or complex procedures, making the system more user-friendly and efficient. Furthermore, the processor's ability to determine the object's shape when it is part of the subject's body provides the advantage of enabling convenient and automatic shape recognition without user input, which is particularly convenient when recognizing multiple objects. Additionally, automatic shape determination can provide highly accurate measurements and annotations of body tissues and structures.

[0017] Tissue attributes can include one or more of the following: name, temperature, size, area, volume, location, and tissue type. By including a wide variety of attributes, the device enables a comprehensive understanding of tissue characteristics. Attributes such as temperature, size, area, and volume help practitioners obtain specific and quantifiable data. This precision ensures that interventions are tailored to each patient's unique anatomical attributes. For example, attributes such as the location of body parts aid spatial awareness, allowing physicians to precisely locate tissue areas for targeted actions, minimizing invasiveness and potential collateral damage. Attributes such as tissue type and name enhance the differentiation and identification of specific tissue structures, helping to distinguish between different tissue types to avoid errors and complications during surgery. Attributes such as location and size, when presented in real time, can guide practitioners during interventions. This advantage directly aligns with the dynamic nature of medical procedures, providing timely and relevant insights for immediate actions.

[0018] The processor can be configured to determine the properties of a subject's visceral tissue, epithelial tissue, connective tissue, adipose tissue, muscle tissue, nervous tissue, or abnormal tissue. Tissue type differentiation also enhances precision during surgery to prevent damage to non-targeted tissues, such as when removing tumors from internal organs. Furthermore, different tissue types often require different treatment strategies. Identifying the tissue type ensures that treatment and surgical procedures are tailored to the characteristics of the tissue being treated.

[0019] The properties of an organization can be determined based on real-time measurements of a specified region and / or based on mapping the specified region onto a preoperative image containing data representing the subject.

[0020] Determining tissue properties based on real-time measurements of a designated area provides immediate insights during medical procedures and enables practitioners to receive immediate feedback. Real-time measurements are particularly valuable for assessing dynamic aspects of tissue, such as blood flow, temperature, or conductivity, which change rapidly during surgery. Real-time measurements can be performed by a device, such as a detector. Alternatively, real-time measurements can be performed by an external device that communicates with a processor.

[0021] Determining tissue properties based on preoperative images allows for the acquisition of more detailed information or attributes in anatomical structures that cannot be captured by real-time measurements, which typically include additional structural and / or functional information about both superficial and deep tissues. The combination of real-time measurements and preoperative images enhances the accuracy and effectiveness of medical interventions by providing both immediate feedback and detailed anatomical insights. In some examples, real-time measurements include intraoperative imaging devices that acquire data representing the subject. Imaging devices may involve one or more medical imaging techniques, such as fluoroscopy, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound imaging, positron emission tomography (PET), and single-photon emission computed tomography (SPECT).

[0022] Preoperative image datasets can include images from one or more medical imaging techniques, such as those mentioned above. Different imaging techniques help characterize different types of tissue. CT scans can be used to visualize bone and dense structures, while MRI can be used to capture soft tissue details. Ultrasound provides real-time imaging without radiation exposure, making it suitable for a wide range of applications. PET and SPECT are essential for functional imaging and detecting metabolic activity. By combining these different imaging modalities, practitioners can gain a comprehensive three-dimensional understanding of the subject. Furthermore, combining imaging datasets can help detect abnormalities that might be missed when relying on a single imaging source, ultimately leading to more accurate diagnoses and treatment plans. Finally, if multiple imaging techniques are used, the device can assist in a wide variety of medical procedures, such as surgical procedures, radiation therapy planning, or interventional procedures.

[0023] An organization's attributes can be determined using identification algorithms that include artificial intelligence (AI).

[0024] AI algorithms can provide data-driven insights that enhance the accuracy and efficiency of determining tissue attributes. Furthermore, their adaptability ensures that the device continuously improves the accuracy of its attribute determination over time. This results in a more robust device that is adept at identifying and characterizing different tissue types, including anomalous or malformed tissues.

[0025] Tissue properties can be determined instantaneously and / or over time. The option to determine tissue properties before surgery begins supports preoperative planning. This advantage enables practitioners to plan interventions. Simultaneously, the ability to determine properties over time provides dynamic insights during surgery, offering real-time updates on tissue properties as the procedure progresses, aiding decision-making and intervention adjustments.

[0026] Detectors may include image sensors and / or position sensors to detect the location of an object. Image sensors in a detector help recreate visual indications later and allow the device to capture real-time visual information, thus facilitating output to the practitioner with direct visual feedback during surgery. Position sensors can provide a more accurate location and orientation of the object in the detection field, which is advantageous during medical procedures requiring high accuracy. The combination of image and position sensors facilitates the integration of visual and spatial information, improving accuracy. Combining image and position sensors allows the device to integrate visual and spatial data, which can be valuable for guiding surgical instruments or ensuring that treatment is delivered to precise targets.

[0027] The device may further include a memory for storing identified attributes of the organization for future reference and / or analysis. The memory facilitates storing attribute-related information for subsequent reference and / or in-depth analysis. The memory may also retain instructions for the processor to follow.

[0028] The detector can detect multiple objects simultaneously, and the processor can be configured to identify commands and determine attributes individually for each detected object. Detecting multiple objects (such as instruments or different tissues and organs) is helpful in medical procedures when there is more than one instrument in the detection field or when more than one attribute needs to be determined during the procedure. For example, during surgery, the device can monitor both the target organ and surrounding tissues simultaneously.

[0029] The processor can be further configured to analyze tissue properties and provide surgical recommendations based on the analysis. By providing recommendations based on attribute analysis, the device contributes to improved surgical outcomes. If the analysis of tissue properties and the provision of surgical recommendations are data-driven, the recommendations can be based on objective data, thereby improving the accuracy and effectiveness during medical procedures.

[0030] The processor is configured to generate a graphical representation of the identified attributes and to project this representation onto the subject's surface. The graphical representation of the identified attributes allows the user to understand complex attribute information in a visual and easily understandable way. Furthermore, the projected graphical representation allows the user to see the immediate effects of their intervention. Attached Figure Description

[0031] The invention will now be explained in more detail with reference to several exemplary embodiments in the accompanying drawings. The drawings are illustrative and may not be drawn to scale. Similar items may be labeled using the same reference numerals throughout the drawings. In the drawings: Figure 1A The apparatus according to this disclosure is shown schematically; Figure 1B schematically shown Figure 1AThe device in which the detected objects are attributed to the subject.

[0032] Figure 1C schematically shown Figure 1A The device in which the detected object is outside the subject.

[0033] Figure 2A Another apparatus, including a projector, is shown schematically according to this disclosure.

[0034] Figure 2B schematically shown Figure 2A The device projects an indication of the properties of the subject's tissues.

[0035] Figure 2C schematically shown Figure 2A The device assists during medical procedures. Detailed Implementation

[0036] In medicine, various difficulties arise regarding patient information identification, particularly during semi-invasive and invasive medical procedures, such as surgical procedures and biopsies. For example, identifying and locating a patient's vital signs, such as temperature, blood pressure, and heart rate; blood flow; burns; infections; rheumatism; cancer cells; microorganisms; etc. For instance, before or during surgery, it may be advantageous to identify objects, such as parts of the patient's vascular system and / or internal tissues or organs, and to determine the properties of these objects to provide the practitioner with more information about the patient's tissues. Furthermore, it may be advantageous to detect or identify other objects outside the patient's body to assist the practitioner before or during surgery.

[0037] The device disclosed herein is designed to detect objects within the detector's field of view. The detected object may be outside, in front of, or on the surface of the subject. The device may be positioned at a reasonable working distance from the subject, for example, up to 3 meters, so as not to interfere with medical procedures performed on the subject.

[0038] It should be noted that the term "detector field of view" in this document can refer to the area or volume that the detector can detect. The field of view may have a cone shape, with its center (apex) at the detector. It should also be noted that the detector may be able to detect radiation and / or signals emitted or reflected by objects outside the subject or by objects on or below the subject's surface. For example, detecting skin burns, at least a portion of internal organs, or vascular structures below the subject's surface.

[0039] It should be further noted that the surface of the subject (e.g., a patient's skin) is typically not a flat surface and may include three-dimensional contours that follow the curvature of the body and organs (e.g., it may be curved and may have upright walls, recesses, etc.). The detector can detect objects within the field of view in a two-dimensional (2D) manner.

[0040] Figure 1A The apparatus 102 according to this disclosure is illustrated schematically. The subject 114 is not part of the image projection apparatus 102. The apparatus 102 includes a detector 104 configured to detect objects external to the subject (e.g., a surgeon's hand or instruments) or objects belonging to the subject, such as anatomical features (e.g., organs) and / or physiological features. The detector 104 can generate a detected signal and / or image based on a detected radiation beam 110 from the detector's field of view 111. The detected radiation beam 110 can be emitted or reflected by the detected object. The apparatus 102 may include more than one detector 104, and each detector can generate a detected image and / or signal. The detector 104 can be a position detector or an image detector and may include a sensor. For example, the image sensor may be a complementary metal-oxide-semiconductor (CMOS) detector that detects visible light beams. It should be understood that other sensors may also be appropriate depending on the wavelength range of the detected radiation beam 110. Additionally or optionally, suitable filters may be provided such that the detector 104 is configured to sense radiation within a specific bandwidth.

[0041] Detector 104 may include a light emitter configured to emit radiation into the detector's field of view and detect light reflected from the surface of an object and / or subject upon exposure to the emitted light. The emitted light may be configured for a specific frequency. This can be helpful in imaging techniques, including but not limited to fluorescent dyes, laser speckle contrast imaging (LSCI), and laser Doppler imaging. For example, certain bacteria or tissues may emit light at a certain frequency upon exposure to light from the light emitter. In another example, the apparatus may be configured to perform LSCI using a laser that emits light. Additionally, the emitted light may excite certain bacteria or tissues of interest upon exposure to the emitted light.

[0042] In a specific example, detector 104 is a laser Doppler imaging (LDI) system. The LDI system may include a laser exciter, a laser, and a camera configured to detect changes over time. The LDI system can be configured to detect blood vessels as objects and subsequently determine the movement of red blood cells within the blood vessels by reflecting specific wavelengths. In another example, when using laser speckle contrast imaging (LSCI), the laser reaches the object (in this case, a blood vessel), and moving blood cells generate a Doppler component in the reflected light. LSCI can also be configured to identify burns as objects and determine the depth of burns on the subject. Red blood cells can scatter coherent light to cause blurring of the speckle pattern. Red blood cell movement can be determined based on the degree of blurring in the detected image. Burn depth can be determined based on the presence of red blood cell movement.

[0043] Typically, the surface of subject 114 is the skin surface. However, in some applications of device 102, the surface of subject 114 may alternatively be the internal surface of the subject. For example, after surgery has begun and subject 114 has an open cavity, the subject can be detected from within the open cavity.

[0044] Device 102 may include processor 103 configured to recognize a command based on a detected object. Depending on the detected object, the command specifies a region on the surface of the subject. When the object is a body part belonging to the subject, the processor recognizes the command based on the detected body part to specify a region associated with the detected body part.

[0045] Figure 1B schematically shown Figure 1A The device, wherein the detected object 130 belongs to subject 114. Figure 1BIn one example, processor 103 would recognize a command to designate a region on the surface of subject 114, wherein a blood vessel on or within the heart is the detected object 130. In the depicted example, the designated region 132 is the heart of subject 114. In another example, if the detected object is a burn, the processor would recognize a command to designate a burned area of ​​skin as the designated region. In a further example, if the object is an internal organ at least partially exposed from a cavity of the subject, the processor could designate the surface of the internal organ in front of the detector as the designated region. In another example, if the object is a subcutaneous segment of skin (e.g., an open skin flap), the recognized command would designate the subcutaneous surface of the skin containing epithelial tissue, adipose tissue, vascular tissue, etc. In yet another example, the detected object is a marker on the subject's surface. The marker can be a physical or optical marker, such as a fluorescent marker used to identify abnormal cells (e.g., cancer cells), and the designated region could be an organ of the subject with a tumor.

[0046] If the detected object is outside the subject 114 (e.g., an instrument), the processor 103 will specify a region on the surface of the object indicated by the external object. For details regarding the detected object, refer to [reference needed]. Figure 1C Describe it.

[0047] After a region is designated on the subject's surface, processor 103 is configured to determine the properties of the subject's tissue within the designated region. For example, the tissue can be superficial tissue, vascular tissue, internal organ tissue, or any other tissue within the designated region. Therefore, a property is any measurable trait or characteristic that the tissue within the designated region may possess. For example, a property could be the superficial shape, temperature, or surface area (which can be expressed, for example, in square centimeters) of a burn, internal organ, or subcutaneous skin segment. 2 (Measured in units) or volume (which can be, for example, cubic centimeters cm) 3 (Measured in units). Attributes can also be the position of an object relative to another identified object or relative to another part of the subject. For example, such as... Figure 1BAs shown, attributes can be the location, length, or blood flow of blood vessels in the heart. Attributes can also be the type and / or composition of tissue within a specified area. For example, an attribute can be one or more types of tissue, such as visceral tissue, epithelial tissue, connective tissue, adipose tissue, muscle tissue, nerve tissue, or abnormal tissue. In this context, abnormal tissue can be inflamed, infected, or damaged tissue, or abnormally growing tissue (e.g., a tumor). The attribute to be determined can be selected by the user through a user interface, or can be associated with the characteristics of a detected object, command, or detector. Attributes can also be automatically selected by the device depending on the ongoing medical procedure, which can also be determined by the processor based on signals detected by the detector. Those skilled in the art will understand that the determined attribute is directly related to the type of detector used. In addition to the LDI and LSCI mentioned above, detector 104 can detect and measure (near) infrared, ultrasound, fluorescence fluoroscopy, photoacoustic, electrical impedance, spectral, and magnetic signals / images in real time. Alternatively or additionally, detector 104 can be an image detector, such as a visible light camera, and processor 103 can be configured to determine tissue attributes using image processing and / or computer vision algorithms. For example, processor 103 may use artificial intelligence-based recognition and / or computer vision algorithms to continuously or intermittently scan the detection field of view to detect objects and recognize commands based on the detected objects.

[0048] Alternatively or additionally, the processor 103 determines the tissue properties based on a mapping algorithm that maps a designated region 130 onto a preoperative image 120 containing data representing the subject 114. These images can be acquired using any medical imaging technique known in the art, such as computed tomography, magnetic resonance imaging, ultrasound imaging, positron emission tomography, single-photon emission computed tomography, etc. The preoperative image 120 can simplify, supplement, or enhance the property determination based on measurements from the detector 104 to provide more precise or detailed information about the tissue within the designated region 130. For example, when the detected object is a burn, the volumetric properties of the burn tissue can be supplemented with preoperative images to identify further damage beneath the skin. Similarly, preoperative image 120 can provide a more precise location of detected blood vessels of the heart relative to the subject's surface. In an example of a subcutaneous segment of skin being detected, preoperative image 120 can provide subcutaneous fat volume (cm²). 3 Information from the spectral measurements. In another example, preoperative images (such as PET scans) can supplement spectral measurements to identify the location and composition of the tumor, especially when the physician cannot visually perceive the tumor during surgery.

[0049] In applications where the detector is an image detector, the image may be used by processor 103 to determine the shape and color of attributes, for example, to identify and classify detected objects based on preoperative image 120. For instance, the image detector may detect several organs or tissues in an open cavity (a designated area) and generate an image. Processor 103 may then map the image onto an MRI scan to identify one or more tissues or organs in the image. In this example, the attribute may be the name of an anatomical structure or organ (e.g., aorta, left lung, etc.) within the designated area 132.

[0050] As previously described, the object detected by detector 104 can be external to, in front of, or on the surface of the subject. Depending on the detector, the object can be in or under the subject's surface. Typically, objects on / in / under the subject's surface are objects belonging to the subject, such as body parts, organs, skin segments, etc., as described above. Objects external to or in front of the subject's surface are generally likely external to the subject and can be detected to allow device 102 to interact with, for example, a practicing physician or robot performing medical procedures. Examples of external objects can include surgical and medical instruments held by a person (practicing physician), such as pens, probes, forceps, scalpels, and other objects, such as human hands or end effectors of robots, such as grippers, tweezers, etc.

[0051] Figure 1C schematically shown Figure 1A The device, wherein the detected object is outside the subject. As described above, when the object is a body part belonging to the subject, the processor identifies a command for specifying a region associated with the detected object based on the detected body part. In the case where the object is outside the subject 114, the processor 103 will identify a command for specifying a region on the subject's surface based on the external object. Figure 1C As shown, if object 130 is a hand over the surface of subject 114, processor 103 will recognize a command to specify the area on the subject's chest that the hand is pointing to. The area of ​​the specified region 132 can be predefined or adjusted by the user depending on the properties of the tissue to be identified within the specified region. For example, if the detected object 130 is a hand, the specified region 130 can be the area of ​​skin surface that the hand is pointing to, which may contain a wound or burn to be treated. Subsequently, the properties of the tissue can be the affected area of ​​the skin (cm²). 2 In another example, if the scalpel (object) is positioned above the heart, the designated area can span the area where the surgeon will make the incision for cardiac surgery. However, if the instrument is an electrocautery pen, the designated area can be restricted to a narrower area for electrocautery, for example, to stop bleeding from a blood vessel. Subsequently, it can be referred to as previously... Figure 1A and1B The attributes are determined as described.

[0052] The tissue of the subject within a designated area can be more precisely defined by object indicators provided by detected external objects. Object indicators can define the designated area as an open or nearly closed shape drawn within the detector's detection field of view. Figure 1C An externally detected object 130 is shown, which indicates the shape of a designated region 132. The detected object 130 (in this case, a hand) has endpoints, i.e., fingers, which are points indicating the shape of the designated region 132. The processor 103 can also be configured to recognize hand gestures to more accurately identify commands and the designated region 132. If the hand moves within the detector's detection field of view 111 to continue indicating the shape of the designated region 132, the processor 103 detects consecutive points indicated by the object 130 as the hand moves; and connects these consecutive points to determine the shape of the designated region 132. The shape of the designated region 132 can be a closed shape or a nearly closed shape, and the properties of the superficial and / or deep tissues defined by the designated region 132 can be determined according to the shape as referenced above. Figure 1A and Figure 1B The wound is determined as described. In this example, a designated area 132, indicated by the hands of a practicing physician, surrounds the boundary of the wound. Subsequently, the processor 103 can provide more precise attributes of the wound, such as its depth or severity.

[0053] Alternatively, object 130 may not indicate a closed shape for a specified area, but rather an open shape for follow-up to identify new commands, thereby providing advice to the practicing physician. For example, when a surgeon is about to make an incision for cardiac surgery, and the indicated shape is an open shape comprising an almost vertical line, the processor can provide advice on where to make the incision and its extent. In another example, if the instrument is a suture needle, and the shape is S-shaped or Z-shaped, the processor can provide advice on where to insert the needle into the skin to complete the suture. As previously mentioned, the processor can use real-time or preoperative images / signals to determine tissue properties and can also provide advice. For example, the processor can use preoperative images to determine the position of the sternum relative to the subject's surface or the depth of the wound. In this way, the processor can provide advice based on information that a practicing physician might not easily observe during surgery. It will be understood that the processor can be configured to provide advice regardless of whether the detected object belongs to the subject.

[0054] After determining the properties of the subject's tissues located within the designated area, processor 103 outputs an indication of the determined properties. The indication can be of any nature, which the physician will recognize to understand the determined properties. For example, the output can be auditory, such as a voice message, or visual.

[0055] Device 102 may include a screen or display to provide a visual indication of the identified attributes. Alternatively, the device includes a projector configured to project a projected image, including an indication of the identified attributes of the tissue, onto the surface of subject 114. Optionally, the projector may also project indications of designated areas to allow a physician to better identify tissue and designated areas on the subject's surface.

[0056] Figure 2A The apparatus 202 according to this disclosure is schematically shown, which includes a projector 106. The projector 106 disclosed herein is generally designed to project images onto the outer surface of a subject 114. Therefore, the projector 106 can be located at a reasonable working distance from the subject 114, for example, up to 3 meters. The projector 106 can generate a projection beam 112 from the field of view 115 (projection field) of the detector. It should be noted that the field of view of the projector 106 herein can refer to the area on the surface of the subject 114 where the projector 106 can project a projected image. It should further be noted that the field of view of the detector 104 and the field of view of the projector 106 can overlap. The field of view of the projector 106 can also have a conical shape, with its center (apex) at the projector 106. While the detector 104 can detect objects anywhere within the volumetric field of view defined by the space between the detector and the subject's surface, the image projected by the projector 106 is only visible when the beam 112 reaches the subject's surface. Therefore, the projected image can be a 2D image projected onto the surface within the field of view.

[0057] Because the projected image is projected from the (exact or approximately) position of detector 104, any deformation caused by the curvature of the surface can be avoided or significantly reduced, thereby reducing or eliminating the need for image transformations to correct such deformations. Alternatively, the image can be corrected for the 3D surface on which the projected 2D image is projected. Device 102 may further include a 3D sensor (not shown), such as a time-of-flight camera, a stereo camera, etc., to adapt the projected image to the 3D contours of the subject's surface.

[0058] Processor 103 can be configured to generate a graphical representation of an object, a specified region, or a defined attribute, and to cause a projector to project the graphical representation onto the surface of the subject. As previously described, Figure 1B The detector 104 has detected at least a portion of the blood vessel as the detected object 130 and designated the region 132 of the heart. Figure 2B schematically shown Figure 2A The device detected a connection with Figure 1BThe image projectes an indication of the attributes of the subject's tissue, similar to the object in the image. Attribute 134, namely the location and length of blood vessels, can be indicated by the projector 106. The projected image representing blood vessels (generated by beam 112) can show a graphic highlighting blood vessels or the heart. When the detected object is a burn, the projected image can project the boundary lines of the damaged tissue (attribute). If the detected object is an organ with abnormal tissue, the projected image can indicate which part of the organ contains the most abnormal tissue (attribute).

[0059] In some embodiments, the processor can provide recommendations based on detected objects and their attributes, without requiring instructions to project a specified region. The processor identifies commands associated with the detected objects and performs attribute analysis to provide recommendations. For example, Figure 2C schematically shown Figure 2A The device assists during medical procedures. In this case, the detected object 130 is a biopsy needle, and the designated area 132 corresponds to the skin area where the physician plans to insert the needle. The processor 103 determines the locational attributes of the target tissue (e.g., liver) within the designated area 132. It then outputs a suggestion to the physician regarding precise needle placement. This suggestion is conveyed via a projected image that may display a cross or dot at the suggested insertion point 135. The projector 106 assists the physician by projecting the current depth and angle of the needle, as well as indications of the suggested depth and angle. This assistance is achieved by dynamically changing the color and / or size of the shape based on factors such as the distance from the image to the target tissue or the needle angle.

[0060] Projected images can be aligned with structures beneath the subject, such as blood vessels, internal organs, or cancerous tissue. When a projected image is projected onto the subject's surface, a graphical representation of the attributes or designated area (showing the structures beneath) is projected onto the subject's surface. This allows practitioners to view information directly on the patient without the need for a separate display screen.

[0061] Optionally, processor 103 can map real-time measurements from detector 104 to pixels of a projected image. Detected signals can be transformed into pixels of a projected image. Similarly, device 102 can use processor 103 to map pixels of preoperative image 120 to pixels of a projected image. In this way, each pixel of the projected image can be a visual representation of preoperative or real-time information from detector 104. For example, if processor 103 has identified an organ (object) in an open cavity (designated area) through real-time measurements, processor 103 can determine the size (attribute) of the organ using a preoperative MRI scan image. Subsequently, processor 103 can output an indication of the organ's size, such as shape or redness, using projector 106, which will assist a physician in identifying inflammation. In the case of subcutaneous tissue, such as fat volume (attribute) in a specific designated (subcutaneous) area, an indication of the subcutaneous fat volume can be provided as a projection onto the subject's surface in cm. 3 The number can be expressed as a unit, or it can be provided as a colored area on the skin surface, where the intensity of the color can indicate the thickness of the fat layer.

[0062] The projected image can be based on multiple images / signals detected by detector 104. For example, if device 102 includes a thermal image detector and an infrared image detector, such as one filtered for indocyanine green (ICG), processor 103 can identify a skin surface segment (object) at the abdominal region (designated region), determine the skin surface temperature (attribute) of subject 114, and output an indication of temperature by causing projector 106 to project an image representing and indicating the presence of dye in subject 114.

[0063] While detector 104 can be configured to detect radiation of any suitable wavelength range, the projector can be configured to emit light in the visible spectrum such that the image projected onto the subject's surface is visible to the human eye. Projector 106 may include a laser projector or a plurality of light-emitting elements (such as LEDs) arranged in a matrix on a substrate. For example, projector 106 may be a laser projector configured to project the image using a coherent parallel beam. As another example, if projector 106 may be an LED projector, each light-emitting element may correspond to a pixel of the output image.

[0064] Processor 103 can be configured to execute a mapping algorithm such that the projected image displays information at the same location on the subject's surface as designated region 132. The mapping algorithm can allow for error magnitude, and the error magnitude can depend on the type of detector. In other words, the projected image may not be exactly at the same location on the subject's surface as the detected image / signal.

[0065] Alternatively, in another example, the projected image may include more than one graphical representation of the detected image / signal. The projected image may also include data related to tissue properties and project this data onto the subject's surface. The projected image may also include the name and / or color overlay of the object identified when it belongs to the subject, such as if the object is a blood vessel or nerve. For example, if the detector can detect the movement of red blood cells in a blood vessel (object), the projected image may incorporate an overlay of blood flow (property) overlaid on the blood vessel in the graphical representation; alternatively, multiple blood vessels with different blood cell flows may be represented by different colors in the projected image. By combining visual cues with attribute-related insights, this provides practitioners and users with a more complete understanding, further assisting the real-time decision-making process. In another example, when the detector is a camera and the procedure is surgical (such as breast reconstruction), the projected image includes guide lines to assist the practitioner in improving symmetry during breast (designated area) reconstruction. In a further example, during internal fixation of a fracture, the detected object may be at least a portion of the bone to be repaired, the designated area may be the entire bone, the property may be the location and / or orientation of the bone, and the projected image may include suggested placement of rods and screws. The projected image may further include an indication of the position of the bone relative to surgical instruments, such as the angle between the saw to be used during, for example, osteotomy, and the bone.

[0066] The processor can also use artificial intelligence-based algorithms to enable the projector to create projected images as described above.

[0067] The device may include a memory for storing the identified attributes(s) for future reference or analysis. The memory can act as a repository for the attributes(s) identified during the surgical procedure. This memory facilitates storing attribute-related information for subsequent reference or in-depth analysis. For example, the processor may be able to identify tissue offline after surgery.

[0068] Memory can hold instructions for the processor to follow. Detectors can detect multiple objects simultaneously, supplemented by a memory repository of instructions derived from machine learning and AI algorithms. These algorithms enable the processor to identify objects and, for each object, the corresponding command.

[0069] In some embodiments, the processor may be associated with a database that maintains information collected from previous surgeries facilitated by the same device. The database may contain historical attribute data and related details. The processor can access the database to improve the accuracy of attributes and more efficiently determine attributes for each detected object. This enhances the device's ability to provide information to practitioners and users during complex medical procedures.

[0070] Some or all aspects of this invention can be adapted to be implemented in the form of software, particularly computer program products. This applies, for example, to software for control unit 204, as well as software for controlling image detectors and projectors, and software for processing detected images and converting them into projected images. The computer program product may include a computer program stored on a non-transitory computer-readable medium. Furthermore, the computer program may be represented by signals (such as optical or electromagnetic signals) carried by a transmission medium (such as fiber optic cable or air). The computer program may have, in part or in whole, source code, object code, or pseudocode suitable for execution by a computer system. For example, the code may be executed by one or more processors.

[0071] For example, certain embodiments of the present invention can be implemented as a computer-implemented method. This method may include: detecting at least a portion of a subject and an object in a detection field by a detector having a detection field; identifying a command by a processor system based on the detected object, wherein the command specifies a region on the surface of the subject; determining the properties of the subject's tissue within the specified region by the processor system; and outputting an indication of the determined properties under the control of the processor system. The method may further include: projecting an indication of the specified region and / or an indication of the determined properties of the tissue onto the surface of the subject by a projector. For example, the object is outside the subject, and the object indicates points of the shape of the specified region. The method may include: detecting consecutive points on the surface of the subject indicated by the object as the object moves relative to the subject; and connecting the consecutive points to determine the shape (outline) of the specified region. The method may include: projecting an indication of the shape of the specified region onto the surface of the subject by a projector, provided points on the subject's surface have been detected. The shape of the specified region may be a closed shape or a nearly closed shape. Determining the properties of the tissue may include determining the properties of superficial and / or deep tissues of the subject on and / or below the region defined by the specified region. The step of detecting an object may include detecting at least a portion of a hand or instrument as an object.

[0072] Alternatively, the object can be part of the subject's body. The method may include determining the shape of the object within a specified region.

[0073] To determine the attributes, the method may include performing real-time measurements of a specified region and / or mapping the specified region onto a preoperative image containing data representing the subject.

[0074] The method may further include: generating a graphical representation of the determined attribute, and causing a projector to project the graphical representation onto the surface of the subject. In this text, the projected graphical representation of the determined attribute may be aligned with a designated area.

[0075] The characteristics described about the apparatus can be advantageously applied to the method, and vice versa.

[0076] The examples and embodiments described herein are illustrative and not limiting of the invention. Those skilled in the art will be able to devise alternative embodiments without departing from the spirit and scope of this disclosure as defined by the appended claims and their equivalents. Reference numerals enclosed in parentheses in the claims should not be construed as limiting the scope of the claims. Items described as separate entities in the claims or specification may be implemented as a single hardware or software item combining the features of the described items.

[0077] Therefore, modifications and alternative implementations of some parts or elements are possible and are included within the scope of protection as defined by the appended claims.

[0078] Certain aspects are defined in the following clauses.

[0079] 1. An apparatus comprising: A detector having a detection field and configured to detect at least a portion of a subject in the detection field and to detect objects in the detection field; and Processor, the processor being configured to: A command is identified based on the detected object, wherein the command specifies a region on the surface of the subject; Determine the properties of the subject's tissues within the designated region; and Outputs an indication of the determined attributes.

[0080] 2. The apparatus as described in Clause 1, further comprising a projector configured to project an indication of the designated area and / or an indication of a determined property of the tissue onto the surface of the subject.

[0081] 3. The apparatus as described in clause 1 or 2, wherein the object is outside the subject, and wherein the object indicates a point of shape of the designated region, and the processor is configured to: Detecting continuous points indicated by the object on the surface of the subject as the object moves relative to the subject; and Connect the consecutive points to determine the shape of the specified region.

[0082] 4. The apparatus as described in Clause 3, wherein the projector is configured to project an indication of the shape of the designated area after a point on the surface of the subject has been detected.

[0083] 5. The apparatus as described in Clause 3 or Clause 4, wherein the shape of the designated region is a closed shape or a nearly closed shape, and wherein determining the properties of the tissue includes determining the properties of the superficial and / or deep tissues of the subject in the region defined by the designated region and / or below it.

[0084] 6. The apparatus as described in any of the preceding clauses, wherein the detector is configured to detect at least a portion of a hand or instrument as the object.

[0085] 7. The apparatus as described in Clause 1 or Clause 2, wherein the object is part of the subject's body, and the processor is further configured to determine the shape of the object within the designated area.

[0086] 8. The apparatus as described in any of the preceding clauses, wherein the attributes of the tissue include one or more of the following: name, temperature, size, area, volume, location, and tissue type.

[0087] 9. The apparatus as described in any of the preceding clauses, wherein the processor is configured to determine the properties of the subject's visceral tissue, epithelial tissue, connective tissue, adipose tissue, muscle tissue, nervous tissue, or abnormal tissue.

[0088] 10. The apparatus as described in any of the preceding clauses, wherein the properties of the tissue are determined based on real-time measurements of the designated region and / or based on mapping the designated region onto a preoperative image containing data representing the subject.

[0089] 11. The apparatus as described in Clause 10, wherein the preoperative image comprises images from one or more medical imaging techniques.

[0090] 12. The apparatus as described in any of the preceding clauses, wherein the attributes of the organization are determined by an identification algorithm including artificial intelligence.

[0091] 13. The apparatus as described in any of the preceding clauses, wherein the properties of the organization are determined instantaneously and / or over time.

[0092] 14. The apparatus as described in any of the preceding clauses, wherein the detector includes an image sensor and / or a position sensor to detect the position of the object.

[0093] 15. The apparatus as described in any of the preceding clauses, wherein the apparatus further includes a memory for storing determined properties of the tissue for future reference or analysis.

[0094] 16. The apparatus as described in any of the preceding clauses, wherein the detector is capable of detecting multiple objects simultaneously, and the processor is configured to identify commands and determine attributes individually for each detected object.

[0095] 17. The apparatus as described in any of the preceding clauses, wherein the processor is further configured to analyze the properties of the tissue and provide recommendations for surgery based on the analysis.

[0096] 18. The apparatus as described in any of the preceding clauses, wherein the processor is configured to generate a graphical representation of the determined attributes and to cause the projector to project the graphical representation onto the surface of the subject.

Claims

1. An apparatus comprising: A detector having a detection field and configured to detect at least a portion of a subject within the detection field and to detect objects within the detection field, wherein the objects are outside the subject, and wherein the objects are points indicating the shape of the designated area; and Processor, the processor being configured to: Detecting continuous points indicated by the object on the surface of the subject as the object moves relative to the subject; Connect the consecutive points to determine the shape of the specified region; A command is identified based on the detected object, wherein the command specifies the area on the surface of the subject; Determine the properties of the subject's tissues within the designated region; and Outputs an indication of the determined attributes.

2. The apparatus of claim 1, further comprising a projector configured to project an indication of the designated area and / or an indication of a determined property of the tissue onto the surface of the subject.

3. The apparatus of claim 1, wherein, The projector is configured to project an indication of the shape of the designated area after points on the surface of the subject have been detected.

4. The apparatus as claimed in claim 1 or claim 3, wherein, The shape of the designated region is a closed or nearly closed shape, and wherein determining the properties of the tissue includes determining the properties of the superficial and / or deep tissues of the subject in the area defined by the designated region and / or below it.

5. The apparatus as claimed in any of the preceding claims, wherein, The detector is configured to detect at least a portion of a hand or instrument as the object.

6. The apparatus as claimed in claim 1 or claim 2, wherein, The object is part of the subject's body, and the processor is further configured to determine the shape of the object within the designated area.

7. The apparatus as claimed in any of the preceding claims, wherein, The attributes of the organization include one or more of the following: name, temperature, size, area, volume, location, and organization type.

8. The apparatus as claimed in any of the preceding claims, wherein, The processor is configured to determine the properties of the subject's visceral tissue, epithelial tissue, connective tissue, adipose tissue, muscle tissue, nervous tissue, or abnormal tissue.

9. The apparatus as claimed in any of the preceding claims, wherein, The properties of the tissue are determined based on real-time measurements of the designated region and / or based on mapping the designated region onto a preoperative image containing data representing the subject.

10. The apparatus as claimed in any of the preceding claims, wherein, The detector includes an image sensor and / or a position sensor to detect the position of the object.

11. The apparatus as claimed in any of the preceding claims, wherein, The device further includes a memory for storing determined attributes of the organization for future reference or analysis, or wherein the attributes of the organization are determined by a recognition algorithm including artificial intelligence.

12. The apparatus as claimed in any of the preceding claims, wherein, The detector is capable of detecting multiple objects simultaneously, and the processor is configured to identify commands and determine attributes individually for each detected object.

13. The apparatus as claimed in any of the preceding claims, wherein, The processor is further configured to analyze the properties of the tissue and provide recommendations for surgery based on the analysis.

14. The apparatus as claimed in any of the preceding claims, wherein, The processor is configured to generate a graphical representation of the determined attributes and to cause the projector to project the graphical representation onto the surface of the subject.

Images