Robotic ultrasound diagnostic system and method for controlling robotic ultrasound diagnostic system

Abstract

This robotic ultrasound diagnostic system comprises: a first robotic arm that supports an ultrasound probe; a second robotic arm that supports an auxiliary device; a camera that acquires images of a subject; and a control unit that identifies a target diagnostic region of the subject on the basis of the images of the subject, controls the first robotic arm to scan the identified target diagnostic region with the ultrasound probe, processes ultrasound image data acquired by scanning with the ultrasound probe so as to identify a region of interest and anatomical structures, and controls the second robotic arm such that the auxiliary device contacts the subject at a target coordinate and at a target angle.

Description

Robotic Ultrasound Diagnostic System and Control Method of Robotic Ultrasound Diagnostic System

[0001] The disclosed invention relates to a robotic ultrasound diagnostic system combining robotic technology and medical ultrasound technology, and more specifically, to a robotic ultrasound diagnostic system and a control method thereof capable of performing automated ultrasound diagnosis and procedures by controlling an ultrasound probe and an auxiliary device using a plurality of robotic arms.

[0002] Recently, various medical imaging devices are widely used in the medical field to visualize and acquire information about human biological tissues for the purpose of early diagnosis of various diseases or surgery. Representative examples of such medical imaging devices include ultrasound imaging devices, CT (Computed Tomography) devices, and MRI (Magnetic Resonance Imaging) devices.

[0003] An ultrasound imaging device is a device that irradiates an object with an ultrasound signal generated from a transducer of a probe and receives information on the ultrasound signal reflected from the object to non-invasively acquire at least one image of an internal area of ​​the object (e.g., soft tissue or blood flow). The ultrasound imaging device can be used for medical purposes, such as observing the inside of an object, detecting foreign substances, and measuring injuries. Compared to imaging devices using X-rays, such ultrasound imaging devices have the advantages of being safe, allowing for real-time image display, and eliminating radiation exposure, so they are widely used in conjunction with other imaging devices.

[0004] However, conventional ultrasound diagnosis has the problem that diagnostic results vary depending on the proficiency of the medical professional, and the diagnosis and interpretation of ultrasound images are difficult. Furthermore, specialists in specific disciplines are required to carry out diagnosis, measurement, and report writing, which places limitations on the provision of medical services.

[0005] With the recent advancements in artificial intelligence (AI) and robotics technology, research is actively underway on solutions in which robots can replace high-difficulty ultrasound diagnosis. These robotic ultrasound systems have the advantage of providing diagnostic services of consistent quality without relying on the proficiency of medical personnel, and can provide accurate diagnoses and information on results while ensuring patient safety.

[0006] According to one aspect of the disclosed invention, a robotic ultrasound diagnostic system is provided that can perform automated ultrasound diagnosis and procedures without relying on the skill level of medical personnel by independently controlling an ultrasound probe and an auxiliary device using a plurality of robotic arms.

[0007] According to one aspect of the disclosed invention, a robotic ultrasound diagnostic system is provided that enables accurate and safe medical procedures by controlling an auxiliary device of a second robotic arm to an optimal position and angle based on ultrasound image information acquired by a first robotic arm.

[0008] According to one aspect of the disclosed invention, a robotic ultrasound diagnostic system is provided that monitors patient condition information in real time to ensure patient safety and can automatically move to a safe position when a dangerous situation occurs.

[0009] According to one aspect of the disclosed invention, a robotic ultrasound diagnostic system is provided that can perform continuous diagnosis by automatically replacing the probe without a separate calibration operation when a probe replacement is required during diagnosis.

[0010] According to one aspect of the disclosed invention, a robotic ultrasound diagnostic system is provided that can improve the accuracy and efficiency of diagnosis by automatically performing ultrasound image analysis, lesion detection, and prediction of optimal procedure paths using artificial intelligence technology.

[0011] According to one aspect of the disclosed invention, a robotic ultrasound diagnostic system is provided that includes an auxiliary battery to ensure patient safety even in emergency situations such as power outages, and a safety device to protect the patient even in the event of a malfunction of the robotic arm.

[0012] A robotic ultrasound diagnostic system according to one embodiment of the present disclosure may include: a first robotic arm supporting an ultrasound probe; a second robotic arm supporting an auxiliary device; a camera acquiring an image of an object; and a control unit that identifies a target diagnostic area of ​​an object based on the image of the object, controls the first robotic arm to scan the identified target diagnostic area with the ultrasound probe, processes ultrasound image data acquired by scanning the ultrasound probe to identify a region of interest and an anatomical structure, calculates the target coordinates of a target point where the auxiliary device must contact the object and the target angle between the auxiliary device and the object based on the contact coordinates of a point where the ultrasound probe supported by the first robotic arm contacts the object, the location information of the identified region of interest, and the location information of the anatomical structure, and controls the second robotic arm so that the auxiliary device contacts the object at the calculated target coordinates and the calculated target angle.

[0013] The above auxiliary device may include at least one of an external vibrator, a cauterization needle, a biopsy needle, or a contrast agent injection needle.

[0014] In response to the external vibration device coming into contact with the object at the calculated target angle at the calculated target coordinates, the control unit can control the external vibration device to irradiate a push pulse toward the object or generate vibration of the object.

[0015] The above anatomical structure may include bone structure and vascular structure.

[0016] The control unit can determine a region further than a preset first threshold distance from the bone structure as a first candidate region, determine a region further than a preset second threshold distance from the blood vessel structure as a second candidate region, determine a region within a preset third threshold distance from the contact coordinates as a third candidate region, and calculate the target coordinates within the intersection of the first candidate region, the second candidate region, and the third candidate region.

[0017] The above auxiliary device may be any one of the above cauterization needle, the above biopsy needle, or the above contrast agent injection needle.

[0018] In response to the needle making contact with the object while forming the calculated target angle at the calculated target coordinates, the control unit can control the second robot arm so that the needle reaches the target object within the region of interest.

[0019] The control unit can calculate the target coordinates and the target angle so that the needle avoids the bone structure and the blood vessel structure and reaches the target object within the region of interest.

[0020] The control unit can control the first robot arm to acquire ultrasonic image frames according to preset time intervals and spatial intervals, match and store coordinate information of the contact coordinates corresponding to each acquired ultrasonic image frame, and generate a three-dimensional ultrasonic image based on the stored ultrasonic image frames and the contact coordinates corresponding to each.

[0021] The control unit receives a medical image of another modality, processes the received medical image of another modality to identify a target landmark, and controls the first robot arm to scan the identified target diagnostic area with the ultrasound probe. In response to the identification of the target landmark in the ultrasound image data, the control unit stores the contact coordinates at the time when the ultrasound image data was acquired and controls the first robot arm based on the stored contact coordinates to align the ultrasound image data with the medical image of another modality.

[0022] The control unit can calculate the target coordinates and the target angle using an artificial intelligence model trained to calculate the target coordinates and the target angle using the contact coordinates and the ultrasonic image data as input data.

[0023] The above-described robotic ultrasonic diagnostic system may further include an electric bed on which the target object can be positioned.

[0024] The control unit can move the electric bed to a predefined position in response to the detection of a predefined danger condition.

[0025] The above-described robotic ultrasonic diagnostic system may further include a pressure sensor that detects the pressure exerted by the ultrasonic probe on the object.

[0026] The above-defined predefined risk conditions may include the pressure detected by the pressure sensor being greater than the critical pressure.

[0027] The above control unit can detect the risk of the object by inputting an image of the object into a risk analysis artificial intelligence model.

[0028] The above-mentioned predefined risk conditions may include the risk of the object being detected by the risk analysis artificial intelligence model.

[0029] The above-mentioned robotic ultrasound diagnostic system may further include a microphone.

[0030] The above control unit can detect the risk to the object by inputting the acoustic signal received through the microphone into a risk analysis artificial intelligence model.

[0031] The above-mentioned predefined risk conditions may include the risk of the object being detected by the risk analysis artificial intelligence model.

[0032] The operating range of the first robot arm and the second robot arm can be designed so as not to interfere with the predefined position.

[0033] In response to the detection of the predefined danger condition while the above auxiliary device is in contact with the object, the control unit can move the electric bed in a direction corresponding to the calculated target angle.

[0034] A control method for a robotic ultrasound diagnostic system according to an embodiment of the present invention may include: acquiring an image of an object; identifying a target diagnostic area of ​​the object based on the image of the object; controlling a first robotic arm to scan the identified target diagnostic area with an ultrasound probe; processing ultrasound image data acquired by scanning with the ultrasound probe to identify a region of interest and an anatomical structure; calculating the target coordinates of a target point where the auxiliary device must contact the object and the target angle between the auxiliary device and the object based on the contact coordinates of a point where the ultrasound probe supported by the first robotic arm contacts the object, the location information of the identified region of interest, and the location information of the anatomical structure; and controlling a second robotic arm so that the auxiliary device contacts the object at the calculated target coordinates and the calculated target angle.

[0035] The control method of the above-described robot ultrasonic diagnostic system may further include controlling the external vibration device to irradiate a push pulse toward the target or generate vibration of the target in response to the external vibration device coming into contact with the target while forming a calculated target angle at the calculated target coordinates.

[0036] Calculating the target coordinates may include determining a region farther than a preset first threshold distance from the bone structure as a first candidate region; determining a region farther than a preset second threshold distance from the blood vessel structure as a second candidate region; determining a region within a preset third threshold distance from the contact coordinates as a third candidate region; and calculating the target coordinates within the intersection of the first candidate region, the second candidate region, and the third candidate region.

[0037] The control method of the above-described robot ultrasonic diagnostic system may further include controlling the second robot arm so that the needle reaches a target object within the region of interest in response to the needle contacting the target object while forming a calculated target angle at the calculated target coordinates.

[0038] Calculating the above target coordinates may include calculating the target coordinates and the target angle such that the needle avoids the bone structure and the blood vessel structure and reaches the target object within the region of interest.

[0039] A robotic ultrasonic diagnostic system according to one embodiment of the present disclosure may include: a support device configured to fix an ultrasonic probe so as to face a first direction and an auxiliary device so as to face a second direction different from the first direction and be rotatable; a robotic arm supporting the support device; and a control unit that controls the rotation of the support device and the movement of the robotic arm so that, when an equipment replacement condition is satisfied while performing a diagnostic operation on an object using the ultrasonic probe, the auxiliary device forms the same relative positional relationship and relative angular relationship that the ultrasonic probe had formed with respect to the object based on position and attitude control information of the robotic arm.

[0040] The support device may include a rotating body comprising a first fixing part for fixing either the ultrasonic probe or the auxiliary device and a second fixing part for fixing the other; and a driving part for rotating the rotating body.

[0041] The first angle formed by the rotation axis of the support device and the first imaginary line extending in the first direction may be the same as the second angle formed by the rotation axis and the second imaginary line extending in the second direction.

[0042] When the equipment replacement condition is satisfied while performing a diagnostic operation on an object using the above-mentioned ultrasonic probe, the control unit may control the robot arm so that the ultrasonic probe moves away from the object by a predetermined distance, and then rotate the support device so that the support angles of the ultrasonic probe and the auxiliary device are mutually interchanged.

[0043] The control unit above can determine the predetermined distance based on the difference between the length of the auxiliary device and the length of the ultrasonic probe.

[0044] The above ultrasonic probe is a first ultrasonic probe, and the above auxiliary device is a second ultrasonic probe. When the equipment replacement condition is satisfied while performing a diagnostic operation on an object using the first ultrasonic probe, the control unit can deactivate the first ultrasonic probe and activate the second ultrasonic probe.

[0045] The control unit sets second control information of the activated second ultrasonic probe based on first control information of the first ultrasonic probe prior to the first ultrasonic probe being deactivated, and the control information may include at least one of transmission angle information of the ultrasonic signal and focal point information.

[0046] The control unit can activate the second ultrasonic probe in response to the completion of the rotation of the support device and the movement of the robot arm to ensure that the second ultrasonic probe forms the same relative positional relationship and relative angle relationship that the first ultrasonic probe had formed with respect to the object.

[0047] The above auxiliary device is a gel application device, and the control unit performs a gel application operation on a diagnostic area of ​​the target using the gel application device before performing the diagnostic operation using the ultrasonic probe, and when the gel application operation is completed, controls the rotation of the support device and the movement of the robot arm so that the ultrasonic probe forms the same relative positional relationship and relative angle relationship that the gel application device had formed with respect to the target based on the position and attitude control information of the robot arm, and then performs a pre-scan operation on the target using the ultrasonic probe.

[0048] The control unit can perform the pre-scan operation using the ultrasonic probe and generate an ultrasonic image, identify a gel-uncoated area based on the ultrasonic image, control the rotation of the support device and the movement of the robot arm so that the gel-coating device forms the same relative positional relationship and relative angle relationship that the ultrasonic probe had formed with respect to the target object based on the identification of the gel-uncoated area, and then control the gel-coating device to apply gel to the identified gel-uncoated area.

[0049] The control unit may perform the diagnostic operation using the ultrasonic probe and generate an ultrasonic image, identify a gel-unapplied area based on the ultrasonic image, and determine that the equipment replacement condition is satisfied based on the identification of the gel-unapplied area.

[0050] The control unit can identify the location of the gel-uncoated area based on the position of the ultrasonic probe based on the position and attitude control information of the robot arm, control the rotation of the support device and the movement of the robot arm so that the gel-coating device forms the same relative positional relationship and relative angle relationship that the ultrasonic probe had formed with respect to the target object, and then control the gel-coating device to apply gel to the identified gel-uncoated area.

[0051] The support device may further include a camera having a shooting field of view facing either the ultrasonic probe or the auxiliary device located at a first position according to the rotation of the support device, regardless of the rotation of the support device.

[0052] The above auxiliary device is a cleaning device, and the control unit can control the cleaning device to suck up the gel applied to the object and spray water onto the object in response to the completion of the rotation of the support device and the movement of the robot arm so that the cleaning device forms the same relative positional relationship and relative angle relationship that the ultrasonic probe had formed with respect to the object.

[0053] The control unit may perform the diagnostic operation using the ultrasound probe and generate an ultrasound image, identify a target lesion based on the ultrasound image, and determine that the equipment replacement condition is satisfied in response to the identification of the target lesion.

[0054] The above auxiliary device is a first auxiliary device, and the support device comprises a rotating body including a first fixing part for fixing any one of the ultrasonic probe, the first auxiliary device, and the second auxiliary device, a second fixing part for fixing another of the ultrasonic probe, the first auxiliary device, and the second auxiliary device, and a third fixing part for fixing the remaining one of the ultrasonic probe, the first auxiliary device, and the second auxiliary device; and a driving part for rotating the rotating body; wherein the first fixing part, the second fixing part, and the third fixing part may be arranged at equal intervals on the rotating body with respect to the rotation axis of the support device, or arranged at equal angles to each other.

[0055] The above control unit can control the robot arm so that when diagnosis begins, the support device forms a predefined angle with the horizontal plane.

[0056] The above auxiliary device is a gel application device, and the gel application device includes a gel application unit and a water spray unit. The control unit controls the rotation of the support device and the movement of the robot arm so that the gel application unit is positioned at the diagnostic site of the target, and then performs a gel application operation on the diagnostic site of the target using the gel application unit. When the gel application is completed, the control unit controls the rotation of the support device and the movement of the robot arm so that the ultrasonic probe forms the same relative positional relationship and relative angle relationship that the gel application unit formed with respect to the target based on the position and attitude control information of the robot arm, and then performs the diagnostic operation on the target using the ultrasonic probe. When the diagnostic operation is completed, the control unit controls the rotation of the support device and the movement of the robot arm so that the water spray unit forms the same relative positional relationship and relative angle relationship that the ultrasonic probe formed with respect to the target based on the position and attitude control information of the robot arm, and then performs a cleaning operation on the target using the water spray unit.

[0057] The above-mentioned robot arm is a first robot arm, and the robot ultrasound diagnostic system may further include a second robot arm configured to support a medical needle.

[0058] When the equipment replacement condition is satisfied while performing an auxiliary operation on the target using the auxiliary device, the control unit controls the rotation of the support device and the movement of the robot arm based on the position and attitude control information of the robot arm so that the ultrasonic probe forms the same relative positional relationship and relative angular relationship that the auxiliary device had formed with respect to the target, wherein the rotation direction of the support device when equipment replacement occurs from the ultrasonic probe to the auxiliary device and the rotation direction of the support device when equipment replacement occurs from the auxiliary device to the ultrasonic probe may be different from each other.

[0059] According to one aspect of the disclosed invention, by independently controlling an ultrasound probe and an auxiliary device using a plurality of robotic arms, automated ultrasound diagnosis and procedures can be performed without the intervention of a medical professional, thereby improving the consistency and accuracy of medical services.

[0060] According to one aspect of the disclosed invention, by analyzing ultrasound image information acquired from a first robot arm in real time and controlling an auxiliary device of a second robot arm at an optimal position and angle, the accuracy and safety of the procedure can be significantly improved compared to the conventional manual operation method.

[0061] According to one aspect of the disclosed invention, various auxiliary devices such as an external vibration device, a biopsy needle, an RF thermal therapy needle, and a contrast agent injection system can be mounted on the second robotic arm and utilized, thereby enabling various types of ultrasound diagnosis and procedures to be performed with a single system.

[0062] According to one aspect of the disclosed invention, the success rate of a procedure can be improved by automatically identifying anatomical structures and lesions through image analysis using artificial intelligence technology and automatically calculating the optimal procedure path based thereon.

[0063] According to one aspect of the disclosed invention, the patient's condition can be identified by monitoring the patient's facial expressions, voice, movements, etc., in real time, and the patient's safety can be effectively ensured by automatically moving to a safe position when a dangerous situation occurs.

[0064] According to one aspect of the disclosed invention, by automatically maintaining the existing ultrasonic beam transmission angle and focal point when replacing the probe, continuous diagnosis can be performed without separate calibration work, thereby significantly improving diagnostic efficiency.

[0065] According to one aspect of the disclosed invention, even in emergency situations such as a power outage, a robot arm can be moved to a safe position using an auxiliary battery, and even in the event of a malfunction of the robot arm, the safety of the patient can be protected through the coordinated control of another robot arm and the bed.

[0066] According to one aspect of the disclosed invention, a medical professional can control a robotic ultrasound system from a remote location through a remote control function, thereby significantly improving the accessibility of medical services.

[0067] According to one aspect of the disclosed invention, the possibility of developing into an unmanned ultrasound system is provided, thereby laying the foundation for providing high-quality ultrasound diagnostic services with minimal human intervention in the future.

[0068] The present disclosure can be easily understood from the combination of the following detailed description and the accompanying drawings, where reference numerals denote structural elements.

[0069] FIGS. 1a and FIGS. 1b are block diagrams illustrating the configuration of an ultrasound imaging system according to one embodiment of the present disclosure.

[0070] FIGS. 2a, FIGS. 2b, FIGS. 2c, and FIGS. 2d are drawings illustrating an ultrasound imaging system according to one embodiment of the present disclosure.

[0071] FIG. 3 is a diagram illustrating the overall configuration of a robotic ultrasound diagnostic system according to one embodiment of the present invention.

[0072] FIG. 4 is a control block diagram of a robotic ultrasound diagnostic system according to one embodiment of the present invention.

[0073] FIG. 5 is a flowchart illustrating a control method for a robotic ultrasound diagnostic system according to an embodiment of the present invention.

[0074] FIG. 6 is a diagram illustrating a method for calculating a target position and a target angle based on coordinate information, tilt angle information, and object information of a robot arm according to an embodiment of the present invention.

[0075] FIG. 7 is a flowchart for explaining the operation when an auxiliary device according to one embodiment of the present invention is an external vibration device.

[0076] FIG. 8 is a diagram illustrating a method for selecting a target position of an external vibration device in an ultrasonic image according to an embodiment of the present invention.

[0077] FIG. 9 is a drawing illustrating an external vibration device according to one embodiment of the present invention in contact with an object at a target location.

[0078] FIG. 10 is a flowchart illustrating the operation when the auxiliary device according to one embodiment of the present invention is a needle.

[0079] FIG. 11 is a diagram illustrating a method for selecting a target position and a target angle of a needle in an ultrasound image according to an embodiment of the present invention.

[0080] FIG. 12 is a drawing illustrating the needle approaching an object at a target position and target angle when the auxiliary device according to one embodiment of the present invention is a biopsy needle or a cauterization needle.

[0081] FIG. 13 is a diagram illustrating the appearance of a needle reaching a vein within the region of interest at a target position and target angle when the auxiliary device according to one embodiment of the present invention is a contrast agent injection needle.

[0082] FIG. 14 is a flowchart illustrating the operation of moving an electric bed to a predefined position when a predefined danger condition is detected according to one embodiment of the present invention.

[0083] FIG. 15 is a drawing illustrating the range of motion of a robot arm and an electric bed according to one embodiment of the present invention.

[0084] FIG. 16 is a drawing showing a cable connected to an ultrasonic probe according to one embodiment of the present invention extended into the interior of a robot arm.

[0085] FIG. 17 is a drawing for explaining the structure of a support device according to one embodiment of the present invention.

[0086] FIGS. 18a and FIGS. 18b are drawings for explaining the configuration of a fixed part when viewed from above of a support device according to one embodiment of the present invention.

[0087] FIG. 19 is a drawing for explaining the structure of a gel application device according to one embodiment of the present invention.

[0088] FIG. 20 is a flowchart illustrating a series of processes for diagnosing an object through equipment replacement when a robotic ultrasonic diagnostic system according to one embodiment of the present invention supports a support device.

[0089] FIG. 21 is a drawing illustrating the robot arm moving upward, rotating, and then moving downward again when the equipment replacement condition according to one embodiment of the present invention is satisfied.

[0090] FIG. 22 is a drawing for explaining a method for calculating the required upper movement distance of a robot arm during rotation based on the difference in length between an ultrasonic probe supported by a first fixed part of a support device and an auxiliary device supported by a second fixed part according to an embodiment of the present invention.

[0091] FIG. 23 is a flowchart illustrating the operation of deactivating a first probe and then activating a second probe when an auxiliary device supported by a support device according to one embodiment of the present invention is a probe.

[0092] FIG. 24 is a drawing for explaining a structure in which an ultrasonic probe supported by a support device according to one embodiment of the present invention includes first and second transducer arrays having different scan directions.

[0093] FIG. 25 is a drawing illustrating that the focal point is maintained even after equipment replacement according to one embodiment of the present invention.

[0094] FIG. 26 is a flowchart illustrating the operation of replacing the equipment with a gel application device and proceeding with gel application when identifying a gel-unapplied area according to an embodiment of the present invention.

[0095] The present disclosure clarifies the scope of the claims of the present disclosure and explains the principles of the embodiments of the present disclosure and discloses the embodiments so that a person skilled in the art to which the embodiments of the present disclosure pertain can practice the embodiments of the present disclosure. The embodiments of the present disclosure may be implemented in various forms.

[0096] Throughout the specification, the same reference numerals refer to the same components. This specification does not describe all elements of the embodiments, and general content in the art to which the invention pertains or content that overlaps between embodiments is omitted. The terms 'module' or 'unit' as used in the specification may be implemented as one or more combinations of software, hardware, or firmware, and depending on the embodiments, multiple 'modules' or 'units' may be implemented as a single element, or a single 'module' or 'unit' may include multiple elements.

[0097] The singular form of the noun corresponding to the item may include one or multiple items, unless the relevant context clearly indicates otherwise.

[0098] In the present disclosure, each of the phrases such as “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B, or C”, “at least one of A, B, and C”, and “at least one of A, B, or C” may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0099] The term “and / or” includes a combination of multiple related described components or any of the multiple related described components.

[0100] For example, expressions such as "A and / or B" or "at least one of A or B" may include all possible combinations of items listed together. For example, "A and / or B" or "at least one of A or B" may refer to cases including (1) only A, (2) only B, or (3) both A and B.

[0101] Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another component and do not limit the components in other aspects (e.g., importance or order).

[0102] Additionally, terms such as 'front,' 'rear,' 'top,' 'bottom,' 'side,' 'left,' 'right,' 'top,' and 'bottom' used in this disclosure are defined based on the drawings, and the shape and location of each component are not limited by these terms.

[0103] Terms such as “include” or “have” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this disclosure, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0104] When it is said that a component is "connected," "combined," "supported," or "in contact" with another component, this includes not only cases where the components are directly connected, combined, supported, or in contact, but also cases where they are indirectly connected, combined, supported, or in contact through a third component.

[0105] When it is said that a component is located "on" another component, this includes not only cases where one component is in contact with the other, but also cases where another component exists between the two components.

[0106] An ultrasonic device according to various embodiments will be described in detail below with reference to the attached drawings. In describing with reference to the attached drawings, identical or corresponding components are assigned similar drawing numbers, and redundant descriptions thereof may be omitted.

[0107] In the present disclosure, 'object' refers to the subject of imaging and may include a person, an animal, or a part thereof. For example, the object may include a part of the body (such as an organ or tissue) or a phantom.

[0108] In the present disclosure, the term "ultrasonic image" may refer to an image of an object that is transmitted to an object and is generated or processed based on an ultrasonic signal (echo signal) reflected from the object.

[0109] In the present disclosure, the expression "the robot arm supports A" may mean that the robot arm grasps A, is docked with A, and / or fixes A, and may include the robot arm supporting A directly or indirectly.

[0110] Embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0111] FIGS. 1a and FIGS. 1b are block diagrams illustrating the configuration of an ultrasound imaging system according to one embodiment of the present disclosure.

[0112] Referring to FIGS. 1a and FIGS. 1b, the ultrasound imaging system (100) may include a probe (20) and an ultrasound imaging device (40).

[0113] The ultrasound imaging device (40) can be implemented as a cart type as well as a portable type. Examples of portable ultrasound imaging devices may include, but are not limited to, a probe, a smartphone including an application, a laptop computer, a PDA (Personal Digital Assistant), or a tablet PC. The ultrasound imaging device (40) can also be implemented as a probe-integrated type.

[0114] The probe (20) may include a wired probe connected to the ultrasonic imaging device (40) via a wire and communicating with the ultrasonic imaging device (40) via a wire, a wireless probe connected to the ultrasonic imaging device (40) via a wireless connection and communicating with the ultrasonic imaging device (40) via a wireless connection, and / or a hybrid probe connected to the ultrasonic imaging device (40) via a wired or wireless connection and communicating with the ultrasonic imaging device (40) via a wired or wireless connection.

[0115] According to various embodiments of the present disclosure, as shown in FIG. 1a, the ultrasonic imaging device (40) may include an ultrasonic transceiver module (110), and as shown in FIG. 1b, the probe (20) may include an ultrasonic transceiver module (110). According to various embodiments of the present disclosure, it is also possible for both the ultrasonic imaging device (40) and the probe (20) to include an ultrasonic transceiver module (110).

[0116] According to various embodiments of the present disclosure, the probe (20) may further include at least one of an image processor (130), a display (140), or an input interface (170), or a combination thereof. In the present disclosure, the description of the ultrasonic transceiver module (110), image processor (130), display (140), or input interface (170) included in the ultrasonic imaging device (40) may also apply to the ultrasonic transceiver module (110), image processor (130), display (140), or input interface (170) included in the probe (20).

[0117] FIG. 1a is a block diagram showing the configuration of an ultrasound imaging system (100) when the probe (20) is a wired probe or a hybrid probe.

[0118] The probe (20) may include a plurality of transducers. The plurality of transducers may be arranged in a predetermined array to be implemented as a transducer array. The transducer array may correspond to a one-dimensional (1D) array or a two-dimensional (2D) array. The plurality of transducers may transmit an ultrasonic signal to a target (10) according to a transmission signal applied from a transmission module (113). The plurality of transducers may receive an ultrasonic signal (echo signal) reflected from the target (10) to form a reception signal. Additionally, the probe (20) may be implemented as an integral unit with the ultrasonic imaging device (40) or as a separate unit connected to the ultrasonic imaging device (40) via a wire. Additionally, the ultrasonic imaging device (40) may be connected to one or a plurality of probes (20) depending on the implementation type.

[0119] If the probe (20) is a wired probe or a hybrid probe, it may include a cable and a connector that can be connected to the connector of the ultrasound imaging device (40).

[0120] A probe (20) according to one embodiment of the present disclosure may be implemented as a two-dimensional probe. When the probe (20) is implemented as a two-dimensional probe, a plurality of transducers included in the probe (20) may be arranged in two dimensions to form a two-dimensional transducer array.

[0121] For example, a two-dimensional transducer array may be in the form of including a plurality of sub-arrays, each comprising a plurality of transducers arranged in a first direction, in a second direction different from the first direction.

[0122] In addition, when the probe (20) according to one embodiment of the present disclosure is implemented as a two-dimensional probe, the ultrasonic transceiver module (110) may include at least one of an analog beamformer or a digital beamformer. In addition, according to one embodiment of the present disclosure, the two-dimensional probe may include at least one of an analog beamformer or a digital beamformer or a combination thereof, depending on the implementation form.

[0123] The processor (120) controls the transmission module (113) to form a transmission signal to be applied to each of the transducers (117) by taking into account the positions and focal points of the plurality of transducers included in the probe (20).

[0124] The processor (120) can control the receiving module (115) to generate ultrasonic data by converting the received signal received from the probe (20) into analog-to-digital and summing the digitally converted received signal, taking into account the positions and focusing points of the plurality of transducers.

[0125] When the probe (20) is implemented as a two-dimensional probe, the processor (120) can calculate a time delay value for digital beamforming for each of the multiple sub-arrays included in the two-dimensional transducer array. Additionally, the processor (120) can calculate a time delay value for analog beamforming for each of the transducers included in any one of the multiple sub-arrays. The processor (120) can control an analog beamformer and a digital beamformer to form a transmission signal to be applied to each of the multiple transducers according to the time delay value for analog beamforming and the time delay value for digital beamforming. Additionally, the processor (120) can control an analog beamformer to sum the signals received from the multiple transducers according to the time delay value for analog beamforming for each sub-array. Additionally, the processor (120) can control an ultrasonic transceiver module (110) to convert the signals summed for each sub-array into analog-to-digital. Additionally, the processor (120) can control the digital beamformer to generate ultrasonic data by summing the digitally converted signals according to a time delay value for digital beamforming.

[0126] The image processor (130) uses the generated ultrasound data to generate or process an ultrasound image.

[0127] The display (140) can display the generated ultrasound image and various information processed by the ultrasound imaging device (40) or the probe (20). The probe (20) or the ultrasound imaging device (40) may include one or more displays (140) depending on the implementation form. Additionally, the display (140) may include a touch panel or a touch screen. Additionally, the display (140) may include a flexible display.

[0128] The processor (120) can control the overall operation of the ultrasound imaging device (40) and control the operation of the components of the ultrasound imaging device (40). The processor (120) can perform or control various operations or functions of the ultrasound imaging device (40) by executing programs or instructions stored in memory (150). Additionally, the processor (120) can control the operation of the ultrasound imaging device (40) by receiving a control signal from an input interface (170) or an external device.

[0129] The ultrasound imaging device (40) includes a communication module (160) and can be connected to and communicate with an external device (e.g., probe (20), server, medical device, portable device (smartphone, tablet PC, wearable device, etc.)) through the communication module (160).

[0130] The communication module (160) may include one or more components that enable communication with an external device. The communication module (160) may include, for example, at least one of a short-range communication module, a wired communication module, or a wireless communication module.

[0131] The communication module (160) can receive control signals or data from an external device. The processor (120) can control the operation of the ultrasonic imaging device (40) according to the control signals received through the communication module (160). Additionally, it is possible for the processor (120) to transmit control signals to an external device through the communication module (160) and control the external device according to the transmitted control signals. The external device may operate according to the control signals received from the ultrasonic imaging device (40) or process data received from the ultrasonic imaging device (40).

[0132] A program or application related to the ultrasonic imaging device (40) may be installed on an external device. The program or application installed on the external device may control the ultrasonic imaging device (40) or operate according to a control signal or data received from the ultrasonic imaging device (40).

[0133] An external device may receive or download a program or application related to the ultrasound imaging device (40) from the ultrasound imaging device (40), probe (20), or server, and install and execute the program or application on the external device. The ultrasound imaging device (40), probe (20), or server providing the program or application may include a recording medium that stores instructions, commands, installation files, executable files, or related data of the said program or application. It is also possible for the external device to be sold with the program or application installed.

[0134] The memory (150) can store various data or programs for driving and controlling the ultrasonic imaging device (40), input / output ultrasonic data, ultrasonic images, etc.

[0135] The input interface (170) can receive user input for controlling the ultrasonic imaging device (40). For example, user input may include, but is not limited to, inputs such as operating a button, keypad, mouse, trackball, jog switch, knob, etc., inputs such as touching a touchpad or touchscreen, voice input, motion input, biometric information input (e.g., iris recognition, fingerprint recognition, etc.).

[0136] FIG. 1b illustrates a control block diagram of an ultrasound imaging system (100) when the probe (20) is a wireless probe or a hybrid probe.

[0137] According to various embodiments of the present disclosure, the ultrasonic imaging device (40) illustrated in FIG. 1b may be replaced with the ultrasonic imaging device (40) described with reference to FIG. 1a.

[0138] According to various embodiments of the present disclosure, the probe (20) illustrated in FIG. 1a may be replaced with the probe (20) described with reference to FIG. 1b.

[0139] The probe (20) may include a display (112), a transmitting module (113), a battery (114), a receiving module (115), a charging module (116), a transducer (117), an input interface (109), a processor (118), and a communication module (119). Although FIG. 1b shows the probe (20) including both the transmitting module (113) and the receiving module (115), depending on the implementation, the probe (20) may include only some of the components of the transmitting module (113) and the receiving module (115), and some of the components of the transmitting module (113) and the receiving module (115) may be included in the ultrasound imaging device (40). Additionally, according to one embodiment of the present disclosure, the probe (20) may further include an image processor (130).

[0140] The transducer (117) may include a plurality of transducers. The plurality of transducers may be arranged in a predetermined array to form a transducer array. The transducer array may correspond to a one-dimensional (1D) array or a two-dimensional (2D) array. The plurality of transducers may transmit an ultrasonic signal to a target (10) according to a transmission signal applied from a transmission module (113). Additionally, the plurality of transducers may receive an ultrasonic signal reflected from the target (10) and form or generate an electrical reception signal.

[0141] The charging module (116) can charge the battery (114). The charging module (116) can receive power from an external source. According to one embodiment of the present disclosure, the charging module (116) can receive power wirelessly. Additionally, according to one embodiment of the present disclosure, the charging module (116) can receive power via a wired connection. The charging module (116) can transfer the received power to the battery (114).

[0142] The processor (118) controls the transmission module (113) to generate or form a transmission signal to be applied to each of the multiple transducers by taking into account the positions and focal points of the multiple transducers.

[0143] The processor (118) controls the receiving module (115) to generate ultrasound data by converting the received signal received from the transducer (117) into analog-to-digital and summing the digitally converted received signal, taking into account the positions and focusing points of the plurality of transducers. According to one embodiment of the present disclosure, if the probe (20) includes an image processor (130), an ultrasound image can be generated using the generated ultrasound data.

[0144] When the probe (20) is implemented as a two-dimensional probe, the processor (118) can calculate a time delay value for digital beamforming for each of the multiple sub-arrays included in the two-dimensional transducer array. Additionally, the processor (118) can calculate a time delay value for analog beamforming for each of the transducers included in any one of the multiple sub-arrays. The processor (118) can control the analog beamformer and the digital beamformer to form a transmission signal to be applied to each of the multiple transducers according to the time delay value for analog beamforming and the time delay value for digital beamforming. Additionally, the processor (118) can control the analog beamformer to sum the signals received from the multiple transducers according to the time delay value for analog beamforming for each sub-array. Additionally, the processor (118) can control the ultrasonic transceiver module (110) to convert the signals summed for each sub-array into analog-to-digital. Additionally, the processor (118) can control the digital beamformer to generate ultrasonic data by summing the digitally converted signals according to a time delay value for digital beamforming.

[0145] The processor (118) can control the overall operation of the probe (20) and the operation of the components of the probe (20). The processor (118) can perform or control various operations or functions of the probe (20) by executing programs or instructions stored in memory (111). Additionally, the processor (118) can control the operation of the probe (20) by receiving a control signal from the input interface (109) of the probe (20) or an external device (e.g., an ultrasound imaging device (40)). Additionally, the processor (118) can control the operation of the probe (20) by receiving a control signal from the input interface (109) or an external device. The input interface (109) can receive user input for controlling the probe (20). For example, user input may include, but is not limited to, inputs involving the operation of buttons, keypads, mice, trackballs, jog switches, knobs, etc., inputs involving the touch of a touchpad or touchscreen, voice inputs, motion inputs, and biometric information inputs (e.g., iris recognition, fingerprint recognition, etc.).

[0146] The display (112) can display an ultrasound image generated by the probe (20), an ultrasound image generated by processing ultrasound data generated by the probe (20), an ultrasound image received from the ultrasound imaging device (40), or various information processed by the ultrasound imaging system (100). Additionally, the display (112) can further display status information of the probe (20). The status information of the probe (20) may include at least one of device information of the probe (20), battery status information of the probe (20), frequency band information of the probe (20), output information of the probe (20), information on whether there is an abnormality of the probe (20), setting information of the probe (20), or temperature information of the probe (20).

[0147] The probe (20) may include one or more displays (112) depending on the implementation form. Additionally, the displays (112) may include a touch panel or a touch screen. Additionally, the displays (112) may include a flexible display.

[0148] The communication module (119) can wirelessly transmit generated ultrasound data or ultrasound images to the ultrasound imaging device (40) via a wireless network. Additionally, the communication module (119) can receive control signals and data from the ultrasound imaging device (40).

[0149] The ultrasonic imaging device (40) can receive ultrasonic data or ultrasonic images from the probe (20).

[0150] In one embodiment of the present disclosure, when the probe (20) includes an image processor (130) capable of generating an ultrasound image using ultrasound data, the probe (20) can transmit ultrasound data or an ultrasound image generated by the image processor (130) to an ultrasound imaging device (40).

[0151] In one embodiment of the present disclosure, if the probe (20) does not include an image processor (130) capable of generating an ultrasound image using ultrasound data, the probe (20) can transmit ultrasound data to an ultrasound imaging device (40). The ultrasound data may include ultrasound raw data, and the ultrasound image may mean ultrasound image data.

[0152] The ultrasonic imaging device (40) may include a processor (120), an image processor (130), a display (140), a memory (150), a communication module (160), and an input interface (170).

[0153] The image processor (130) generates or processes an ultrasound image using ultrasound data received from the probe (20).

[0154] The display (140) can display an ultrasound image received from the probe (20), an ultrasound image generated by processing ultrasound data received from the probe (20), or various information processed by the ultrasound imaging system (100). The ultrasound imaging device (40) may include one or a plurality of displays (140) depending on the implementation form. Additionally, the display (140) may include a touch panel or a touch screen. Additionally, the display (140) may include a flexible display.

[0155] The processor (120) can control the overall operation of the ultrasound imaging device (40) and control the operation of the components of the ultrasound imaging device (40). The processor (120) can execute a program or application stored in memory (150) to perform or control various operations or functions of the ultrasound imaging device (40). Additionally, the processor (120) can control the operation of the ultrasound imaging device (40) by receiving a control signal from an input interface (170) or an external device.

[0156] The ultrasound imaging device (40) includes a communication module (160) and can be connected to and communicate with an external device (e.g., probe (20), server, medical device, portable device (smartphone, tablet PC, wearable device, etc.)) through the communication module (160).

[0157] The communication module (160) may include one or more components that enable communication with an external device. The communication module (160) may include, for example, at least one of a short-range communication module, a wired communication module, or a wireless communication module.

[0158] The communication module (160) of the ultrasound imaging device (40) and the communication module (119) of the probe (20) may communicate using a network or may communicate using a short-range wireless communication method. For example, the communication module (160) of the ultrasound imaging device (40) and the communication module (119) of the probe (20) may communicate using any one of wireless data communication methods including Wireless LAN, Wi-Fi, Bluetooth, Zigbee, WFD (Wi-Fi Direct), infrared communication (IrDA, infrared Data Association), BLE (Bluetooth LowEnergy), NFC (Near Field Communication), Wibro (Wireless Broadband Internet), WiMAX (World Interoperability for Microwave Access), SWAP (Shared Wireless Access Protocol), WiGig (Wireless Gigabit Alliance), RF communication, or 60 GHz millimeter wave (mm Wave) short-range communication.

[0159] To this end, the communication module (160) of the ultrasound imaging device (40) and the communication module (119) of the probe (20) may include at least one of a Wireless LAN communication module, a Wi-Fi communication module, a Bluetooth communication module, a Zigbee communication module, a Wi-Fi Direct (WFD) communication module, an Infrared Data Association (IrDA) module, a Bluetooth LowEnergy (BLE) communication module, a Near Field Communication (NFC) communication module, a Wireless Broadband Internet (Wibro) communication module, a World Interoperability for Microwave Access (WiMAX) communication module, a Shared Wireless Access Protocol (SWAP) communication module, a Wireless Gigabit Alliance (WiGig) communication module, an RF communication module, or a 60 GHz millimeter wave (mm Wave) short-range communication module.

[0160] In one embodiment of the present disclosure, the probe (20) transmits device information (e.g., ID information) of the probe (20) to an ultrasound imaging device (40) using a first communication method (e.g., BLE) and can be wirelessly paired with the ultrasound imaging device (40). Additionally, the probe (20) can transmit ultrasound data and / or ultrasound images to the paired ultrasound imaging device (40).

[0161] The device information of the probe (20) may include various information related to the serial number, model name, or battery status of the probe (20).

[0162] The ultrasound imaging device (40) can receive device information (e.g., ID information) of the probe (20) from the probe (20) using a first communication method (e.g., BLE) and can be wirelessly paired with the probe (20). Additionally, the ultrasound imaging device (40) can transmit an activation signal to the paired probe (20) and receive ultrasound data and / or ultrasound images from the probe (20). At this time, the activation signal may include a signal for controlling the operation of the probe (20).

[0163] In one embodiment of the present disclosure, the probe (20) can transmit device information (e.g., ID information) of the probe (20) to an ultrasound imaging device (40) using a first communication method (e.g., BLE) and can be wirelessly paired with the ultrasound imaging device (40). Additionally, the probe (20) can transmit ultrasound data and / or ultrasound images to the ultrasound imaging device (40) paired by the first communication method using a second communication method (e.g., 60 GHz millimeter wave, Wi-Fi).

[0164] The ultrasound imaging device (40) can receive device information (e.g., ID information) of the probe (20) from the probe (20) using a first communication method (e.g., BLE) and can be wirelessly paired with the probe (20). Additionally, the ultrasound imaging device (40) can transmit an activation signal to the paired probe (20) and receive ultrasound data and / or ultrasound images from the probe (20) using a second communication method (e.g., 60 GHz millimeter wave, Wi-Fi).

[0165] According to one embodiment of the present disclosure, a first communication method used to pair a probe (20) and an ultrasonic imaging device (40) with each other may have a frequency band lower than the frequency band of a second communication method used to transmit ultrasonic data and / or ultrasonic images to the probe (20) and the ultrasonic imaging device (40).

[0166] The display (140) of the ultrasound imaging device (40) can display UIs (User Interfaces) indicating device information of the probe (20). For example, the display (140) can display identification information of the wireless ultrasound probe (20), a pairing method indicating a method of pairing with the probe (20), a data communication status between the probe (20) and the ultrasound imaging device (40), a method of performing data communication with the ultrasound imaging device (40), or a UI indicating the battery status of the probe (20).

[0167] If the probe (20) includes a display (112), the display (112) of the probe (20) may display a UI indicating device information of the probe (20). For example, the display (112) may display identification information of the wireless ultrasound probe (20), a pairing method indicating a pairing method with the probe (20), a data communication status between the probe (20) and the ultrasound imaging device (40), a method for performing data communication with the ultrasound imaging device (40), or a UI indicating the battery status of the probe (20).

[0168] The communication module (160) can receive control signals or data from an external device. The processor (120) can control the operation of the ultrasonic imaging device (40) according to the control signals received through the communication module (160).

[0169] Additionally, the processor (120) can transmit a control signal to an external device through the communication module (160) and control the external device according to the transmitted control signal. The external device can operate according to the control signal received from the ultrasonic imaging device (40) or process data received from the ultrasonic imaging device (40).

[0170] An external device may receive or download a program or application related to the ultrasound imaging device (40) from the ultrasound imaging device (40), probe (20), or server, and install and execute the program or application on the external device. The ultrasound imaging device (40), probe (20), or server providing the program or application may include a recording medium that stores instructions, commands, installation files, executable files, or related data of the said program or application. It is also possible for the external device to be sold with the program or application installed.

[0171] The memory (150) can store various data or programs for driving and controlling the ultrasonic imaging device (40), input / output ultrasonic data, ultrasonic images, etc.

[0172] An example of an ultrasound imaging system (100) according to one embodiment of the present disclosure is described later through FIGS. 2a, 2b, 2c, and 2d.

[0173] FIGS. 2a, FIGS. 2b, FIGS. 2c, and FIGS. 2d are drawings illustrating an ultrasonic imaging device according to one embodiment of the present disclosure.

[0174] Referring to FIGS. 2a and 2b, the ultrasound imaging device (40a, 40b) may include a main display (121) and a sub-display (122). The main display (121) and the sub-display (122) may correspond to the display (140) of FIGS. 1a and 1b. At least one of the main display (121) or the sub-display (122) may be implemented as a touchscreen. At least one of the main display (121) or the sub-display (122) may display various information processed in the ultrasound image or the ultrasound imaging device (40a, 40b). Additionally, at least one of the main display (121) or the sub-display (122) may be implemented as a touchscreen and provide a GUI (Graphic User Interface) to receive data input from a user for controlling the ultrasound imaging device (40a, 40b). For example, the main display (121) displays an ultrasound image, and the sub-display (122) can display a control panel for controlling the display of the ultrasound image in the form of a GUI. The sub-display (122) can receive data for controlling the display of the image through the control panel displayed in the form of a GUI. For example, a TGC (Time Gain Compensation) button, an LGC (Lateral Gain Compensation) button, a Freeze button, a trackball, a jog switch, or a knob, etc., can be provided as a GUI on the sub-display (122).

[0175] The ultrasonic imaging device (40a, 40b) can control the display of the ultrasonic image displayed on the main display (121) using the input control data. Additionally, the ultrasonic imaging device (40a, 40b) can be connected to the probe (20) via a wired or wireless connection to transmit and receive ultrasonic signals to and from the target object.

[0176] Referring to FIG. 2b, the ultrasound imaging device (40b) may further include a control panel (165) in addition to the main display (121) and sub-display (122). The control panel (165) may include buttons, trackballs, jog switches, knobs, etc., and may receive data input from a user to control the ultrasound imaging device (40b). For example, the control panel (165) may include a TGC button (171), a Freeze button (172), etc. The TGC button (171) is a button for setting TGC values ​​according to the depth of the ultrasound image. Additionally, when the ultrasound imaging device (40b) detects input from the Freeze button (172) while scanning the ultrasound image, it may maintain the state in which the frame image at that point in time is displayed, capture the frame image at that point in time, or save the frame image at that point in time.

[0177] Meanwhile, buttons, trackballs, jog switches, knobs, etc. included in the control panel (165) may be provided as GUIs on the main display (121) or sub-display (122). Additionally, the ultrasonic imaging device (40a, 40b) may be connected to the probe (20) to transmit and receive ultrasonic signals to and from an object.

[0178] Additionally, the ultrasonic imaging device (40a, 40b) may include various types of input / output interfaces, such as speakers, LEDs, and vibration devices. For example, the ultrasonic imaging device (40a, 40b) may output various information in the form of graphics, sound, or vibration through the input / output interface. Additionally, the ultrasonic imaging device (40a, 40b) may output various notifications or data through the input / output interface.

[0179] Referring to FIG. 2c and FIG. 2d, the ultrasound imaging device (40c, 40d) can also be implemented as a portable device. Examples of the portable ultrasound imaging device (40c, 40d) may include, but are not limited to, a smartphone, laptop computer, PDA, or tablet PC including a probe and an application.

[0180] The ultrasound imaging device (40c) may include a main body (41). Referring to FIG. 2c, a probe (20) may be wired to one side of the main body (41). To this end, the main body (41) may include a detachable connection terminal to which the cable connected to the probe (20) is attached. The probe (20) may include a cable having a connection terminal that can be connected to the main body (41).

[0181] Referring to FIG. 2d, the probe (20) can be wirelessly connected to an ultrasound imaging device (40d). The main body (41) may include an input / output interface (e.g., a touch screen). The input / output interface may display ultrasound images, various information processed by the ultrasound imaging device, or a GUI.

[0182] The ultrasound imaging device (40d) and the probe (20) can establish communication or be paired using short-range wireless communication. For example, the ultrasound imaging device (40d) and the probe (20) can communicate using Bluetooth, BLE, Wi-Fi, or Wi-Fi Direct.

[0183] The ultrasound imaging device (40c, 40d) can execute a program or application related to the probe (20) to control the probe (20) and output information related to the probe (20). The ultrasound imaging device (40c, 40d) can perform operations related to the probe (20) while communicating with a predetermined server. The probe (20) can be registered with the ultrasound imaging device (40c, 40d) or registered with a predetermined server. The ultrasound imaging device (40c, 40d) can communicate with the registered probe (20) and perform operations related to the probe (20).

[0184] Additionally, the ultrasonic imaging device (40c, 40d) may include various types of input / output interfaces, such as speakers, LEDs, and vibration devices. For example, the ultrasonic imaging device (40c, 40d) may output various information in the form of graphics, sound, or vibration through the input / output interface. Additionally, the ultrasonic imaging device (40c, 40d) may output various notifications or data through the input / output interface.

[0185] According to one embodiment of the present disclosure, an ultrasound imaging device (40a, 40b, 40c, or 40d) may use an artificial intelligence (AI) model to process ultrasound images or obtain additional information from ultrasound images. According to one embodiment of the present disclosure, an ultrasound imaging device (40a, 40b, 40c, or 40d) may use an AI model to generate ultrasound images or perform processing such as correction, image quality improvement, encoding, or decoding on ultrasound images. Additionally, according to one embodiment of the present disclosure, an ultrasound imaging device (40a, 40b, 40c, or 40d) may use an AI model to perform processing such as defining a baseline, obtaining anatomical information, obtaining lesion information, surface extraction, defining boundaries, measuring length, measuring area, measuring volume, or generating annotations from ultrasound images.

[0186] The AI ​​model may be provided on an ultrasound imaging device (40a, 40b, 40c, or 40d) or on a server.

[0187] AI models can be implemented using various artificial neural network models or deep neural network models. Additionally, AI models can be trained and generated using various machine learning algorithms or deep learning algorithms. For example, AI models can be implemented using models such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), GAN (Generative Adversarial Network), or LSTM (Long Short-Term Memory).

[0188] Referring to FIG. 3, the robotic ultrasound diagnostic system (200) may include a first robotic arm (210) configured to support an ultrasound probe (20), a second robotic arm (220) configured to support an auxiliary device (20a), and / or an electric bed (230).

[0189] The first robot arm (210) is composed of a multi-joint robot arm that supports and controls the ultrasonic probe (20). The first robot arm (210) may be configured to include multiple joints to enable movement of six or more degrees of freedom, thereby allowing the ultrasonic probe (20) to be brought into contact with the diagnostic area of ​​the target at an accurate position and angle. Each joint of the first robot arm (210) is controlled by a driving motor such as a servo motor or a stepper motor, and can provide accurate position feedback through an encoder.

[0190] An ultrasonic probe (20) may be directly fixed to the end portion of the first robot arm (210), or the ultrasonic probe (20) may be mounted via a support device (300, see FIG. 17) to be described later. The first robot arm (210) can precisely control the position, direction, and pressure of the ultrasonic probe (20), perform a scanning operation along a preset path, or adaptively adjust its movement based on an ultrasonic image acquired in real time.

[0191] The second robot arm (220) is composed of a multi-joint robot arm that supports and controls the auxiliary device (20a). Like the first robot arm (210), the second robot arm (220) may also be configured to include multiple joints to enable movement of 6 degrees of freedom or more. The second robot arm (220) is controlled independently of the first robot arm (210) and can contact or insert the auxiliary device (20a) at a specific location on the target at an accurate angle.

[0192] The auxiliary device (20a) may include a device for various auxiliary medical procedures performed together with ultrasound diagnosis. For example, the auxiliary device (20a) may include an external vibrator, a cauterization needle (e.g., RF thermal therapy needle), a biopsy needle, a contrast agent injection needle, or other medical procedure tools. As described below, the robotic ultrasound diagnostic system (200) may calculate the optimal position and angle of the auxiliary device (20a) based on ultrasound image information acquired by the first robotic arm (210), and control the second robotic arm (220) to move it accurately to the corresponding position.

[0193] The external vibration device is a device that generates vibration of an object for the generation of transverse wave elastic images, and may include a pulser that generates high-intensity ultrasonic pulses and a mechanical vibrator that transmits mechanical vibrations.

[0194] Biopsy needles are needles used to collect tissue from a subject, and depending on the purpose, they can have different needle diameters and lengths.

[0195] The contrast agent injection needle may include a storage unit for storing a contrast agent and a needle for delivering the contrast agent stored in the storage unit to a subject.

[0196] The electric bed (230) is configured as an electric bed on which a subject (patient) can be positioned. The electric bed (230) can be configured to move up and down, left and right, and forward and backward, and also to allow for tilt angle adjustment. The electric bed (230) includes a plurality of drive motors and a control system, and can optimize the position of the subject in conjunction with the operation of the first robot arm (210) and the second robot arm (220).

[0197] In particular, the electric bed (230) plays an important role in safety functions. When a predefined hazardous condition is detected, for example, when a malfunction of the robot arm or excessive pressure is detected, the electric bed (230) can automatically move to a predefined safety position to protect the object. This safety position is designed so as not to interfere with the operating range of the first robot arm (210) and the second robot arm (220), thereby ensuring the safety of the object even in emergency situations.

[0198] The robot ultrasonic diagnostic system (200) also includes a control unit (250, see FIG. 4) to integrally control the operation of the first robot arm (210), the second robot arm (220), and the electric bed (230). The control unit (250) can perform functions such as real-time analysis of ultrasonic image data, safety monitoring of the target object, path planning and control of the robot arms, and integrated processing of various sensor data.

[0199] FIG. 4 is a control block diagram of a robotic ultrasound diagnostic system according to one embodiment of the present invention.

[0200] Referring to FIG. 4, the robotic ultrasound diagnostic system (200) may include a camera (260), a microphone (270), a pressure sensor (280), a control unit (250) including a processor (251) and a memory (252), an ultrasound imaging system (100), a robotic arm (205) including a first robotic arm (210) and a second robotic arm (220), an electric bed (230) and / or a communication unit (290).

[0201] The control unit (250) is a core component that controls the overall operation of the robot ultrasound diagnostic system (200), and can identify a target diagnostic area based on an image of the target, control the scanning of the ultrasound probe, and process ultrasound image data to identify a region of interest and anatomical structures. In addition, the control unit (250) can perform a high-level computational function to calculate the target coordinates of the target point where the ultrasound probe (20) supported by the robot arm (205) (e.g., the first robot arm (210)) contacts the target, the location information of the identified region of interest, and the location information of the anatomical structures, and the target angle between the auxiliary device (20a) supported by the robot arm (205) (e.g., the second robot arm (220)) and the target, based on the contact coordinates of the point where the ultrasound probe (20) supported by the robot arm (205) (e.g., the first robot arm (210)) contacts the target, the location information of the identified region of interest, and the location information of the anatomical structures.

[0202] The processor (251) is a central processing unit that performs all computational and control operations of the system and may include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an NPU (Neural Processing Unit). The processor (251) performs complex computational tasks such as real-time analysis of ultrasound image data, execution of an artificial intelligence model, path planning and control signal generation for the robot arm, and safety monitoring. In particular, the processor (251) can identify anatomical structures and regions of interest, including bone and vascular structures, in the ultrasound image, and calculate the target position and target angle of the auxiliary device in real time based on this. Additionally, the processor (251) can execute an artificial intelligence model trained to calculate target coordinates and target angles using contact coordinates and ultrasound image data as input data.

[0203] The memory (252) may be a storage device that stores various data and programs necessary for system operation. The memory (252) may include RAM, ROM, flash memory, hard disk, etc., and may store an operating system, control algorithm, artificial intelligence model, ultrasound image data, robot arm control parameters, safety setting values, etc. In particular, the memory (252) may store ultrasound image frames acquired according to preset time intervals and spatial intervals by matching and storing coordinate information of contact coordinates corresponding to each, and may manage data for generating a three-dimensional ultrasound image based thereon. Additionally, the memory (252) may store other modality medical images (MRI, CT, etc.) and store target landmark information for aligning these images with ultrasound image data.

[0204] The camera (260) is an image sensor (or vision sensor) that acquires an image of an object and may include an RGB camera, a depth camera, or a thermal imaging camera. The camera (260) can recognize the external shape of the object to identify the target diagnostic area and monitor the object's movement or changes in facial expression to determine the safety status. The control unit (250) can detect the object's danger by inputting the image of the object acquired by the camera (260) into a risk analysis artificial intelligence model, and this can be utilized as one of the predefined risk conditions. In particular, based on the image of the object acquired from the camera (260), the control unit (250) can detect pain or discomfort by analyzing the object's facial expression, and the object's facial expression information can be used as a basis for judgment to adjust the operation of the robot arm (205) or stop the diagnosis.

[0205] In one embodiment, the camera (260) can be installed without limitation as long as it is in a location where it can acquire an image of the object.

[0206] The microphone (270) may be an acoustic sensor that collects acoustic signals from the subject and the surrounding environment. The microphone (270) may detect the subject's voice, breathing sounds, or sounds indicating pain, and the control unit (250) may detect the subject's danger by inputting the acoustic signals received through the microphone (270) into a risk analysis artificial intelligence model. For example, if the subject emits sounds such as "It hurts" or "It's hard," the microphone (270) may detect this, and the control unit (250) may determine this as a dangerous condition and take appropriate countermeasures.

[0207] The pressure sensor (280) may be a sensor that detects the pressure exerted by the ultrasonic probe (20) on the target in real time. The pressure sensor (280) can measure the contact pressure between the ultrasonic probe (20) and the target to monitor so that excessive pressure is not applied. If the measured pressure value is greater than a preset threshold pressure, this corresponds to a preset danger condition, and the control unit (250) can detect this and move the electric bed (230) to a preset position or adjust the operation of the robot arm (205).

[0208] The communication unit (290) is a communication interface that enables the robot ultrasound diagnostic system (200) to communicate with an external device, and may include at least one of a wired communication module, a wireless communication module, or a short-range communication module. The communication unit (290) can receive a remote control signal so that medical personnel located at a remote location can remotely control the robot ultrasound diagnostic system (200), and can transmit diagnostic results or system status information to the remote location. In addition, the communication unit (290) can receive patient information, past diagnostic records, and other modality medical images by linking with a hospital information system (HIS), a medical image storage and transmission system (PACS), or an electronic medical record (EMR) system. In particular, the communication unit (290) can receive other modality medical images, such as MRI and CT, and transmit them to the control unit (250) to be used for alignment with ultrasound images, and can store or transmit ultrasound image data and diagnostic results obtained during the diagnostic process to an external system. Other modality medical images received through the communication unit (290) can be stored in memory (252).

[0209] The ultrasound imaging system (100) may be a system that transmits and receives ultrasound signals through an ultrasound probe (20) and generates ultrasound images. The ultrasound imaging system (100) includes the configuration described above in FIGS. 1a and 1b and can provide ultrasound image data in real time in conjunction with a control unit (250). The ultrasound image data generated by the ultrasound imaging system (100) can be processed by the control unit (250) and utilized for identifying regions of interest and anatomical structures.

[0210] The robot arm (205) is a concept that encompasses the first robot arm (210) and the second robot arm (220), and may refer to multi-joint robot arms capable of independent control. The first robot arm (210) supports the ultrasonic probe (20) and can operate to scan the identified target diagnostic area with the ultrasonic probe according to the control of the control unit (250). The second robot arm (220) supports the auxiliary device and can be controlled by the control unit (250) so that the auxiliary device contacts the target object at a target angle calculated from the calculated target coordinates. The robot arm (205) receives a control signal from the control unit (250) to perform precise position control and operation, and can feed back position and velocity information of each joint to the control unit (250).

[0211] The electric bed (230) is an electric bed in which an object can be positioned, and it can adjust the position of the object according to a control signal from the control unit (250), and can automatically move to a predefined position to protect the object, especially when a predefined dangerous condition is detected. The operating range of the electric bed (230) is designed so as not to interfere with the operating range of the first robot arm (210) and the second robot arm (220), thereby ensuring the safety of the object even in emergency situations.

[0212] These components are organically linked around the control unit (250), enabling safe and accurate robotic ultrasound diagnosis and procedure.

[0213] FIG. 5 is a flowchart illustrating a control method for a robotic ultrasound diagnostic system according to an embodiment of the present invention.

[0214] Referring to FIG. 5, a control method for a robotic ultrasound diagnostic system may include the steps of receiving a diagnostic start command (1000), identifying a diagnostic area based on an image of an object (1100), controlling a first robotic arm to scan the diagnostic area (1200), identifying a region of interest and anatomical structure (1300), calculating target control information for an auxiliary device (1400), and controlling a second robotic arm (1500) based on the target control information.

[0215] In step 1000, the control unit (250) may receive a diagnostic start command. The diagnostic start command may be received in the form of at least one of manual input through a user interface, automatic execution according to a preset schedule, or a remote command from a remote control system.

[0216] The diagnostic start command may include the designation of a specific diagnostic protocol, for example, an abdominal ultrasound protocol, a cardiac ultrasound protocol, a thyroid ultrasound protocol, a breast ultrasound protocol, a musculoskeletal ultrasound protocol, or a vascular ultrasound protocol. Additionally, the diagnostic protocol may include more detailed examination types, for example, an abdominal ultrasound protocol for liver biopsy, a transrectal ultrasound protocol for prostate biopsy, an ultrasound protocol for thyroid nodule evaluation, or an echocardiogram protocol for heart valve evaluation may be selected.

[0217] Depending on the selection of the diagnostic protocol, the type of auxiliary device (20a) to be used may be automatically determined. For example, in the case of a liver fibrosis examination requiring elasticity measurement, an external vibration device may be selected as the auxiliary device (20a); in the case of a tissue examination requiring tissue examination, a biopsy needle may be selected as the auxiliary device (20a); and in the case of tumor treatment requiring tumor treatment, a cauterization needle may be selected as the auxiliary device (20a). Additionally, in the case of a blood flow evaluation using a contrast agent requiring contrast agent, a contrast agent injection needle may be designated as the auxiliary device (20a).

[0218] The diagnosis start command may also include diagnosis priority settings, for example, an emergency diagnosis mode, a precision diagnosis mode, or a screening diagnosis mode may be set. In response to receiving the diagnosis start command, the control unit (250) may initialize each component of the system and switch to a diagnosis ready state corresponding to the selected diagnosis protocol.

[0219] For example, the control unit (250) may activate the probe (20) supported by the first robot arm (210) upon receiving a diagnostic start command. Activating the probe (20) may include causing the probe (20) to start transmitting an ultrasonic signal.

[0220] According to various embodiments, the control unit (250) may activate the probe (20) after step 1100.

[0221] In step 1100, the control unit (250) can identify a target diagnostic area based on an image of the subject. To do this, the camera (260) captures a full or partial image of the subject, and the control unit (250) can analyze the acquired image data to automatically detect a specific body part requiring diagnosis. The process of identifying the diagnostic area may utilize image processing algorithms, pattern recognition technology, or deep learning-based body part segmentation models. Additionally, the control unit (250) can more accurately identify the diagnostic target area by referring to pre-entered patient information, diagnostic prescriptions, or instructions from medical staff. The identified diagnostic area may include various anatomical regions such as the abdomen, heart, thyroid, breast, and musculoskeletal system.

[0222] In step 1200, the control unit (250) can control the first robot arm (210) to scan the identified target diagnostic site with an ultrasound probe. The control of the first robot arm (210) can be performed by taking into account the anatomical characteristics of the diagnostic site, the calculation of the optimal scan path, and the maintenance of appropriate contact pressure between the ultrasound probe (20) and the target. The control unit (250) can determine a scan pattern based on the shape and size of the diagnostic site and generate joint-specific control signals so that the first robot arm (210) performs accurate and consistent movements along the pattern. Additionally, the contact force can be adjusted in real time to prevent excessive pressure from being applied by utilizing feedback from the pressure sensor (280) during the scanning process.

[0223] In step 1300, the control unit (250) can process ultrasound image data obtained by scanning the ultrasound probe to identify regions of interest and anatomical structures. The identification of regions of interest may be specialized according to a selected diagnostic protocol; for example, in the case of a liver biopsy protocol, a mass or nodule within the liver parenchyma, an area suspected of liver cirrhosis, or an area requiring histological evaluation may be identified as a region of interest. In the case of a thyroid examination protocol, a thyroid nodule, cyst, or calcified area may be identified as a region of interest, and in the case of a heart examination protocol, an area such as a valve structure or an area of ​​ventricular wall motion abnormality may be identified as a region of interest. In the case of a breast examination protocol, a mass, microcalcification, or structural distortion area may be identified as a region of interest, and in the case of a musculoskeletal protocol, a tendon injury, joint effusion, or inflammatory change area may be identified as a region of interest. In the case of a contrast agent examination protocol requiring the injection of a contrast agent prior to ultrasound diagnosis, an area including a vein to be injected with the contrast agent may be identified as a region of interest.

[0224] The identification of anatomical structures may involve the process of recognizing bone structures, vascular structures, nerve structures, or the boundaries of major organs, and this may vary depending on the diagnostic protocol. For example, during an abdominal examination, the boundaries of the liver, gallbladder, pancreas, and kidneys, as well as major blood vessels (aorta, inferior vena cava, portal vein, etc.), are identified, while during a cardiac examination, the boundaries of the atria and ventricles, as well as the major branches of the coronary arteries, may be identified. This identification task can be performed using image segmentation algorithms, feature extraction techniques, or artificial intelligence models specialized for medical image analysis. The identified structures can be utilized as important reference points for determining the location of auxiliary devices in subsequent steps.

[0225] In one embodiment, at step 1200 and / or step 1300, the control unit (250) controls the first robot arm (210) to acquire ultrasonic image frames according to preset time intervals and spatial intervals, matches and stores coordinate information of contact coordinates corresponding to each acquired ultrasonic image frame, and can generate a three-dimensional ultrasonic image based on the stored ultrasonic image frames and contact coordinates corresponding to each.

[0226] For example, the control unit (250) can acquire continuous two-dimensional ultrasound images by moving the ultrasound probe (20) along a pre-planned three-dimensional path through the first robot arm (210). At this time, the time interval can generally be set in the range of 50-200ms, and the spatial interval can be adjusted in the range of 0.5-2mm according to the size of the diagnostic site and the required resolution.

[0227] At the time each ultrasound image frame is acquired, the accurate 3D position coordinates {x, y, z} and attitude angles {a, b, c} of the ultrasound probe (20) can be calculated in real time through the joint encoder information of the first robot arm (210). The control unit (250) can match this position information with the ultrasound image data at the corresponding time and store it in memory (252).

[0228] In the process of generating a three-dimensional ultrasound image, the control unit (250) can first convert each two-dimensional image frame into a common three-dimensional coordinate system. To do this, a coordinate transformation matrix that converts the pixel coordinates of each frame into actual anatomical spatial coordinates can be applied. Accurate spatial mapping can be performed by comprehensively considering the depth information of the ultrasound beam, the position and angle of the probe, and the physical characteristics of the ultrasound.

[0229] The control unit (250) can generate regular 3D volume data from irregularly distributed sampling points by applying an interpolation algorithm. Techniques such as linear interpolation, cubic spline interpolation, or Gaussian weighted interpolation can be applied to form a smooth and continuous 3D image.

[0230] The generated 3D ultrasound images can be displayed in various visualization modes. This may include translucent 3D display via volume rendering, tomographic reconstruction in an arbitrary plane, or visualization of tracheal boundaries via surface rendering. These 3D images can provide important information for improving diagnostic accuracy and planning procedures.

[0231] In one embodiment, at step 1200 and / or step 1300, the control unit (250) receives a medical image of another modality, processes the received medical image of another modality to identify a target landmark, and controls the first robot arm to scan the identified target diagnostic area with the ultrasound probe (20). In response to the identification of a target landmark in the ultrasound image data, the control unit stores the contact coordinates at the time when the ultrasound image data was acquired, and controls the first robot arm (210) based on the stored contact coordinates to align the ultrasound image data with the medical image of another modality.

[0232] Other modality medical images may include MRI (magnetic resonance imaging), CT (computed tomography), PET (positron emission tomography), or previously taken ultrasound images. These images may be received via the communication unit (290) through a hospital information system (HIS), a medical image storage and transmission system (PACS), or direct file transmission.

[0233] The control unit (250) can automatically identify anatomical landmarks in the received other modality images. Landmarks may be blood vessel branching points, characteristic structures of bones, organ boundaries, or specific shapes of lesions. In this identification process, an image segmentation algorithm, a feature point extraction technique, or a deep learning-based anatomical structure recognition model may be utilized.

[0234] During the ultrasound scanning process, the control unit (250) can detect the same landmark in the ultrasound image acquired in real time. Since the ultrasound image has different contrast and resolution compared to other modality images due to its characteristics, a multimodal image matching algorithm may be applied for landmark matching. Mutual information, normalized cross-correlation, or a deep learning-based feature matching network may be used.

[0235] When a landmark is successfully matched, the control unit (250) can precisely record the contact coordinates of the ultrasonic probe (20) at that time. These coordinates can serve as reference points for establishing a spatial correspondence between two image modalities. When multiple landmarks are identified, a least squares method or a robust estimation technique may be applied for more accurate matching.

[0236] Once alignment is complete, the control unit (250) can generate a fused image by converting information from other modality images into an ultrasound coordinate system. For example, the location of a tumor identified in MRI can be overlaid onto a real-time ultrasound image to display it, or a vascular structure identified in CT can be used as reference information for an ultrasound-guided biopsy.

[0237] In step 1400, the control unit (250) can calculate target control information for the auxiliary device (20a) based on the contact coordinates of the point where the ultrasonic probe (20), supported by the first robot arm (205), contacts the target object, the location information of the identified region of interest, and the location information of the anatomical structure. The operation of calculating the target control information for the auxiliary device (20a) may include complex three-dimensional spatial coordinate transformation, multiple constraint optimization, and real-time collision avoidance path planning.

[0238] For example, the control unit (250) can determine the contact coordinates where the ultrasonic probe (20) and the object are in contact based on control information of the first robot arm (205) supporting the ultrasonic probe (20) and image processing obtained by the camera (260).

[0239] Encoders are installed at each joint of the first robot arm (210), so that the rotation angle of each joint can be precisely measured in real time. Based on this joint angle information, the control unit (250) can calculate the end position and orientation of the ultrasonic probe (20) on the reference coordinate system of the robot system. Specifically, the control unit (250) can derive the three-dimensional coordinates and direction vector where the contact surface of the ultrasonic probe is finally located by sequentially multiplying the rotation transformation matrix of each joint.

[0240] At this time, since the physical shape (length, width, shape of the contact surface, etc.) of the ultrasonic probe (20) is pre-calibrated and stored in the memory (252), the control unit (250) can calculate the exact position of the contact surface where the actual ultrasonic signal is transmitted from the coordinates of the robot arm end. In addition, attitude information such as the inclination angle and rotation angle of the probe is also calculated and can be utilized in the subsequent coordinate transformation process.

[0241] The image of the object acquired through the camera (260) allows for visual confirmation of the actual shape of the object's surface and the contact state with the probe (20), which are difficult to determine solely from the robot arm control coordinates. The control unit (250) can automatically detect the contour of the ultrasonic probe (20), the surface of the object's skin, and the contact boundary between the two from the acquired image through an image processing algorithm.

[0242] Specifically, the control unit (250) can identify the boundary between the probe and the object within the image by utilizing computer vision techniques such as edge detection, contour extraction, or region segmentation. Additionally, when using stereo vision, structured light, or a Time-of-Flight (ToF) camera, three-dimensional depth information can be extracted from the two-dimensional image to more accurately determine the spatial location of the contact point.

[0243] Since the robot arm control coordinates and the camera image coordinates have different reference points and axes, a precise coordinate system transformation and calibration process is required to fuse them. The control unit (250) can accurately set the transformation relationship between the robot coordinate system and the camera coordinate system by utilizing the results of a previously performed Hand-Eye calibration.

[0244] For example, the control unit (250) can convert image pixel coordinates into 3D world coordinates by utilizing the camera's intrinsic parameters (focal length, principal point, lens distortion coefficient, etc.) and extrinsic parameters (position and attitude relationship between the camera and the robot reference point). At this time, a pinhole camera model or a fisheye lens model may be applied, and lens distortion correction may also be performed.

[0245] The control unit (250) can establish a probe coordinate system based on the contact coordinates of the ultrasonic probe (20) and calculate a coordinate transformation matrix that converts it into the global coordinate system of the robot system. At this time, the direction vector, tilt angle, and contact angle with the surface of the target object of the ultrasonic probe (20a) can all be taken into account.

[0246] In the process of calculating target coordinates, the control unit (250) can calculate the three-dimensional center point coordinates of the region of interest and set constraints to secure a minimum safe distance from the anatomical structure. For example, the control unit (250) can determine a region farther than a preset first threshold distance from the bone structure as a first candidate region, determine a region farther than a preset second threshold distance from the blood vessel structure as a second candidate region, determine a region within a preset third threshold distance from the contact coordinates as a third candidate region, and then calculate the optimal target coordinates within the intersection of these candidate regions. As another example, the control unit (250) can calculate target coordinates and target angles that can reach the region of interest while avoiding the bone structure and the blood vessel structure.

[0247] The determination of the target angle can be solved as a multi-objective optimization problem that simultaneously considers the physical characteristics of the auxiliary device (length, diameter, stiffness), insertion depth requirements, and the minimization of interference with surrounding structures. For example, the control unit (250) can apply an optimization pattern learned from a large amount of surgical data in real time by utilizing a deep neural network model that takes contact coordinates and ultrasound image data as input variables and target coordinates and target angle as output variables.

[0248] In one embodiment, at step 1400, the control unit (250) can calculate the target coordinates and the target angle using an artificial intelligence model trained to calculate the target coordinates and the target angle using contact coordinates and ultrasonic image data as input data.

[0249] Artificial intelligence models can be implemented in various forms, such as Deep Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks, or Transformer architectures. The choice of model can be determined based on the characteristics of the input data and the output requirements.

[0250] Training data can consist of a large volume of contact coordinates, ultrasound images, and information on optimal target coordinates and target angles for the respective situations, collected from actual medical procedures. The inclusion of data on various anatomical structures, lesion types, patient body types, and procedure types can improve the generalization performance of the model.

[0251] During the input data preprocessing process, contact coordinates can be normalized, and ultrasound images can be adjusted to standardized size and contrast. Additionally, the robustness of the model can be enhanced by applying transformations such as rotation, scaling, and brightness adjustment through data augmentation techniques.

[0252] The architecture of an artificial intelligence model can have a multi-input structure. Coordinate information is processed through a fully connected layer, and features from ultrasound images can be extracted through a convolutional layer. Subsequently, the two information streams are fused, and the target coordinates and target angle can finally be output via regression.

[0253] In the model training process, loss functions such as Mean Squared Error, Mean Absolute Error, or Huber Loss may be used. Additionally, constraints considering procedural safety may be added to the loss function. For example, maintaining a minimum distance from blood vessels or bone structures and adhering to anatomical limits of the insertion angle may be included as normalization items.

[0254] During the real-time inference process, the control unit (250) inputs the current contact coordinates and the ultrasonic image into the model to immediately obtain the optimal target coordinates and target angle. The inference time of the model is generally maintained within tens of milliseconds, making it suitable for real-time control.

[0255] Uncertainty estimation techniques may be applied to evaluate the reliability of the model. Confidence intervals of prediction results are calculated using methods such as Bayesian Neural Networks or Monte Carlo Dropout, and if the uncertainty is high, additional verification processes or user confirmation procedures may be initiated.

[0256] Furthermore, through a continual learning mechanism, the model can be continuously improved as new procedure data is accumulated. This enables adaptation to the preferences of specific hospitals or medical staff, the characteristics of patient groups, or new procedural techniques.

[0257] In step 1500, the control unit (250) can perform precise mechanical control of the second robot arm (220) based on the calculated target control information.

[0258] In the control process of the second robot arm (220), the control unit (250) can first derive target angle values ​​for each joint to achieve a target angle at the target coordinates of the auxiliary device through inverse kinematics calculation. At this time, iterative numerical analysis using a Jacobian matrix, geometric analysis methods, or polynomial approximation methods may be utilized. The control unit (250) can also perform additional optimization calculations to avoid singularities of the robot arm, comply with joint angle limits, and satisfy workspace constraints.

[0259] During the actual operation of the robot arm, the control unit (250) can ensure smooth and accurate movement to a target position by applying a feedback control algorithm, such as PID control, adaptive control, or model predictive control, to each joint. During the movement, the control unit (250) can calculate the probability of collision with the auxiliary device, the target object, the first robot arm (210), or the surrounding environment in real time, and dynamically replan the avoidance path if necessary. To this end, feedback information from a distance sensor, a vision sensor (260), or a force / torque sensor can be integrated and processed in real time.

[0260] After the auxiliary device (20a) reaches the target position, the control unit (250) can perform fine control to execute the specific function of the auxiliary device (20a). For example, if the auxiliary device (20a) is an external vibration device, it can generate vibration at a specific frequency and amplitude; in the case of a biopsy needle, it can perform insertion at a set speed and depth; and in the case of a contrast agent injection needle, it can inject the contrast agent at an accurate flow rate and pressure. Each of these control processes can be precisely adjusted to meet the safety standards and performance specifications of the medical device.

[0261] Through this series of control processes, the robotic ultrasound diagnostic system can automatically perform accurate and safe auxiliary procedures based on ultrasound image information.

[0262] FIG. 6 is a diagram illustrating a method for calculating a target position and a target angle based on coordinate information, tilt angle information, and object information of a robot arm according to an embodiment of the present invention.

[0263] Referring to FIG. 6, it can be seen that during the process of a robotic ultrasonic diagnostic system diagnosing an object, the actual contact coordinates of the ultrasonic probe (20) and the transmission angle of the ultrasonic beam are accurately calculated from the mechanical position information of the first robotic arm (210). This calculation process may include a complex calculation process based on robotic kinematics, coordinate transformation theory, and ultrasonic physics.

[0264] The coordinates {x1, y1, z1} of the first robot arm (210) represent the three-dimensional position of the end-effector of the robot arm in the reference coordinate system of the robot system. Here, x1, y1, and z1 represent position coordinates in the forward, backward, left, right, and up, respectively, and the units can generally be expressed in millimeters (mm) or centimeters (cm). The angles {a1, b1, c1} of the first robot arm (210) represent the three-dimensional rotational posture of the end-effector, and generally use Euler angles or roll-pitch-yaw expressions. Here, a1 may represent a yaw angle, rotation around the z-axis, b1 may represent a pitch angle, rotation around the y-axis, and c1 may represent a roll angle, rotation around the x-axis.

[0265] The control unit (250) can calculate the actual position of the ultrasonic probe (20) by performing forward kinematics calculations based on the coordinate and angle information of the first robot arm (210). In this process, the mechanical connection relationship between the robot arm end and the ultrasonic probe must be taken into account.

[0266] Specifically, when the ultrasonic probe (20) is fixed to the end of the first robot arm (210), the offset vector {△x, △y, △z} to the contact surface of the probe (20) may be measured in advance through calibration and stored in memory (252). The control unit (250) can calculate the contact coordinates {x2, y2, z2} by applying the following transformation matrix:

[0267] First, the rotation matrix R1 of the robot arm end section can be constructed as follows:

[0268] R1 = Rz(a1) Х Ry(b1) Х Rx(c1)

[0269] Here, Rx, Ry, and Rz are rotation matrices about the x, y, and z axes, respectively. In this case, the final contact coordinates can be calculated by rotating the probe's offset vector according to the robot arm's orientation and adding it to the robot arm's position:

[0270] {x2, y2, z2} = {x1, y1, z1} + R1 Х {△x, △y, △z}

[0271] The transmission angle {a2, b2, c2} of the ultrasonic beam (ultrasonic signal) transmitted by the ultrasonic probe (20) toward the target is closely related to the physical direction of the probe. The control unit (250) can calculate the actual transmission direction of the ultrasonic beam by considering the angle information {a1, b1, c1} of the robot arm and the mounting direction of the probe.

[0272] Generally, since the ultrasonic probe (20) transmits an ultrasonic beam along its axis, the main axis direction vector of the probe becomes the transmission direction of the ultrasonic beam. When the unit vector representing the direction in which the probe (20) is fixed to the robot arm (210) is denoted as {ux, uy, uz}, the transmission direction vector of the actual ultrasonic beam can be calculated as follows:

[0273] Transmission direction vector = R1 Х {ux, uy, uz}

[0274] To extract the transmission angle {a2, b2, c2} from this direction vector, an angle transformation using inverse trigonometric functions is required. For example, when the transmission direction vector is {dx, dy, dz}:

[0275] a2 = atan2(dy, dx) / Calculate angle

[0276] b2 = atan2(-dz, sqrt(dx² + dy²)) / Calculate pitch angle

[0277] c2 = Roll angle of the probe itself (generally 0 or a fixed value since probes are circular)

[0278] Meanwhile, the angle of the ultrasonic signal transmitted through the ultrasonic probe (20) can be changed according to the transmission settings, and the control unit (250) can correct the transmission angle {a2, b2, c2} based on the transmission settings of the ultrasonic signal.

[0279] For example, the control unit (250) can dynamically calculate the focusing direction and angle of the actual ultrasonic beam according to the focal point setting of the ultrasonic probe (20). When the focal point is changed, the transmission time delay for each element of the ultrasonic transducer array is adjusted, and as a result, the focusing direction of the ultrasonic beam changes.

[0280] Specifically, when the focal point coordinates are {xf, yf, zf}, the control unit (250) can calculate a direction vector from the probe contact coordinates {x2, y2, z2} to the focal point:

[0281] Focal direction vector = {xf-x2, yf-y2, zf-z2}

[0282] After normalizing this direction vector, the actual beam transmission angle can be calculated by adding the focal correction angles {△af, △bf, △cf} to the previously calculated base transmission angles {a2, b2, c2}:

[0283] Actual transmission angle = {a2+△af, b2+△bf, c2+△cf}

[0284] As another example, the control unit (250) can calculate a plurality of beam transmission angles by considering a beamforming setting that implements a steering function.

[0285] The location information of the region of interest (ROI) and anatomical structures (SS) identified in the image acquired by the ultrasound probe (20) is expressed in the ultrasound image coordinate system. The control unit (250) can establish a transformation relationship between the ultrasound image coordinate system and the global coordinate system of the robot system based on the previously calculated contact coordinates {x2, y2, z2} and transmission angles {a2, b2, c2}.

[0286] Through this coordinate system transformation, the actual location in three-dimensional space of the region of interest and anatomical structures identified in the ultrasound image can be calculated, and this can be used as key information for determining the target location of the auxiliary device (20a) in a subsequent step.

[0287] According to one embodiment of the present invention, the accurate contact coordinates of the ultrasonic probe (20) and the transmission angle of the ultrasonic beam can be calculated in real time from the mechanical position information of the first robot arm (210), and accordingly, the optimal target position and / or target angle of the auxiliary device (20a) can be determined.

[0288] FIG. 7 is a flowchart for explaining the operation when an auxiliary device according to one embodiment of the present invention is an external vibration device.

[0289] Referring to FIG. 7, the process of calculating target control information of the auxiliary device (20a) corresponding to step 1400 of FIG. 5 is more specifically subdivided, and for example, when the auxiliary device (20a) is an external vibration device, the entire process up to the step-by-step process for determining the optimal position of the external vibration device and the vibration generation operation is illustrated.

[0290] In step 2100, the control unit (250) can determine a first candidate region based on three-dimensional positional information of a bone structure identified in the ultrasound image data. Since the bone structure acts as a major obstacle in the propagation of ultrasound signals and shear waves, it is an anatomical structure that must be considered as a key factor in the positioning of the external vibration device.

[0291] For example, the control unit (250) may set all three-dimensional spatial coordinates located at least a first threshold distance (e.g., 15-20 mm) away from the boundary surface of each identified bone structure as first candidate regions. The first threshold distance may be dynamically determined by the control unit (250) by considering the acoustic shadow effect of the bone structure on transverse wave propagation, interference by reflected waves, and inefficient transmission of vibration energy through the bone, or it may be stored in advance in memory (252). If the shape of the bone structure is complex, the control unit (250) may set more precise candidate regions by calculating the shortest distance for each point, and if there is information on the thickness or density of the bone, it may adjust the threshold distance by additionally considering this.

[0292] In step 2200, the control unit (250) can determine a second candidate region based on the location of major vascular structures identified in the ultrasound image. Vascular structures are important anatomical structures that must be excluded from the contact location of the external vibration device, considering pulsation due to blood flow, potential deformation upon compression, and risk of bleeding.

[0293] The control unit (250) can apply different second threshold distances depending on the type and size of each blood vessel, such as arteries, veins, and capillaries. For example, a safety distance of 10-15 mm can be set for the aorta or major artery, 5-8 mm for medium-sized blood vessels, and 2-3 mm for small blood vessels. Additionally, the control unit (250) can utilize Doppler ultrasound information to apply a larger safety distance around blood vessels with high blood flow velocity and set a prohibited area extended along the blood vessel by considering the direction of travel of the blood vessel.

[0294] The first critical distance and the second critical distance may be different. For example, the second critical distance may be shorter than the first critical distance.

[0295] In step 2300, the control unit (250) may set an area within a third threshold distance centered on the contact coordinates {x2, y2, z2} of the ultrasonic probe (20) as a third candidate area. This is because if the external vibration device is too far from the ultrasonic probe, the efficiency of vibration energy transfer is reduced, and the measurement accuracy in the area of ​​interest may decrease due to the attenuation of the transverse wave.

[0296] The third critical distance can generally be set in the range of 30-50 mm and can be adjusted according to the tissue characteristics of the target, the vibration frequency, and the range of elasticity to be measured. The control unit (250) can set a spherical or elliptical region centered on the contact coordinates as the third candidate region, and can also set an asymmetric candidate region by considering the direction of the ultrasonic probe and the location of the region of interest. In addition, if there is information on the elastic characteristics or thickness of the tissue, the propagation distance of the transverse wave can be predicted based on this information to calculate a more precise critical distance.

[0297] In step 2400, the control unit (250) can calculate the target coordinates of the external vibration device by calculating the intersection of three previously determined candidate regions. This intersection region represents optimal location candidates that avoid interference with bone structures, minimize the risk of blood vessel damage, and simultaneously maintain an appropriate distance from the ultrasound probe.

[0298] When the intersection area is divided into multiple separate zones, the control unit (250) can determine the final target coordinates from the perspective of minimizing the distance to the area of ​​interest, optimizing accessibility from the surface of the target object, avoiding collision with the first robot arm (210), and optimizing the expected transverse wave propagation path. To solve this multi-objective optimization problem, the control unit (250) may apply optimization techniques such as weight-based score calculation, Pareto optimization, or genetic algorithms. For example, the control unit (250) can determine the target position of the external vibration device using an AI model trained to calculate the optimal target position of the external vibration device using contact position and ultrasonic image data as input data.

[0299] In step 1500, based on the determination of the target position of the external vibration device, the control unit (250) can control the second robot arm (220) so that the external vibration device reaches the target position.

[0300] After the second robot arm (220) moves the external vibration device to the target coordinates, the actual vibration generation operation can be performed in step 2500.

[0301] The control unit (250) can control the external vibration device to irradiate a push pulse toward the target or generate vibration of the target in response to the external vibration device making contact with the target at a target angle calculated at the calculated target coordinates.

[0302] During the vibration generation process, the control unit (250) can precisely control the vibration frequency (typically 50-400 Hz), vibration intensity, vibration duration (typically 100-500 ms), and vibration pattern (single pulse, continuous pulse, or frequency sweep). At the same time, the ultrasonic probe (20) is switched to a high-speed video mode to observe and record the propagation process of the transverse wave generated by the vibration in real time.

[0303] According to the present invention, much more consistent and accurate transverse wave elasticity images can be obtained compared to manual operation, which has clinically significant implications for the quantitative evaluation of liver fibrosis, muscle stiffness, tumor elasticity, etc.

[0304] Referring to FIG. 8, a specific process for determining the optimal position of an external vibration device within an ultrasound image acquired based on the contact position (x2, y2, z2) of the ultrasound probe (20) is visually illustrated.

[0305] In the ultrasound image, the region of interest (ROI), bone structure (SS1), and vascular structure (SS2) are clearly identified and displayed. The region of interest (ROI) is the target tissue for which transverse wave elastography is required, such as liver parenchyma, thyroid nodules, or muscle tissue. Bone structure (SS1) appears hyperechoic in the ultrasound image and can be identified by its characteristic appearance accompanied by a posterior acoustic shadow. Vascular structure (SS2) is identified through hypoechoic characteristics within the lumen and Doppler signals, and can also be distinguished by dynamic characteristics such as pulsating movement or deformation upon compression.

[0306] FIG. 8 illustrates the boundary of an area located at a first threshold distance (d1) from a bone structure (SS1). The first threshold distance (d1) is typically set to 15-20 mm, which is the minimum safe distance to minimize ultrasonic attenuation and transverse wave propagation interference caused by the bone. The control unit (250) can calculate an avoidance area by accurately extracting the contour of the bone structure and then performing a distance transform of d1 from each contour point. If the bone structure has a complex shape, the control unit (250) can set a more precise avoidance area by calculating the Euclidean distance from each point to the nearest bone surface.

[0307] FIG. 8 shows the boundary of an area located at a second critical distance (d2) from the blood vessel structure (SS2). The second critical distance (d2) can be set differently depending on the type and size of the blood vessel and is generally determined in the range of 5-15 mm. A larger safety distance is required for large-diameter arteries, while a relatively smaller distance may be sufficient for small blood vessels. The control unit (250) can set individual protection areas for each blood vessel by considering the diameter of the blood vessel, blood flow velocity, thickness of the blood vessel wall, etc., and may also apply an elliptical or cylindrical protection area extended along the direction of travel of the blood vessel.

[0308] FIG. 8 shows an area within a third threshold distance (d3) centered on the contact position (x2, y2, z2) of the ultrasound probe. The third threshold distance (d3) is generally set to 30-50 mm, which represents the maximum distance over which a transverse wave generated by an external vibration device can be effectively transmitted to the area of ​​interest. Since the degree of attenuation of the transverse wave varies depending on the viscoelastic properties of the tissue, the control unit (250) can adjust the value of d3 by considering the tissue characteristics measured in advance, or the patient's BMI, age, etc.

[0309] The area indicated by region (Ca) in FIG. 8 represents an intersection region that satisfies all three conditions described above. That is, it is a region that is further away from the bone structure than d1, further away from the blood vessel structure than d2, and simultaneously located within d3 of the contact location. All points within this region (Ca) can be considered as candidate locations for the placement of the external vibration device, and the control unit (250) ultimately selects one optimal point from among them.

[0310] FIG. 9 is a drawing illustrating an external vibration device according to one embodiment of the present invention in contact with an object at a target location.

[0311] Referring to FIG. 9, the control unit (250) can control the second robot arm (220) to position the external vibration device (20a) at a target position (x3, y3, z3) within the intersection area (Ca).

[0312] Based on the external vibration device (20a) being placed at a target location (x3, y3, z3), the control unit (250) can control the external vibration device (20a) to perform a vibration generation operation.

[0313] FIG. 10 is a flowchart illustrating the operation when the auxiliary device according to one embodiment of the present invention is a needle.

[0314] Referring to FIG. 10, the process of calculating target control information of the auxiliary device (20a) corresponding to step 1400 of FIG. 5 is more specifically subdivided, and for example, when the auxiliary device (20a) is a needle (e.g., cauterization needle, biopsy needle, contrast agent injection needle), the entire process up to the step-by-step process for determining the optimal position of the needle and the needle insertion process is illustrated.

[0315] In step 3100, the control unit (250) can calculate all possible paths through which the needle can reach a target within the region of interest without colliding with the bone, based on three-dimensional positional information of the bone structure identified in the ultrasound image, as first candidate paths. The bone structure is the most significant physical obstacle to needle insertion, which can cause bending, change of direction, or complete impossibility of penetration of the needle.

[0316] For example, the control unit (250) can calculate a straight path from all possible needle entry points on the surface of the object to the target object and determine whether each path intersects with a bone structure through a three-dimensional geometric intersection test. In this case, the bone structure is represented as a polygon mesh or voxel-based three-dimensional model, and accurate collision detection can be performed using a ray tracing algorithm or a distance field technique.

[0317] In addition, the control unit (250) can consider not only a complete straight path but also a fine curved path by considering the bending characteristics of the needle. By modeling the bending stiffness according to the material, diameter, and length of the needle, paths that can maintain a safe minimum distance even when passing near a bone structure can be calculated. Structural analysis techniques such as finite element analysis or beam theory can be applied in this calculation process.

[0318] In step 3200, the control unit (250) can calculate second candidate paths that can safely reach the target while avoiding major vascular structures identified in the ultrasound image. Avoiding vascular structures is essential for preventing bleeding, preventing vascular damage, and minimizing side effects of the procedure.

[0319] The control unit (250) can apply different safety distances depending on the type and size of the blood vessel. For example, a safety distance of 10-15 mm can be set for the aorta or major artery, 5-8 mm for medium-sized blood vessels, and 2-3 mm for small blood vessels. In addition, a dynamic safety zone can be set by considering the direction of travel of the blood vessel, branching pattern, and pulsation characteristics.

[0320] In step 3300, the control unit (250) can calculate the intersection paths that simultaneously satisfy the bone structure avoidance condition and the blood vessel structure avoidance condition.

[0321] In the process of calculating the intersection path, the control unit (250) can first represent the first candidate path and the second candidate path as sets, respectively, and calculate the intersection of the two sets. If the path is represented as a continuous curve, the path can be discretized and converted into a set of path points, and then the intersection can be calculated.

[0322] When the intersection result appears as multiple paths, the control unit (250) may determine the priority of the paths by applying additional evaluation criteria. Such criteria may include minimizing path length, optimizing needle insertion angle, minimizing damage to surrounding tissues, and improving surgical accuracy.

[0323] In step 3400, the control unit (250) can calculate the target coordinates and target angle required for actual needle insertion from the calculated intersection path. The target coordinates represent the three-dimensional position of the entry point where the needle first contacts the surface of the object, and the target angle represents the direction of needle insertion.

[0324] The target coordinates {x_target, y_target, z_target} are set as the starting point of the selected optimal path, which corresponds to a point on the surface of the target. The control unit (250) can fine-tune the optimal entry point by taking into account the curvature of the surface of the target, the thickness of the skin, and the characteristics of the underlying tissue.

[0325] The target angle {a_target, b_target, c_target} can be calculated from the direction vector from the entry point to the target. If the path is a straight line, this is calculated using simple vector normalization and angle transformation; however, if the path is curved, the tangent direction at the entry point can be used as the initial insertion angle. Additionally, the initial angle can be corrected by considering the bending characteristics of the needle and tissue deformation so that the expected final arrival position coincides with the target.

[0326] After the second robot arm (220) positions the needle at the target coordinates and target angle in step 1500, the actual needle insertion process can be performed in step 3500. The control unit (250) may apply a specialized insertion protocol depending on the type of needle (biopsy needle, cauterization needle, contrast agent injection needle, etc.).

[0327] For example, the control unit (250) can control the second robot arm (220) so that the needle reaches a target object within the region of interest (ROI) in response to the needle making contact with the target object at the calculated target angle at the calculated target coordinates.

[0328] During the needle insertion process, the control unit (250) can precisely control the insertion speed, insertion depth, and insertion force. The position of the needle can be continuously monitored through real-time ultrasound imaging, and deviations from the planned path can be detected and the path corrected if necessary. In addition, changes in tissue resistance, the degree of bending of the needle, and deformation of surrounding structures can be evaluated in real time to ensure safe and accurate insertion.

[0329] According to the present invention, significantly improved safety, accuracy, and reproducibility compared to conventional manual needle insertion procedures can be provided, which can directly contribute to the improvement of clinical outcomes for various invasive procedures such as biopsy, cauterization, and contrast agent injection.

[0330] FIG. 11 is a diagram illustrating a method for selecting a target position and a target angle of a needle in an ultrasound image according to an embodiment of the present invention.

[0331] Referring to FIG. 11, a specific process for determining the optimal insertion path and insertion conditions of a needle within an ultrasound image acquired based on the contact position (x2, y2, z2) of the ultrasound probe (20) is visually illustrated. This figure corresponds to an example showing the actual application results of steps 3100-3400 described in FIG. 10.

[0332] In the ultrasound image, various anatomical structures (SS1, SS2, SS3) to be avoided are clearly identified and displayed along with the target (od) located within the region of interest (ROI). The region of interest (ROI) is the target tissue requiring needle treatment, which may be, for example, tumor tissue, a cyst, or a suspected lesion requiring a biopsy. The target (od) is the specific point within the region of interest that the needle must actually reach, which may be the center of a tumor, a specific part of a cyst, or a representative site where a biopsy is most useful.

[0333] Anatomical structures (SS1, SS2, SS3) may each have different characteristics and avoidance requirements. For example, the first anatomical structure (SS1) may be a bone structure that is an absolute obstacle impossible to physically penetrate, the second anatomical structure (SS2) may be a major blood vessel requiring a safety distance of a certain amount, and the third anatomical structure (SS3) may be an important nerve or other organ that could cause serious complications upon damage.

[0334] The illustrated path (pt1) represents an optimized needle insertion path that accurately reaches the target (od) while safely avoiding all anatomical structures (SS1, SS2, SS3). This path is determined through the following complex optimization process as a result of the comprehensive calculation of the previously described steps 3100-3400.

[0335] The control unit (250) can first apply different avoidance strategies for each anatomical structure. In the case of SS1 (bone structure), since complete avoidance is required, only non-intersecting paths are considered through collision detection; in the case of SS2 (vascular structure), a safety distance is set according to the diameter and importance of the blood vessel and a path that maintains a distance greater than that is selected; and in the case of SS3 (other important structures), individual safety criteria according to the characteristics of the structure can be applied.

[0336] In the path optimization process, multi-objective optimization is performed that comprehensively considers the accuracy, safety, and efficiency of needle insertion, rather than merely satisfying avoidance conditions. For example, objective functions such as minimizing path length, minimizing needle bending, minimizing surrounding tissue damage, and optimizing insertion angle can be evaluated integrally by applying weights.

[0337] The coordinates {x3, y3, z3} of the point where path (pt1) contacts the surface of the object are of great significance as the starting point for needle insertion. These coordinates are not merely geometric intersection points, but are determined by comprehensively considering the following various factors.

[0338] The target angle {a3, b3, c3}, calculated from the direction of path (pt1), can define the three-dimensional insertion direction of the needle. The target angle {a3, b3, c3} can correspond to the angle that path (pt1) makes with the object.

[0339] In the actual needle insertion process, real-time monitoring and dynamic correction of the planned path and angle are important. The control unit (250) can continuously track the actual position and direction of the needle through ultrasound images and calculate the deviation from the planned path (pt1) in real time. If it exceeds an acceptable error range, the control unit can recalculate the remaining insertion path and modify the path so that it can reach the target (od).

[0340] In addition, if the position of anatomical structures (SS1, SS2, SS3) changes due to tissue deformation or movement of the object, the control unit (250) can detect this and dynamically update the avoidance strategy. Through this adaptive control, the safety and accuracy of the procedure can be maintained.

[0341] FIG. 12 is a diagram illustrating the needle approaching a target at a target position and target angle when the auxiliary device according to one embodiment of the present invention is a biopsy needle or a cauterization needle. FIG. 13 is a diagram illustrating the needle reaching a vein within the region of interest at a target position and target angle when the auxiliary device according to one embodiment of the present invention is a contrast agent injection needle.

[0342] Referring to FIG. 12, a biopsy needle or cauterizing needle supported by a second robot arm (220) is shown approaching a target object (od) within a region of interest (ROI) according to the target coordinates {x3, y3, z3} and target angle {a3, b3, c3} calculated through the path planning process described in FIG. 10 and FIG. 11.

[0343] In this case, the target (od) is the lesion tissue; in the case of a biopsy needle, it may be a suspected tumor or nodule requiring sample collection for tissue examination, and in the case of a cauterization needle, it may be tumor tissue requiring thermal treatment. The needle is inserted along a path designed to accurately reach the center or most representative part of the lesion while safely avoiding anatomical structures (SS1, SS2, SS3). The control unit (250) can continuously track the position of the needle through real-time ultrasound image monitoring and precisely control the second robotic arm (220) to minimize deviation from the planned path.

[0344] Referring to FIG. 13, a contrast agent injection needle supported by a second robotic arm (220) is shown accurately reaching a target object (od), a vein, located within the region of interest (ROI). Since the contrast agent injection needle must be accurately inserted into the lumen of the blood vessel, more precise position control may be required than in FIG. 12.

[0345] The target vein (od) is a blood vessel in which blood flow has been confirmed via Doppler ultrasound, and it may be a site requiring blood flow evaluation or angiography through the injection of a contrast agent. The control unit (250) can precisely control the insertion depth and angle so that the needle is safely positioned in the lumen of the blood vessel without penetrating the blood vessel wall, taking into account the diameter, direction of travel, and pulsation characteristics of the blood vessel. In addition, during the contrast agent injection process, the injection pressure and injection speed can be monitored to prevent damage to the blood vessel or leakage of the contrast agent.

[0346] FIG. 14 is a flowchart illustrating an operation to move an electric bed to a predefined position when a predefined danger condition is detected according to an embodiment of the present invention. FIG. 15 is a diagram illustrating the operating range of a robot arm and an electric bed according to an embodiment of the present invention.

[0347] Referring to FIG. 14, the safety control process of the robotic ultrasonic diagnostic system (200) may consist of steps 4100 and 4200.

[0348] In step 4100, the control unit (250) can continuously monitor whether a predefined risk condition occurs by analyzing data collected from various sensors in real time during system operation. The predefined risk condition may include a plurality of conditions such as the following.

[0349] For example, this is the case where the pressure value measured by the pressure sensor (280) is greater than a preset threshold pressure. This means that the ultrasound probe (20) is applying excessive pressure to the subject, which corresponds to a dangerous situation that may cause discomfort or tissue damage to the subject. The threshold pressure value can be adjusted according to the subject's age, body type, and characteristics of the diagnostic site, and can generally be set in the range of 2-5 N / cm².

[0350] Another example is when an image of an object acquired by a camera (260) is input into a risk analysis AI model and a risk to the object is detected. The risk analysis AI model can automatically detect pain, anxiety, or other stressful situations by comprehensively analyzing changes in the object's facial expressions, body movements, and posture. For example, the calculation of a pain index through facial expression analysis, detection of body tremors or stiffness, or abnormal changes in posture may be determined as danger signals.

[0351] Another example is when an acoustic signal received through the microphone (270) is input into a risk analysis artificial intelligence model, and the risk to the subject is detected. Acoustic-based risk detection can be performed by analyzing the subject's voice, groans, changes in breathing patterns, or sounds indicating pain. By utilizing natural language processing technology, specific appeals such as "it hurts," "it is difficult," or "please stop" can be detected, or the stress level can be evaluated through changes in the tone, pitch, and intensity of the voice.

[0352] Another example is when a malfunction of the first robot arm (210) or the second robot arm (220) is detected. A malfunction of the robot arm may be detected as an encoder signal anomaly, a motor drive error, a loss of control signal, or an excessive deviation from the expected trajectory. Additionally, vibration, shaking, or abnormal noise of the robot arm may also be judged as signs of a malfunction.

[0353] Another example is when a power outage or system power supply anomaly is detected. If fluctuations in input voltage or momentary power outages are detected by the power monitoring system, they can be immediately classified as hazardous conditions.

[0354] In order to control the movement of the electric bed (230) when a power outage or system power supply abnormality is detected, the electric bed (230) may be provided with an auxiliary battery that supplies power to control the movement of the electric bed (230).

[0355] Another example is when other system anomalies are detected. This may include communication errors, software exceptions, temperature anomalies, or mechanical failures.

[0356] If a dangerous condition is detected in step 4100, the control unit (250) can immediately execute step 4200 to automatically move the electric bed (230) to a predefined safe position. In this process, the control unit (250) can immediately stop all ongoing diagnostic and surgical operations and switch to an emergency mode that prioritizes the safety of the subject.

[0357] A predefined position can be set to a position that is safely separated from the operating range of the first robot arm (210) and the second robot arm (220) to completely block the possibility of physical contact between the object and the robot arm. The movement of the electric bed (230) is performed at a maximum safe speed, and continuous obstacle detection and collision avoidance functions can be operated during the movement process. For example, the operating range of the first robot arm (210) and the second robot arm (220) can be designed so as not to interfere with the predefined position of the electric bed (230).

[0358] Referring to FIG. 15, the operating range design principle for ensuring the safety of the system is clearly illustrated. An intentional height difference is set between the lowest operating range (m1) in the z-axis direction of the first robot arm (210) and the second robot arm (220) and the lowest operating range (m2) in the z-axis direction of the electric bed (230).

[0359] For example, the lowest operating range (m2) in the z-axis direction of the electric bed (230) is designed to be lower than the lowest operating range (m1) in the z-axis direction of the first robot arm (210) and the second robot arm (220), and this difference (m1-m2) corresponds to a safety distance k.

[0360] The safety distance k is a preset design distance that can be set generously by taking into account the thickness of the object (human body size). Generally, k can be set in the range of 200-400 mm, which ensures that even the largest body type of object can be completely separated from the robot arm's range of motion when the motorized bed moves to its lowest height.

[0361] With this design, when the electric bed (230) is moved to the lowest height (m2) in the event of a dangerous situation, the object is automatically moved out of the operating range of all robot arms, thereby ensuring physical safety. Additionally, the safety distance k provides sufficient clearance so that even if an unexpected malfunction or control error occurs in the robot arm, it cannot reach the object.

[0362] Through the combination of such hardware safety design and software risk detection system, the robotic ultrasonic diagnostic system of the present invention can respond quickly and effectively to various dangerous situations and ensure the safety of the subject as the top priority.

[0363] Meanwhile, if a dangerous condition is detected while the auxiliary device (20a) is in contact with the subject, for example, while the needle is inserted into the subject, moving the electric bed (230) to a normal safe position may cause greater danger to the subject. This is because if the subject moves suddenly while the needle is inserted, serious complications such as tissue damage, bleeding, or needle breakage may occur.

[0364] To prevent this, the control unit (250) may move the electric bed (230) in a direction corresponding to a calculated target angle in response to the detection of a predefined danger condition while the auxiliary device (20a) is in contact with the object. The direction corresponding to the calculated target angle may mean a direction toward the calculated target angle. For example, the control unit (250) may move the electric bed in a direction parallel to the insertion direction of the needle so that the needle can naturally come out of the object. Through this directional movement, safe separation can be achieved while minimizing the change in the relative angle between the needle and the object.

[0365] FIG. 16 is a drawing showing a cable connected to an ultrasonic probe according to one embodiment of the present invention extended into the interior of a robot arm.

[0366] Referring to FIG. 16, a structure is shown in which a wire (20w) connected to an ultrasonic probe (20) is extended and installed inside the body (210b) of the first robot arm (210). This internal wiring design can improve the safety, reliability, and operational efficiency of the robot ultrasonic diagnostic system.

[0367] The wire (20w) is a cable that provides an electrical connection for transmitting and receiving ultrasonic signals from an ultrasonic probe (20) to an ultrasonic imaging system (100), and may include a plurality of conductors. Specifically, the wire (20w) may include transmission lines for transmitting transmission signals to an ultrasonic transducer, reception lines for transmitting received echo signals, power lines for supplying power to the probe, and communication lines for communicating control signals between the probe and the system.

[0368] In a design for extending a wire (20w) into the body (210b) of the first robot arm (210), an internal path penetrating each joint of the robot arm must be precisely planned. The wire (20w) can start from the base of the robot arm and extend to the end portion while passing through a central hole along the rotation axis of each joint. At this time, the path and allowance length of the wire can be designed so that the torsional stress and bending stress applied to the wire are maintained within safe limits, taking into account the rotation range and rotation speed of each joint.

[0369] During the internal routing process of the wire (20w), protective measures are particularly important at the joints. Wire guides, bearing systems, or rotary connectors may be installed at each joint to protect the wire from damage during continuous rotational movement of the joint. Additionally, to prevent fatigue failure of the wire, bending radius limits, twist angle limits, and tension management systems may be applied.

[0370] In one embodiment, a connector may be formed on the main body of the probe (20), and a slot to which the connector can be docked may be provided at the end of the first robot arm (210). The slot to which the connector can be docked formed on the main body of the probe (20) may be connected to a wire (20w), and the control unit (250) can control the probe (20) and receive data obtained from the probe (20) using transmission lines for transmitting a transmission signal to an ultrasonic transducer included in the wire (20w), reception lines for transmitting a received echo signal, power lines for supplying power to the probe, and communication lines for communicating control signals between the probe and the system.

[0371] This internal wiring design of the present invention significantly improves the overall performance and safety of the robot ultrasonic diagnostic system, and in particular, can prevent accidents caused by the wire connected to the probe (20) twisting due to the movement of the robot arm.

[0372] FIG. 17 is a drawing for explaining the structure of a support device according to one embodiment of the present invention.

[0373] Referring to FIG. 17, the support device (300) may include a first support member (310), a second support member (320), a camera (330), and a fixed member (340) (or a rotating body (340)).

[0374] The first support member (310) is a main structural component of the support device (300) and corresponds to a base portion connected to a robot arm (e.g., the first robot arm (210)). The first support member (310) supports the weight of the entire support device (300) and performs the role of transmitting driving force and control signals transmitted from the robot arm to internal components.

[0375] The second support member (320) is a connecting structure extending from the first support member (310), providing an installation space for the camera (330) and serving as a mechanical connection with the fixed member (340). The second support member (320) is designed to provide stable structural support even during rotational movement of the support device (300), and power supply lines and signal transmission lines may be arranged inside. The shape of the second support member (320) may be implemented as a column extending upward from the first support member (310) as illustrated, and the height and angle may be precisely designed to secure the optimal viewing angle of the camera (330).

[0376] A camera (260) according to one embodiment may include a camera (330) installed on a support device (300).

[0377] The camera (330) is a visual sensor installed on the second support member (320) to monitor the operating status of the support device (300) and the contact situation with the object in real time. The camera (330) is positioned in a location where it can continuously observe the interaction between the devices mounted on the fixed member (340) and the object, and provides visual feedback information to the control member (250). The camera (330) can be implemented in various forms, such as an RGB color camera, a depth camera, or an infrared camera, and must have the capability to acquire high-resolution images and process images in real time.

[0378] The installation position of the camera (330) is optimized so that it can always capture the currently active device (first device (20b) or second device (20c)) regardless of the rotation of the support device (300). To this end, the camera (330) is positioned above the rotational center axis of the fixed part (340), and a wide field of view can be secured through a wide-angle lens or a rotation mechanism.

[0379] For example, a camera (330) installed on a support device (300) may have a shooting field of view facing either the first device (20b) (e.g., ultrasonic probe (20)) or the second device (20c) (e.g., auxiliary device (20a)) which is positioned at a first position according to the rotation of the support device (300), regardless of the rotation of the support device (300) (e.g., rotation of the fixed part (340)).

[0380] The fixed part (340) is a core component of the support device (300) and is implemented in the form of a rotatable platform.

[0381] The fixed part (340) is designed to be rotatable about a rotation axis (ra), and this rotation axis (ra) is a vertical axis penetrating the geometric center of the support device (300). The rotation axis (ra) can be supported through a high-precision bearing system.

[0382] The first fixing part (341) is a mounting interface located at one corner of the fixing part (340) to fix the first device (20b). The first device (20b) may correspond to either an ultrasonic probe or an auxiliary device, but the type of the first device (20b) is not limited thereto.

[0383] The second fixing part (342) is a mounting interface located at another corner of the fixing part (340) to fix the second device (20c). The second device (20c) is a device of a different type from the first device (20b), for example, if the first device (20b) is an ultrasonic probe, the second device (20c) can be an auxiliary device such as an external vibration device, a gel application device, or a cleaning device.

[0384] As another example, if the first device (20b) is a first ultrasonic probe (e.g., linear probe), the second device (20c) may be a second ultrasonic probe (e.g., convex probe) of a different type from the first ultrasonic probe.

[0385] The extension direction (dr1) of the device fixed to the first fixed part (341) and the extension direction (dr2) of the device fixed to the second fixed part (342) may have a symmetrical angular relationship with respect to the rotation axis (ra). For example, the angle formed by the rotation axis (ra) and the imaginary line (V1) extending the extension direction (dr1) of the device fixed to the first fixed part (341) may be the same as the angle formed by the rotation axis (ra) and the imaginary line (V2) extending the extension direction (dr2) of the device fixed to the second fixed part (342).

[0386] In terms of ensuring positional accuracy, when the device that is positioned at the first position by rotating the fixed part (340) 180 degrees is replaced from the first device (20b) to the second device (20c), the new device can form the same geometric relationship with respect to the object as the previous device. This means that accurate positional control is possible without a separate complex positional correction or recalibration process.

[0387] Due to the symmetrical design in terms of simplifying the control algorithm, the control unit (250) can perform device replacement with only a simple 180-degree rotation command, and complex 3D coordinate transformation or inverse kinematics calculation is not required. This improves the response speed of the system and reduces the computational load.

[0388] The rotational movement of the fixed part (340) is controlled by a precision stepper motor or servo motor, and the accuracy of the rotation angle is monitored in real time through a high-resolution encoder. When the equipment replacement condition is satisfied, the control unit (250) rotates the fixed part (340) exactly 180 degrees to move the device from an inactive state to an active position, and at the same time finely adjusts the position and orientation of the robot arm to maintain an optimal contact state with the target object.

[0389] According to one embodiment of the present invention, if equipment replacement is required during a diagnostic operation, the equipment can be replaced quickly, thereby drastically reducing the diagnostic time.

[0390] Although only the first fixing part (341) and the second fixing part (342) are shown in FIG. 17, according to one embodiment of the present invention, the support device (300) may be implemented in an extended configuration that additionally includes a third fixing part.

[0391] The first fixed part, the second fixed part, and the third fixed part may be arranged at equal intervals on the rotating body with respect to the rotation axis of the support device, or arranged at equal angles to each other. Specifically, a configuration in which the three fixed parts are arranged at equal intervals of 120 degrees around the rotation axis is the most common and efficient.

[0392] When n fixed parts (n is a natural number greater than or equal to 2) are provided, each fixed part can be arranged at equal intervals of 360 / n degrees around the axis of rotation.

[0393] The combination of devices fixed to the first fixed part (341) and the second fixed part (342) of the support device (300) can be varied according to the diagnostic protocol.

[0394] Since the support device (300) is a method of replacing devices through rotation, the devices fixed to the first fixed part (341) and the second fixed part (342) cannot be used simultaneously and can be used sequentially one by one.

[0395] In one embodiment, according to a diagnostic protocol (e.g., abdominal ultrasound), a convex probe may be fixed to the first fixing part (341), and a linear probe may be fixed to the second fixing part (342). For example, in an abdominal ultrasound, the control unit (250) may first perform a wide-range overall scan with the convex probe to evaluate the overall condition of the organ, and then, if observation of suspected lesions or detailed structures is required, switch to the linear probe to perform a high-resolution precision examination.

[0396] In one embodiment, according to a diagnostic protocol (e.g., cardiac ultrasound), a linear probe may be fixed to the first fixing part (341), and a cardiac probe may be fixed to the second fixing part (342). For example, the control unit (250) may first perform a wide-range overall scan with the linear probe to evaluate the condition of the blood vessels in a cardiac ultrasound, and then switch to the cardiac probe to perform a high-resolution precision examination to further evaluate the effect of arteriosclerosis on the heart.

[0397] In one embodiment, according to a diagnostic protocol (e.g., abdominal ultrasound), a convex probe may be fixed to the first fixation part (341), and a microconvex probe may be fixed to the second fixation part (342). For example, in an abdominal ultrasound, the control unit (250) may first perform a wide-range overall scan with a linear probe and then switch to a microconvex probe with a small curvature to view the anatomical structure of the kidney in more detail.

[0398] In one embodiment, according to a diagnostic protocol (e.g., fetal ultrasound examination), a 2D probe may be fixed to the first fixing part (341), and a 3D / 4D ultrasound probe may be fixed to the second fixing part (342). For example, the control unit (250) may first perform basic fetal condition evaluation and measurement with the 2D probe during a fetal ultrasound examination, and then switch to the 3D / 4D probe to evaluate fetal malformations or obtain detailed stereoscopic images.

[0399] In one embodiment, according to a diagnostic protocol (e.g., precision inspection), a low-frequency probe may be fixed to the first fixed part (341), and a high-frequency probe may be fixed to the second fixed part (342). For example, the control unit (250) can scan and inspect a wide area entirely with the low-frequency probe, and then switch to the high-frequency probe to inspect a narrow area with high resolution for abnormalities and areas requiring further review.

[0400] In one embodiment, according to a diagnostic protocol (e.g., a general ultrasound examination), a gel application device may be fixed to the first fixing part (341), and an ultrasound probe (20) may be fixed to the second fixing part (342). For example, the control unit (250) can perform an actual ultrasound diagnosis by having the gel application device evenly apply an appropriate amount of ultrasound gel to the diagnostic area of ​​the target body at the start of the diagnosis, and then having the support device rotate to be replaced with the ultrasound probe.

[0401] The device that can be fixed to the first fixed part (341) and the second fixed part (342) is not limited to the example described above and may be changed according to the ultrasonic diagnostic method and diagnostic protocol.

[0402] According to the diagnostic protocol, the first robot arm (210) of the robot ultrasound diagnostic system (200) can support a support device (300) or other device (e.g., an auxiliary device (20a), an ultrasound probe (20)), and the second robot arm (220) can support the support device (300) or other device (e.g., an auxiliary device (20a), an ultrasound probe (20)).

[0403] FIGS. 18a and FIGS. 18b are drawings for explaining the configuration of a fixed part when viewed from above of a support device according to one embodiment of the present invention.

[0404] Referring to FIG. 18a and FIG. 18b, the fixing portions (341, 342) of the support device (300) can provide an adaptive fixing mechanism to securely and accurately fix devices of various sizes and shapes.

[0405] FIG. 18a illustrates a fixing mechanism in which a fixed part (341, 342) separates and rotates a part of a rotating body (340). In this method, the fixed part (341, 342) is composed of two detachable clamping elements, which can rotate in opposite directions around a central axis of rotation.

[0406] FIG. 18b illustrates a fixing mechanism that separates and pulls (slides) a part of the rotating body (340). In this method, the fixing parts (341, 342) consist of a fixed clamp and a movable clamp, and the movable clamp can perform linear movement along a linear guide.

[0407] According to various embodiments, the first fixing mechanism (e.g., rotational type) applied to the first fixing part (341) and the second fixing mechanism (e.g., sliding type) applied to the second fixing part (342) may be different from each other.

[0408] Accordingly, the user can intuitively recognize the first fixed part (341) and the second fixed part (342).

[0409] When a diagnostic start command is input, the robot ultrasonic diagnostic system (200) can notify the user of the device to be fixed to each of the first fixed part (341) and the second fixed part (342) through the output interface.

[0410] In order for the user to intuitively distinguish between the first fixing part (341) and the second fixing part (342), the robot ultrasonic diagnostic system (200) can intuitively know which device to fix at which location by sharing information that the first device should be fixed to the first fixing part utilizing the first fixing mechanism and the second device should be fixed to the second fixing part utilizing the second fixing mechanism.

[0411] FIG. 19 is a drawing for explaining the structure of a gel application device according to one embodiment of the present invention.

[0412] Referring to FIG. 19, the gel application device (400) can perform the application and / or washing of the gel required during the ultrasonic diagnostic process. The gel application device (400) can be mounted on a fixed part of the support device (300) and used together with the ultrasonic probe (20).

[0413] The gel application device (400) may include a gel application section (410). The gel application section (410) may be implemented in various ways, such as in the form of a nozzle, a brush, or a roller, and may be optimized according to the size and shape of the area to be applied.

[0414] The gel storage unit (415) can store the gel supplied to the gel application unit (410). The gel storage unit (415) maintains the medical ultrasound gel at an appropriate viscosity and temperature, and may include a gel heating function or viscosity control function as needed. A pump system or a pressure control system is installed between the gel storage unit (415) and the gel application unit (410) to control the supply of gel at a constant flow rate and pressure.

[0415] During the gel application process, the control unit (250) determines the optimal gel application amount and application pattern by considering the size of the diagnostic area, the condition of the skin, and the characteristics of the ultrasound probe to be used. For example, in the case of a large abdominal area, a sufficient amount of gel is applied evenly, and in the case of a small area such as the thyroid, the use of excessive gel is prevented through precise amount control.

[0416] A cleaning unit (420) may be provided at the second end of the gel application device (400). The cleaning unit (420) may include a water spraying unit (421) and a contaminated gel suction unit (422), and automatically performs gel removal and cleaning operations on the surface of the target object after diagnosis is completed.

[0417] The water spray unit (421) can dissolve and remove the gel on the skin surface by spraying purified water or physiological saline solution stored in the water storage unit (425) at an appropriate pressure and temperature. The water spray unit (421) can precisely control the spray angle, pressure, and flow rate through an adjustable nozzle.

[0418] The water storage unit (425) is a tank for storing cleaning liquid and includes a temperature control function to minimize discomfort to the subject by supplying water at a temperature close to body temperature. Additionally, the water storage unit (425) is equipped with a water level sensor and a temperature sensor to monitor the storage status in real time and provide notifications for replenishment or replacement when necessary.

[0419] The contaminated gel suction part (422) can absorb the gel diluted by water spray and the washing water. For example, the contaminated gel suction part (422) may include a vacuum pump that provides suction power.

[0420] The contaminated gel storage unit (426) is a waste tank that temporarily stores the inhaled contaminated gel and washing water. The contaminated gel inhaled through the contaminated gel suction unit (422) can be stored in the contaminated gel storage unit (426).

[0421] According to various embodiments, the gel application device (400) may include only a gel application section (410) and a gel storage section (415), and a separate cleaning device may be provided that includes a water spraying section (421), a water storage section (425), a contaminated gel suction section (422), and a contaminated gel storage section (426).

[0422] For example, the cleaning function may be implemented as a separate, independent cleaning device from the gel application device (400). The independent cleaning device may be configured as a dedicated cleaning system comprising a water spraying unit (421), a water storage unit (425), a contaminated gel suction unit (422), and a contaminated gel storage unit (426).

[0423] When an independent cleaning device (hereinafter referred to as the 'cleaning device') is fixed to the third fixed part (343) of the support device (300), the performance of each can be optimized due to the functional separation of the gel application device and the cleaning device, and maintenance and replacement can also be performed individually.

[0424] The gel application device (400) (or washing device) can be fixed to the support device (300).

[0425] Various equipment can be fixed in various combinations to the fixed parts (341, 342) of the support device (300) according to the diagnostic protocol.

[0426] For example, a first device (20b, e.g., a first ultrasonic probe) may be fixed to the first fixed part (341), and a second device (20c, e.g., a second ultrasonic probe different in type from the first ultrasonic probe, a gel application device (400) (or a cleaning device)) may be fixed to the second fixed part (342).

[0427] In the case where there are three fixed parts (341, 342), a first device (20b, e.g., a first ultrasonic probe) may be fixed to the first fixed part (341), a second device (20c, e.g., a second ultrasonic probe or a gel application device (400) of a different type from the first ultrasonic probe) may be fixed to the second fixed part (342), and a third device (e.g., a second ultrasonic probe, a gel application device (400), or a cleaning device) may be fixed to the third fixed part.

[0428] FIG. 20 is a flowchart illustrating a series of processes for diagnosing an object through equipment replacement when a robotic ultrasonic diagnostic system according to one embodiment of the present invention supports a support device.

[0429] Referring to FIG. 20, a control method for a robotic ultrasonic diagnostic system may include a step of performing a gel application operation (5100), a step of determining whether the equipment replacement condition is satisfied (5200), a step of performing a first diagnostic operation after equipment replacement (5300), a step of determining whether the equipment replacement condition is satisfied (5400), a step of performing a second diagnostic operation after equipment replacement (5500), a step of determining whether the equipment replacement condition is satisfied (5600), and a step of performing a cleaning operation after equipment replacement (5700).

[0430] At least one of the steps illustrated in FIG. 20 may be omitted depending on the diagnostic protocol and the type of equipment mounted on the support device (300). Depending on the combination of equipment supported on the support device (300), only some of the steps may be performed.

[0431] For example, if only the first ultrasonic probe and the second ultrasonic probe are mounted on the support device (300), steps 5100 (gel application) and 5700 (washing) may be omitted and only steps 5300 and 5500 may be performed. Conversely, if only the gel application device and the ultrasonic probe are mounted, steps 5500 and 5700 may be omitted.

[0432] In step 5100, the control unit (250) controls the rotation of the support device (300) and the movement of the robot arm (205) so that the gel application device is positioned at the diagnostic site of the target, and then uses the gel application device (400) to perform a gel application operation on the diagnostic site of the target.

[0433] For example, the control unit (250) can identify the boundaries and area of ​​the diagnostic site in an image of an object acquired through the camera (260) and control the gel application device (400) so that an appropriate amount of ultrasound gel is evenly applied to the area. At this time, the amount of gel to be applied can be determined by comprehensively considering the size of the diagnostic site, the condition of the skin, and the characteristics of the ultrasound probe to be used.

[0434] During the gel application process, the control unit (250) can move the gel application unit (410) of the gel application device (400) while maintaining an appropriate distance from the surface of the target object so that a gel layer of uniform thickness and even distribution is formed. In addition, the temperature of the gel can be adjusted to be close to body temperature to minimize discomfort to the target object.

[0435] In step 5200, the control unit (250) can determine whether the gel application operation is completed and whether the equipment replacement condition is satisfied. When the gel application is completed (e.g., in 5200), the control unit (250) can perform equipment replacement for the next step, the first diagnostic operation.

[0436] The determination of equipment replacement conditions can be performed based on various criteria. For example, it may be evaluated whether the gel dispensing device has completed a preset dispensing pattern, whether the thickness of the applied gel is within an appropriate range, or whether a uniform gel distribution has been achieved through visual verification via a camera.

[0437] In step 5300, the control unit (250) can control the rotation of the support device (300) and the movement of the robot arm (205) so that the first ultrasonic probe (20) forms the same relative positional relationship and relative angle relationship that the gel application device (400) had formed with respect to the object based on the position and posture control information of the robot arm (205), and then perform a first diagnostic operation on the object using the first ultrasonic probe (20).

[0438] The first diagnostic operation generally corresponds to a diagnosis for the purpose of a full scan or screening. For example, if the first ultrasound probe is a low-frequency convex probe, it can perform an overall assessment of the condition of the entire abdomen or observe the overall morphology and echogenicity of major organs.

[0439] The control unit (250) can automatically identify areas of interest or suspected lesions by analyzing ultrasound images acquired during the first diagnostic operation process in real time. At this time, it can distinguish between normal tissue and abnormal tissue and determine areas requiring additional precision examination by utilizing an image analysis algorithm or an artificial intelligence model.

[0440] In step 5400, the control unit (250) can determine the necessity of a second diagnostic operation by analyzing the results of the first diagnostic operation. If a suspected lesion is found in the first diagnosis or an area requiring more precise observation is identified (e.g., in 5400), the control unit (250) can decide to replace it with a second ultrasound probe.

[0441] The criteria for judgment at this stage may include image characteristics, the size and location of the lesion, the echogenicity of the tissue, or pre-established diagnostic protocols. For example, if a hypoechoic nodule is detected in the liver parenchyma, it may be determined that a detailed examination using a radiofrequency probe is necessary.

[0442] In step 5500, the control unit (250) can control the rotation of the support device (300) and the movement of the robot arm (205) so that the second ultrasonic probe forms the same relative positional relationship and relative angle relationship that the first ultrasonic probe had formed with respect to the object, and then perform a second diagnostic operation on the object using the second ultrasonic probe.

[0443] The second diagnostic operation may correspond to a detailed examination of a specific region of interest identified in the first diagnosis. For example, if the second ultrasound probe is a high-frequency linear probe, detailed diagnosis such as observation of the microstructure of superficial lesions, evaluation of blood flow, or measurement of elasticity may be performed.

[0444] The first ultrasonic probe and the second ultrasonic probe may have different types of transducer arrays. For example, the first probe may use a low-frequency convex array of 2-5 MHz, and the second probe may use a high-frequency linear array of 7-15 MHz. Alternatively, the first probe may use a B-mode dedicated array, and the second probe may use an array with enhanced Doppler capabilities.

[0445] In step 5600, the control unit (250) can determine whether the second diagnostic operation is completed and whether a cleaning operation is necessary. If all necessary diagnoses are completed and the cleaning device is mounted on the support device (300) (e.g., 5600), the control unit (250) can determine to replace the equipment for the cleaning operation.

[0446] At this stage, the criteria for completing the diagnosis, the quality of the acquired images, and whether the diagnostic objective has been achieved can be comprehensively evaluated. Additionally, the condition of the subject or the necessity of further examination may be considered.

[0447] In step 5700, the control unit (250) can control the rotation of the support device (300) and the movement of the robot arm (205) so that the cleaning device forms the same relative positional relationship and relative angle relationship that the second ultrasonic probe had formed with respect to the object, and then perform a cleaning operation on the object using the cleaning device.

[0448] In the cleaning operation, the control unit (250) can first spray water at an appropriate temperature and pressure through the water spray unit to dissolve the gel on the surface of the object. Then, the diluted gel and cleaning water can be effectively removed through the contaminated gel suction unit to clean the surface of the object.

[0449] During the washing process, appropriate pressure and temperature are maintained to prevent skin damage to the subject, and washing may continue until complete gel removal is achieved. Finally, finishing work may be performed to ensure the surface of the subject is dry and clean.

[0450] According to various embodiments, the steps performed may vary depending on the combination of equipment mounted on the support device (300).

[0451] For example, if the first equipment (20b) mounted on the support device (300) is a first ultrasonic probe and the second equipment (20c) is a second ultrasonic probe, only steps 5300, 5400, and 5500 may be performed. As another example, if the first equipment (20b) mounted on the support device (300) is an ultrasonic probe (20) and the second equipment (20c) is a gel application device (400) including a washing unit (420), only steps 5100, 5200, 5300, 5600, and 5700 may be performed. As yet another example, if the first equipment (20b) mounted on the support device (300) is an ultrasonic probe (20) and the second equipment (20c) is a gel application device (400) not including a washing unit (420), only steps 5100, 5200, and 5300 may be performed. As another example, if the first equipment (20b) mounted on the support device (300) is an ultrasonic probe (20) and the second equipment (20c) is a cleaning device, only steps 5300, 5600, and 5700 may be performed. As another example, if the first equipment (20b) mounted on the support device (300) is a first ultrasonic probe and the second equipment (20c) is a second ultrasonic probe, and the third equipment is a gel application device (400) including a cleaning unit (420), all steps may be performed.

[0452] In one embodiment, when a gel application device (400) that does not include a cleaning unit (420), an ultrasonic probe (20), and a cleaning device that includes a cleaning unit (420) are each fixed to the first, second, and third fixed parts, the control unit (250) can perform gel application, diagnosis, and cleaning operations.

[0453] For example, the control unit (250) can control the rotation of the support device (300) and the movement of the robot arm (205) so that the gel application device (400) is positioned at the diagnostic site of the target, and then use the gel application device (400) to perform a gel application operation on the diagnostic site of the target.

[0454] When the gel application is completed, the control unit (250) controls the rotation of the support device (300) and the movement of the robot arm (205) based on the position and posture control information of the robot arm (205) so that the ultrasonic probe (20) forms the same relative positional relationship and relative angle relationship that the gel application device (400) had formed with respect to the target, and then performs a diagnostic operation on the target using the ultrasonic probe (20).

[0455] When the diagnostic operation is completed, the control unit (250) can control the rotation of the support device (300) and the movement of the robot arm (205) based on the position and posture control information of the robot arm (205) so that the cleaning device forms the same relative positional relationship and relative angle relationship that the ultrasonic probe (20) had formed with respect to the object, and then perform a cleaning operation on the object using the cleaning device.

[0456] FIG. 21 is a drawing illustrating the robot arm moving upward, rotating, and then moving downward again when the equipment replacement condition according to one embodiment of the present invention is satisfied.

[0457] Referring to FIG. 21, the process of the control unit (250) controlling the rotation of the support device (300) and the movement of the robot arm (205) so that the second device (20c) forms the same relative positional relationship and relative angle relationship that the first device (20b) had formed with respect to the target object may include the step of moving the robot arm (205) upward, the step of rotating the support device (300), and the step of moving the robot arm (205) downward.

[0458] The control unit (250) can precisely measure and store all positional relationship information currently formed between the first equipment (20b) and the target object at the time when the equipment replacement condition is satisfied. This information may include each joint angle of the robot arm (205), the current rotational position of the support device (300), contact coordinates between the first equipment (20b) and the target object, and the approach angle of the first equipment (20b) to the target object.

[0459] The control unit (250) can calculate a safe upward movement distance for equipment replacement based on stored location information. In this calculation process, the physical size difference between the first equipment (20b) and the second equipment, the turning radius of the support device (300), and a safety margin distance to prevent collision with the target object may be taken into account.

[0460] When executing an upward movement command, the control unit (250) can transmit a control signal corresponding to the upward movement to the vertical axis (Z-axis) direction drive motor of the robot arm (205).

[0461] After the robot arm (205) moves to a target height, the control unit (250) can transmit a rotation command to a drive unit (e.g., a rotary motor) that rotates the rotating body (340) of the support device (300) to perform a rotation operation for switching from the first equipment (20b) to the second equipment (20c). The rotation angle is determined by the fixed part arrangement of the support device (300) and can generally be set to an accurate angle of 180 degrees or 120 degrees.

[0462] The control unit (250) can perform precise control to accurately reach the target angle by monitoring the feedback of the angle encoder in real time during the rotation process.

[0463] When the rotation is complete, the control unit (250) may perform additional position verification to confirm whether the second equipment (20c) is placed in the correct position. At this time, visual verification through the camera (330) or mechanical verification through the position sensor may be performed in parallel.

[0464] After the rotation of the support device (300) is completed, the control unit (250) can perform downward movement of the robot arm (205) so that the second equipment (20c) accurately restores the relative positional relationship with the object that the first equipment (20b) had.

[0465] The downward movement distance can be precisely calculated by taking into account the original position information stored in step 1 and the physical difference between the first equipment (20b) and the second equipment (20c). In particular, if there is a difference in length or shape between the two equipment, additional position adjustments can be performed to compensate for this.

[0466] The control unit (250) can ensure safe access by continuously monitoring the distance to the target object during the downward movement process. Real-time distance measurement is performed using a proximity sensor, a camera (330), or an ultrasonic distance sensor, and can be controlled to accurately reach a set target distance.

[0467] The control unit (250) can fine-tune the posture of the robot arm (205) so that the second equipment (20c) forms an approach angle to the same object as the first equipment (20b). In this process, the required amount of rotation correction can be calculated by comparing the angle information stored in step 1 with the current direction of the second equipment (20c).

[0468] Angle adjustment can be performed by individually controlling the rotational joints (pan, tilt, and roll axes) of the robot arm (205). The control unit (250) can calculate the target angle of each axis and execute rotational movements simultaneously or sequentially to finally achieve the same approach angle as the first equipment (20b).

[0469] After all position and angle adjustments are completed, the control unit (250) can comprehensively verify whether the final position of the second equipment (20c) satisfies the requirements. In this verification process, the distance to the target, the approach angle, and whether there is interference with surrounding structures can be checked.

[0470] When verification is complete, the control unit (250) can activate the second equipment (20c). For example, if the first equipment (20b) is the first ultrasonic probe and the second equipment (20c) is the second ultrasonic probe, the control unit (250) can start beamforming of the second ultrasonic probe when the final movement of the second equipment (20c) is completed. As another example, if the first equipment (20b) is the first ultrasonic probe and the second equipment (20c) is the gel application device (400), the control unit (250) can control the gel application device (400) to apply power to the gel application device (400) and perform a gel application operation when the final movement of the second equipment (20c) is completed.

[0471] FIG. 22 is a drawing for explaining a method for calculating the required upper movement distance of a robot arm during rotation based on the difference in length between an ultrasonic probe supported by a first fixed part of a support device and an auxiliary device supported by a second fixed part according to an embodiment of the present invention.

[0472] Referring to FIG. 22, the control unit (250) can precisely analyze the physical size difference between the first equipment (20b) and the second equipment (20c) to perform a safe rotational operation for equipment replacement, and calculate the optimal upward movement distance based thereon.

[0473] The step of analyzing the physical size difference between the first equipment (20b) and the second equipment (20c) can be performed after receiving a diagnosis start command and before the diagnosis operation.

[0474] The control unit (250) can measure, through the camera (330), the minimum height of the first equipment when the first equipment (20b) is located at the first position (active position) and the minimum height of the second equipment when the second equipment (20c) is located at the first position, respectively, when the support device (300) is in a state prior to diagnostic operation.

[0475] The difference in the lowest height (k2) between the two devices may refer to the difference in vertical distance from the center of rotation of the support device (300) to the lowest point of each device. For example, if the first device (20b) is a relatively short ultrasonic probe and the second device (20c) is a long gel application device, the lowest height of the second device (20c) may be at a position k2 lower than the lowest height of the first device (20b).

[0476] The control unit (250) can calculate an upward target travel distance for safe rotation based on the measured height difference (k2). If the minimum height of the second equipment is k2 lower than the minimum height of the first equipment, there may be a risk that the second equipment will collide with an object or surrounding structures when the support device (300) rotates.

[0477] To prevent such collisions, the control unit (250) may determine the upward target travel distance as the distance k2 plus a safety margin. Information regarding the safety margin may be stored in memory (252) in advance.

[0478] The control unit (250) can calculate the travel distance by considering not only the replacement from the first equipment (20b) to the second equipment (20c) but also the reverse replacement from the second equipment (20c) to the first equipment (20b). If the first equipment (20b) is longer than the second equipment (20c), a separate upward movement may be required to prevent collision of the first equipment (20b) during reverse rotation.

[0479] Through this bidirectional consideration, the control unit (250) can set an integrated safety height based on the size of the longer of the two pieces of equipment, and enable safe replacement without the risk of collision in any direction of rotation.

[0480] In one embodiment, the control unit (250) can control the rotation direction of the support device (300) during the equipment replacement step from the first equipment (20b) to the second equipment (20c) and the rotation direction of the support device (300) during the equipment replacement step from the second equipment (20c) to the first equipment (20b) differently from each other when equipment replacement from the first equipment (20b) to the second equipment (20c) is required.

[0481] For example, the control unit (250) can rotate the support device (300) in a first direction during the equipment replacement step from the first equipment (20b) to the second equipment (20c), and can rotate the support device (300) in a second direction opposite to the first direction during the equipment replacement step from the second equipment (20c) to the first equipment (20b).

[0482] According to one embodiment of the present invention, an accident in which wires connected to equipment become tangled can be prevented when replacing equipment.

[0483] FIG. 23 is a flowchart illustrating the operation of deactivating a first probe and then activating a second probe when an auxiliary device supported by a support device according to one embodiment of the present invention is a probe.

[0484] Referring to FIG. 23, a process in which a control unit (250) performs a replacement between probes is illustrated in detail when a first ultrasonic probe and a second ultrasonic probe are respectively mounted on a support device (300). This process may be included in step 5500 (performing a second diagnostic operation after equipment replacement) of FIG. 20.

[0485] In step 6100, the control unit (250) may deactivate the first ultrasonic probe that is currently active if the equipment replacement condition is satisfied during the first diagnostic operation using the first ultrasonic probe. Deactivating the first ultrasonic probe may mean cutting off the power supplied to the first ultrasonic probe and stopping beamforming control.

[0486] The control unit (250) can stop processing the echo signal being received from the first ultrasound probe. The last image frame and related data acquired before the first ultrasound probe is deactivated are stored in memory (252) and can be used for subsequent analysis or for diagnosis in conjunction with the second ultrasound probe.

[0487] The control unit (250) can record control information of the first ultrasonic probe when the equipment replacement condition is satisfied. This information may include transmission angle information of the ultrasonic signal, focal point information, beamforming parameters, gain setting values, and diagnostic mode settings. This first control information can be used as important reference data for optimizing the settings of the second ultrasonic probe.

[0488] In step 6200, the control unit (250) can control the robot arm (205) and the support device (300) so that the second ultrasonic probe forms the same relative positional relationship and relative angle relationship that the first ultrasonic probe had formed with respect to the target object.

[0489] The control unit (250) can precisely measure and store position information of the first ultrasonic probe when the equipment replacement condition is satisfied. This information may include contact coordinates between the probe and the target, contact pressure, tilt angle of the probe, and transmission direction of the ultrasonic beam.

[0490] The control unit (250) can calculate a safe upward travel distance by considering the physical size difference (k2) between the first ultrasonic probe and the second ultrasonic probe as described in FIG. 22. At this time, the difference in length, width, and shape of the contact surface of the two probes can all be considered to plan an optimal travel path.

[0491] After the upward movement of the robot arm (205) is completed, the control unit (250) can perform precise rotation control through the drive unit of the support device (300). The rotation angle is determined according to the fixed arrangement of the first probe and the second probe, and accurate rotation of a preset angle (e.g., 180 degrees) can be performed. During the rotation process, real-time angle monitoring and precise control can be performed through encoder feedback.

[0492] After the rotation is completed, the control unit (250) can perform downward movement and fine posture adjustment of the robot arm (205) so that the second ultrasonic probe satisfies the same object contact conditions as the first probe.

[0493] In step 6300, the control unit (250) can activate the second ultrasonic probe, for which the positional relationship restoration is complete. In this process, an optimized setting based on control information inherited from the first probe may be applied.

[0494] The control unit (250) can set the second control information of the second ultrasonic probe based on the first control information prior to the first ultrasonic probe being deactivated. At this time, appropriate parameter conversion can be performed by taking into account the difference in technical characteristics between the two probes.

[0495] For example, if the first probe is a low-frequency convex probe (2-5 MHz) and the second probe is a high-frequency linear probe (7-15 MHz), the focal point and transmission angle can be adjusted by considering the penetration depth and resolution characteristics according to the difference in frequency bands. The control unit (250) can apply an optimal conversion algorithm by utilizing the characteristic matrix of the two probes.

[0496] The transmission angle information and focal point information of the ultrasound signal can be recalculated to match the transducer array characteristics of the second probe. The control unit (250) adjusts beamforming parameters to observe the same anatomical region under optimal conditions and can apply multiple focal regions or complex focusing techniques as needed.

[0497] FIG. 24 is a drawing for explaining a structure in which an ultrasonic probe supported by a support device according to one embodiment of the present invention includes first and second transducer arrays having different scan directions.

[0498] Referring to FIG. 24, it is illustrated that a single ultrasonic probe (20) has a bidirectional transducer structure, enabling ultrasonic scanning in different directions through the rotation of the support device (300).

[0499] The ultrasonic probe (20) may include a first transducer array (20g) and a second transducer array (20h) facing different directions within a single physical probe body.

[0500] The first transducer array (20g) is positioned to face the first direction of the probe so as to perform a scan optimized for a specific anatomical structure or diagnostic site. For example, it may be configured as a linear array and specialized for high-resolution observation of superficial structures.

[0501] The second transducer array (20h) is positioned to face the second direction of the probe and can provide scanning characteristics different from those of the first transducer. For example, it may be configured as a curved convex array suitable for extensive observation of deep structures.

[0502] When a bidirectional transducer probe (20) is mounted on a support device (300), the control unit (250) can selectively position either the first transducer array (20g) or the second transducer array (20h) to an active position by rotating the support device (300). This allows different scan modes to be implemented without physical probe replacement.

[0503] FIG. 25 is a drawing illustrating that the focal point is maintained even after equipment replacement according to one embodiment of the present invention.

[0504] Referring to FIG. 25, the focal point (fc) of the ultrasonic probe (20) and the ultrasonic signal transmission angle (Sa) are shown.

[0505] The focal point (fc) is a point where the ultrasound beam is maximally focused and is an important parameter that directly affects the quality and accuracy of the diagnosis. The control unit (250) can precisely record the three-dimensional spatial coordinates of the focal point (fc) that was set in the first ultrasound probe and control the formation of the focal point at the same anatomical location even after replacement with the second ultrasound probe.

[0506] For example, the control unit (250) may store the current focal point (fc) setting value in memory (252) before deactivating the first probe. This information may include the focusing depth from the probe surface, the lateral position of the beam focusing, and the size of the focusing area. In addition, performance indicators such as acoustic intensity, beam width, and focusing quality at the corresponding focal point may also be recorded.

[0507] When the second ultrasound probe is activated, the control unit (250) can convert the stored focal point information to match the physical and electrical characteristics of the second probe. The beamforming parameters can be recalculated to achieve optimal focusing at the same anatomical location by taking into account the difference in frequency band between the two probes, the difference in transducer array size, and the difference in aperture characteristics.

[0508] The transmission angle (Sa) is an important parameter that defines the direction in which the ultrasonic beam is directed toward the target, and can ensure optimal acoustic access to the diagnostic site. The control unit (250) can perform precise angle control so that the transmission angle setting of the first ultrasonic probe can be accurately reproduced in the second ultrasonic probe as well.

[0509] The transmission angle (Sa) set in the first probe may include multidimensional information such as beam steering angle, elevation angle, and composite focusing angle. The control unit (250) can store this angle information in a vector form and convert and apply it to the transducer array structure of the second probe.

[0510] In particular, when the first probe and the second probe have different array shapes (e.g., linear vs. convex), the control unit (250) can apply a geometric transformation matrix to calculate the transmission angle facing the same anatomical direction. This allows optimal acoustic conditions for the same tissue region to be maintained even when the probe is replaced.

[0511] The control unit (250) can perform a customized focal point holding mechanism and / or transmission angle holding mechanism based on each unique characteristic according to the type of probe (convex, linear, phased, etc.).

[0512] For example, if both the first probe and the second probe are convex probes, the control unit (250) can store arc coordinate system information of the focal point (fc) by considering the radius of curvature of the first probe and the fan-shaped scan pattern, and convert it to match the curvature characteristics of the second probe being replaced so that focusing is achieved at the same anatomical location.

[0513] As another example, when the first probe is a convex probe and the second probe is a linear probe, the control unit (250) can control the focal point (fc) set in the convex probe by converting polar coordinate system information based on the center of curvature into the orthogonal coordinate system of the linear probe so that the focal point is formed at the same anatomical location.

[0514] In the case of the transmission angle (Sa), in the convex probe, it is set as a radial beam steering angle based on the center of curvature, whereas in the linear probe, it is expressed as an angle of inclination relative to the linear array surface. The control unit (250) can control the transmission of an ultrasound signal directed toward the same anatomical direction by converting the radial transmission angle of the convex probe into a parallel beam steering angle of the linear probe through geometric transformation.

[0515] Through this customized maintenance mechanism for each probe type, even when replaced with a different type of probe, the previously set optimal focal point and transmission angle are maintained at the anatomically identical position, ensuring consistent diagnostic quality.

[0516] The control unit (250) can convert the first control information of the first ultrasonic probe into the second control information of the second ultrasonic probe. This process may include an intelligent conversion process that takes into account the difference in characteristics between the two probes, rather than a simple parameter copying process.

[0517] In addition to focal point information and transmission angle information, control information may further include various parameters such as transmission frequency and bandwidth settings, transmission power level, pulse repetition frequency, receive gain settings, time compensation gain (TGC) curve, dynamic range settings, and image post-processing parameters.

[0518] The control unit (250) can optimize these parameters to match the performance characteristics of the second probe. For example, when switching from a high-frequency probe to a low-frequency probe, appropriate parameter adjustments can be performed by considering attenuation compensation and changes in resolution characteristics due to increased penetration depth.

[0519] FIG. 26 is a flowchart illustrating the operation of replacing the equipment with a gel application device and proceeding with gel application when identifying a gel-unapplied area according to an embodiment of the present invention.

[0520] Referring to FIG. 26, a process is illustrated in which a control unit (250) automatically detects a gel-unapplied area during a diagnostic operation and performs a gel replenishment operation in response.

[0521] In step 7100, the control unit (250) can perform a diagnostic operation using the ultrasonic probe (20) and generate an ultrasonic image in real time. The ultrasonic image obtained in this process can be used not only for diagnostic purposes but also as information for evaluating the quality of the acoustic coupling state.

[0522] The control unit (250) can monitor image quality indicators by analyzing ultrasound image frames continuously acquired during the diagnostic process in real time. These indicators may include the signal-to-noise ratio, image uniformity, contrast, and degree of artifact occurrence.

[0523] In particular, the control unit (250) can focus on analyzing image characteristics that reflect the acoustic coupling state between the gel and the skin. In areas where acoustic coupling is poor, characteristic artifacts such as multiple reflections (reverberation), acoustic shadows, or signal attenuation may appear, and these patterns can be automatically recognized.

[0524] In step 7200, the control unit (250) can automatically identify areas where gel has not been applied based on the acquired ultrasound image. In this process, image analysis algorithms and artificial intelligence technology may be utilized.

[0525] The control unit (250) can analyze various visual indicators indicating acoustic coupling failure in the ultrasound image. For example, in areas where there is insufficient gel, a strong reflection signal may occur due to the air layer between the probe and the skin, resulting in bright echo lines. Additionally, due to uneven gel distribution, the signal intensity may decrease rapidly or irregular patterns may be observed in some areas of the image.

[0526] The control unit (250) can precisely detect areas where gel is not applied by utilizing a machine learning or deep learning-based image segmentation algorithm. This algorithm is based on a pattern recognition model learned from a large amount of clinical image data, and can perform stably under various environmental conditions and object characteristics.

[0527] In addition, the control unit (250) can quantitatively analyze the location, size, and shape of the gel-unapplied area. This information can be used as important reference data for performing precise gel replenishment work in subsequent steps.

[0528] In step 7300, the control unit (250) can determine whether gel-uncoated areas have actually been identified based on the analysis results performed in step 7200. This determination can be made based on preset thresholds and quality standards.

[0529] The control unit (250) can comprehensively evaluate whether the size of the identified gel non-coating area is greater than a critical area, whether the degree of acoustic coupling failure exceeds a critical level, or whether it is at a level that can have a substantial effect on the diagnostic quality. Minor gel unevenness or non-coating at a level that does not affect the diagnosis may be allowed to continue the diagnosis without replacing the equipment.

[0530] If a gel-uncoated area is identified at a significant level (e.g., 7300), the control unit (250) determines that the equipment replacement condition is satisfied and can proceed to the next step. Conversely, if the gel condition is good (e.g., 7300), the current diagnostic operation can be continued.

[0531] If it is determined that the equipment replacement condition is satisfied based on the identification of the gel-uncoated area in step 7300, the control unit (250) in step 7400 can perform equipment replacement from the currently used ultrasonic probe (20) to the gel application device (400).

[0532] In this process, the control unit (250) can precisely identify the location of the gel-uncoated area based on the position of the ultrasonic probe (20) based on the position and attitude control information of the robot arm (205). The actual spatial location of the gel-uncoated area can be calculated through a transformation between the ultrasonic image coordinate system and the robot physical coordinate system.

[0533] The control unit (250) can control the rotation of the support device (300) and the movement of the robot arm so that the gel application device (400) forms the same relative positional relationship and relative angle relationship that the ultrasonic probe (20) had formed with respect to the target object. At this time, the precise positional relationship maintenance mechanism described above is applied so that the gel application device (400) can be placed in an accurate position.

[0534] In step 7500, the control unit (250) can perform a precise gel application operation on the identified gel-unapplied areas using the gel application device (400) for which equipment replacement has been completed. In this process, instead of full gel application over the entire area, selective supplementary application on specific unapplied areas may be performed.

[0535] The control unit (250) can plan an optimized application pattern by utilizing the location, size, and shape information of the gel-unapplied area analyzed in step 7200. Gradual application can be performed so that a natural gel layer connection is achieved at the boundary of the unapplied area and the connection area where the existing gel is applied.

[0536] In step 7600, the control unit (250) can perform reverse equipment replacement from the gel application device (400) to the ultrasonic probe (20) after the gel application operation is completed. During this process, precise positional relationship restoration is also performed so that the continuity of diagnosis can be ensured.

[0537] The control unit (250) can control the rotation of the support device (300) and the movement of the robot arm so that the ultrasonic probe (20) forms the same relative positional relationship and relative angle relationship that the gel application device (400) had formed with respect to the target object. At this time, fine adjustments can be made considering changes in surface height or increases in gel layer thickness due to gel replenishment.

[0538] After the equipment replacement is complete, the control unit (250) can reactivate the ultrasonic probe (20) and restore the previous diagnostic settings to resume continuous diagnosis. At this time, the focal point and transmission angle maintenance mechanism described in FIG. 25 above is applied to ensure consistent diagnostic quality.

[0539] That is, the focal point and transmission angle maintenance mechanism described in FIG. 25 can be applied not only when the equipment is replaced from the first ultrasonic probe to the second ultrasonic probe, but also when the equipment is replaced from the ultrasonic probe (20) to the gel application device and then replaced back to the ultrasonic probe (20).

[0540] When ultrasound diagnosis resumes after gel replenishment, the control unit (250) can perform steps 7100, 7200, and 7300 again.

[0541] According to the present invention, by detecting and automatically resolving acoustic coupling problems that may occur during the diagnostic process in real time, consistent and high-quality ultrasound images can be ensured and diagnostic accuracy can be significantly improved.

[0542] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium that stores instructions executable by a computer. The instructions may be stored in the form of program code and, when executed by a processor, may generate a program module to perform the operation of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0543] Computer-readable recording media include all types of recording media that store instructions that can be decoded by a computer. Examples include ROM (read-only memory), RAM (random access memory), magnetic tape, magnetic disk, flash memory, optical data storage devices, etc.

[0544] Additionally, computer-readable recording media may be provided in the form of non-transitory storage media. Here, 'non-transitory storage media' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, 'non-transitory storage media' may include a buffer in which data is stored temporarily.

[0545] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable recording medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable recording medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0546] As described above, the disclosed embodiments have been explained with reference to the attached drawings. Those skilled in the art will understand that the present invention may be practiced in forms different from the disclosed embodiments without changing the technical spirit or essential features of the invention. The disclosed embodiments are illustrative and should not be interpreted restrictively.

Claims

1. In a robotic ultrasound diagnostic system, A first robotic arm supporting an ultrasonic probe; A second robot arm supporting an auxiliary device; A camera for acquiring an image of an object; and Identifying the target diagnostic area of ​​the object based on the image of the object, and Control the first robot arm to scan the identified target diagnostic site with the ultrasound probe, and Ultrasound image data acquired by scanning the above-mentioned ultrasound probe is processed to identify regions of interest and anatomical structures, and Based on the contact coordinates of the point where the ultrasonic probe supported by the first robot arm contacts the target, the location information of the identified region of interest, and the location information of the anatomical structure, the target coordinates of the target point where the auxiliary device must contact the target and the target angle between the auxiliary device and the target are calculated. A robot ultrasound diagnostic system comprising: a control unit that controls the second robot arm so that the auxiliary device contacts the object at the calculated target angle at the calculated target coordinates.

2. In Paragraph 1, The above auxiliary device is, A robotic ultrasound diagnostic system comprising at least one of an external vibrator, a cauterization needle, a biopsy needle, or a contrast agent injection needle.

3. In Paragraph 2, The above auxiliary device is the above external vibration device, and In response to the external vibration device coming into contact with the object while forming the calculated target angle at the calculated target coordinates, the control unit, A robotic ultrasonic diagnostic system that controls the external vibration device to irradiate a push pulse toward the object or to generate vibration of the object.

4. In Paragraph 3, The above anatomical structure includes bone structure and vascular structure, and The above control unit is, A region further than a preset first threshold distance from the above bone structure is determined as a first candidate region, and A region further than a preset second threshold distance from the above-mentioned blood vessel structure is determined as a second candidate region, and An area within a preset third threshold distance from the above contact coordinates is determined as a third candidate area, and A robotic ultrasound diagnostic system that calculates the target coordinates within the intersection of the first candidate region, the second candidate region, and the third candidate region.

5. In Paragraph 2, The above auxiliary device is any one of the cauterization needle, the biopsy needle, or the contrast agent injection needle, and In response to the above needle coming into contact with the object while forming the above-calculated target angle at the above-calculated target coordinates, the control unit, A robotic ultrasound diagnostic system that controls the second robotic arm so that the needle reaches a target object within the region of interest.

6. In Paragraph 5, The above anatomical structure includes bone structure and vascular structure, and The above control unit is, A robotic ultrasound diagnostic system that calculates the target coordinates and the target angle so that the needles avoid the bone structure and the blood vessel structure to reach the target object within the region of interest.

7. In Paragraph 1, The above control unit is, The first robot arm is controlled to acquire ultrasonic image frames according to preset time intervals and spatial intervals, and The coordinate information of the contact coordinates corresponding to each of the above-acquired ultrasound image frames is matched and stored, and A robotic ultrasound diagnostic system that generates a three-dimensional ultrasound image based on the stored ultrasound image frames and contact coordinates corresponding to each.

8. In Paragraph 1, The above control unit is, Receive other modality medical images, process the received other modality medical images to identify target landmarks, and In response to the identification of the target landmark in the ultrasound image data during the process of controlling the first robot arm to scan the identified target diagnostic site with the ultrasound probe, the contact coordinates at the time when the ultrasound image data was acquired are stored. A robotic ultrasound diagnostic system that controls the first robot arm based on the stored contact coordinates to align the ultrasound image data with the other modality medical image.

9. In Paragraph 1, The above control unit is, A robotic ultrasound diagnostic system that calculates the target coordinates and the target angle using an artificial intelligence model trained to calculate the target coordinates and the target angle using the contact coordinates and the ultrasound image data as input data.

10. In Paragraph 1, The above-mentioned object can be positioned on an electric bed; further comprising, The above control unit is, A robotic ultrasonic diagnostic system that moves the motorized bed to a predefined position in response to the detection of a predefined danger condition.

11. In Paragraph 10, The above ultrasonic probe further includes a pressure sensor that detects the pressure pressing the object; The above-mentioned predefined risk conditions are, A robotic ultrasonic diagnostic system comprising a pressure detected by the above pressure sensor that is greater than a critical pressure.

12. In Paragraph 10, The above control unit is, By inputting an image of the above-mentioned object into a risk analysis artificial intelligence model, the risk of the above-mentioned object is detected, and The above-mentioned predefined risk conditions are, A robotic ultrasound diagnostic system comprising the detection of a risk to the object by the above-mentioned risk analysis artificial intelligence model.

13. In Paragraph 10, Includes a microphone, The above control unit is, By inputting the acoustic signal received through the above microphone into a risk analysis artificial intelligence model, the risk to the above object is detected, and The above-mentioned predefined risk conditions are, A robotic ultrasound diagnostic system comprising the detection of a risk to the object by the above-mentioned risk analysis artificial intelligence model.

14. In Paragraph 10, A robotic ultrasound diagnostic system in which the operating range of the first robot arm and the second robot arm is designed not to interfere with the predefined position.

15. In Paragraph 10, In response to the detection of the predefined danger condition while the above auxiliary device is in contact with the object, the control unit, A robotic ultrasound diagnostic system that moves the motorized bed in a direction corresponding to the calculated target angle.

16. A control method for a robotic ultrasound diagnostic system comprising a first robotic arm supporting an ultrasound probe and a second robotic arm supporting an auxiliary device, wherein Acquire an image of the object; Identifying the target diagnostic site of the object based on the image of the object; Control the first robot arm to scan the identified target diagnostic site with the ultrasound probe; Processing ultrasound image data acquired by scanning the above ultrasound probe to identify regions of interest and anatomical structures; Based on the contact coordinates of the point where the ultrasonic probe supported by the first robot arm contacts the target, the location information of the identified region of interest, and the location information of the anatomical structure, the auxiliary device calculates the target coordinates of the target point where it must contact the target and the target angle between the auxiliary device and the target; A control method for a robot ultrasound diagnostic system comprising: controlling the second robot arm so that the auxiliary device contacts the object while forming the calculated target angle at the calculated target coordinates.

17. In Paragraph 16, The above auxiliary device is the above external vibration device, and The control method of the above-described robotic ultrasonic diagnostic system is, A control method for a robot ultrasonic diagnostic system further comprising: controlling the external vibration device to irradiate a push pulse toward the target or generate vibration of the target in response to the external vibration device making contact with the target while forming the calculated target angle at the calculated target coordinates.

18. In Paragraph 17, The above anatomical structure includes bone structure and vascular structure, and Calculating the above target coordinates is, Determining a region further than a preset first threshold distance from the above bone structure as a first candidate region; Determining a region further than a preset second threshold distance from the above-mentioned blood vessel structure as a second candidate region; Determining an area within a preset third threshold distance from the above contact coordinates as a third candidate area; A control method for a robotic ultrasound diagnostic system comprising: calculating the target coordinates within the intersection of the first candidate region, the second candidate region, and the third candidate region.

19. In Paragraph 16, The above auxiliary device is a needle among a cauterization needle, a biopsy needle, or a contrast agent injection needle, and The control method of the above-described robotic ultrasonic diagnostic system is, A control method for a robotic ultrasound diagnostic system further comprising: controlling the second robotic arm so that the needle reaches a target object within the region of interest in response to the needle contacting the target object while forming the calculated target angle at the calculated target coordinates.

20. In Paragraph 19, The above anatomical structure includes bone structure and vascular structure, and Calculating the above target coordinates is, A control method for a robotic ultrasound diagnostic system comprising: calculating the target coordinates and the target angle such that the needles avoid the bone structure and the blood vessel structure to reach the target object within the region of interest.

Images