X-ray imaging apparatus including camera and method of operating same

Abstract

An X-ray imaging apparatus including a camera and an operating method for the X-ray imaging apparatus are provided. According to one embodiment of the present disclosure, an X-ray imaging apparatus acquires an object image by capturing an object using a camera, and analyzes the object image using an artificial intelligence model, and thereby can detect a motion of the object, and outputs a notification signal that notifies a user of a result of the detection of the motion of the object. According to one embodiment of the present disclosure, an X-ray imaging apparatus may acquire an object image by photographing an object using a camera, acquire a plurality of divided imaging regions for stitching imaging before X-ray imaging by using an artificial intelligence model, and displaying a plurality of guide lines indicating top, bottom, and left and right boundaries of the plurality of divided imaging regions.

Description

Technical Field

[0001] The present disclosure relates to an X-ray imaging device including a camera and a method of operating the X-ray imaging device. More particularly, the present disclosure relates to an X-ray imaging device for detecting the movement of an object by using an image obtained by photographing the object with a camera. The present disclosure relates to an X-ray imaging device for performing stitching X-raying by using an image of an object captured or photographed by a camera. Background Art

[0002] In recent years, X-ray imaging devices have been allocated and used, which are equipped with cameras, and automatic setting, movement, patient positioning, and status checking of the X-ray imaging devices are performed by using image data of an object (e.g., a patient) obtained by the cameras. Specifically, an X-ray imaging device including a camera can identify the position of a patient, pose information, an X-ray detection effective area, the position of an automatic exposure control (AEC) chamber, etc. from an image obtained by the camera. Compared with a conventional X-ray imaging device without a camera, the X-ray imaging device including a camera has a technical effect of reducing the operation time of a user.

[0003] By using an image of an object (e.g., a patient), an X-ray imaging device including a camera can detect the movement of the patient. A conventional X-ray imaging device detects the movement of a patient by attaching a reference element to a certain body part of the patient and analyzing the displacement of the reference element from an image obtained by photographing the patient to whom the reference element is attached. When no reference element is attached to a certain part of the patient, the conventional technique does not work, and the conventional technique may only detect the movement of the patient during the process of obtaining consecutive X-ray images. The limitation of the conventional technique is that the conventional technique cannot prevent an abnormal image from being obtained by detecting the movement of an object before taking an X-ray image.

[0004] Recently, a technique for achieving more accurate and rapid patient identification by introducing artificial intelligence (AI) technology into an X-ray imaging device is available. An AI system is a computer system that embodies human-level intelligence, which allows a machine to learn and make decisions by itself, and the more it is used, the better the recognition rate. AI technology includes: machine learning technology using an algorithm for self-classifying / self-learning features of input data, or elemental technology using a deep learning algorithm to simulate functions such as perception and determination of a human brain. Summary of the Invention

[0005] Technical Solution

[0006] The present disclosure provides an X-ray imaging device for detecting the movement of an object. According to an embodiment of the present disclosure, the X-ray imaging device may include: an X-ray irradiator configured to generate X-rays and irradiate the X-rays onto the object; an X-ray detector configured to detect the X-rays irradiated by the X-ray irradiator and transmitted through the object; a camera configured to obtain an object image by photographing the object positioned in front of the X-ray detector; a display; and at least one processor. The at least one processor may be configured to detect the movement of the object based on the object image by analyzing the object image via using an artificial intelligence (AI) model. The at least one processor may be configured to output a notification signal on the display to notify a user of a result of the detection of the movement of the object.

[0007] The present disclosure provides a method of operating an X-ray imaging device. According to an embodiment of the present disclosure, the method of operating an X-ray imaging device may include: obtaining image data of the object by photographing the object with a camera. According to an embodiment of the present disclosure, the method of operating an X-ray imaging device may include: detecting the movement of the object based on the image data by analyzing the image data via using an AI model. According to an embodiment of the present disclosure, the method of operating an X-ray imaging device may include: outputting a notification signal to notify a user of a result of the detection of the movement of the object.

[0008] The present disclosure provides an X-ray imaging device for performing stitching X-raying. According to an embodiment of the present disclosure, the X-ray imaging device may include: an X-ray detector configured to detect the X-rays irradiated by the X-ray irradiator and transmitted through the object; a camera configured to obtain an object image by photographing the object positioned in front of the X-ray detector; a display; and at least one processor. The at least one processor may be configured to: input the object image into a trained AI model; and obtain a plurality of divided imaging regions for performing stitching X-raying on the object by performing inference using the AI model. The at least one processor may be configured to: display a plurality of guide lines representing the top, bottom, left boundary, and right boundary of the plurality of divided imaging regions on the display. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present disclosure can be easily understood by a combination of the following detailed description and the accompanying drawings, and reference numerals refer to structural elements.

[0010] Figure 1 is an external view illustrating the configuration of an X-ray imaging device according to an embodiment of the present disclosure.

[0011] Figure 2 is a perspective view of an X-ray detector according to an embodiment of the present disclosure.

[0012] Figure 3 Illustrated is an X-ray imaging device including a movable X-ray detector according to an embodiment of the present disclosure.

[0013] Figure 4 is a conceptual diagram for describing an operation of an X-ray imaging device according to an embodiment of the present disclosure for detecting movement of an object from an image acquired by a camera.

[0014] Figure 5 is a block diagram illustrating components of an X-ray imaging device according to an embodiment of the present disclosure.

[0015] Figure 6 is a flowchart illustrating a method for an X-ray imaging device according to an embodiment of the present disclosure to detect movement of an object from an image acquired by a camera.

[0016] Figure 7 is a schematic diagram according to an embodiment of the present disclosure for describing an operation of an X-ray imaging device for detecting movement of an object by comparing a reference image with subsequent image frames.

[0017] Figure 8 is a flowchart illustrating a method for an X-ray imaging device according to an embodiment of the present disclosure to detect movement of an object by using a machine learning algorithm.

[0018] Figure 9 is a flowchart illustrating a method for an X-ray imaging device according to an embodiment of the present disclosure to detect movement of an object by using a pre-trained deep neural network model.

[0019] Figure 10 is a conceptual diagram according to an embodiment of the present disclosure for describing an operation of an X-ray imaging device for detecting movement of an object by using a pre-trained deep neural network model.

[0020] Figure 11 is a schematic diagram illustrating an operation of an X-ray imaging device according to an embodiment of the present disclosure for detecting positioning of an object by using a depth measurement device.

[0021] Figure 12 is a block diagram illustrating components of an X-ray imaging device and a workstation according to an embodiment of the present disclosure.

[0022] Figure 13 is a conceptual diagram according to an embodiment of the present disclosure for describing an operation of an X-ray imaging device to display a divided imaging area for X-ray imaging on an image acquired by a camera.

[0023] Figure 14 is a block diagram illustrating components of an X-ray imaging device according to an embodiment of the present disclosure.

[0024] Figure 15 is a flowchart illustrating a method according to an embodiment of the present disclosure, by which an X-ray imaging device obtains a divided imaging region for X-ray imaging on an image acquired by a camera and displays a graphical user interface (UI) representing the divided imaging region.

[0025] Figure 16 is a schematic diagram illustrating an operation of an X-ray imaging device according to an embodiment of the present disclosure for determining a divided imaging region according to an imaging protocol and displaying a graphical UI representing the determined divided imaging region.

[0026] Figure 17a is a schematic diagram illustrating an operation of an X-ray imaging device according to an embodiment of the present disclosure for changing at least one of a position, a size, and a shape of a divided imaging region based on a user input.

[0027] Figure 17b is a schematic diagram illustrating an operation of an X-ray imaging device according to an embodiment of the present disclosure for changing at least one of a position, a size, and a shape of a divided imaging region based on a user input.

[0028] Figure 18 is a schematic diagram illustrating an operation of an X-ray imaging device according to an embodiment of the present disclosure for determining a margin of a divided imaging region based on a user input.

[0029] Figure 19 is a schematic diagram illustrating an operation of an X-ray imaging device according to an embodiment of the present disclosure for detecting a positioning of an object by using a depth measurement device.

[0030] Figure 20 is a block diagram illustrating components of an X-ray imaging device and a workstation according to an embodiment of the present disclosure. Detailed Description

[0031] In consideration of the principles of the present disclosure, terms are selected from commonly used general terms that are currently widely used. However, this may depend on the intention of those of ordinary skill in the art, judicial cases, the emergence of new technologies, etc. Some terms used herein are selected based on the judgment of the applicant. In such cases, these terms will be explained in detail below in conjunction with embodiments. Therefore, the terms should be defined based on their meanings and the description throughout the present disclosure.

[0032] As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context clearly dictates otherwise. All terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0033] The term "comprising (or including)" or "having (or carrying)" is inclusive or open-ended and does not exclude additional unrecited elements or method steps. As used herein, terms such as "unit", "module", "block", etc. each represent a unit for handling at least one function or operation and can be implemented in hardware, software, or a combination thereof.

[0034] In this disclosure, as used herein, the expression "configured to..." can be used interchangeably with "suitable for...", "capable of...", "designed to...", "adapted to...", "made for...", or "able to..." depending on the given situation. The expression "configured to..." does not necessarily mean "specially designed in hardware to...". For example, in some cases, the expression "a system configured to do something" can refer to "an entity capable of collaborating with another device or part to do something". For example, "a processor configured to perform functions A, B, and C" can refer to a dedicated processor, such as an embedded processor for performing functions A, B, and C, or a general-purpose processor, such as a central processing unit (CPU) or an application processor that can execute functions A, B, and C by running one or more software programs stored in a memory.

[0035] When the terms "connected" or "coupled" are used, a component can be directly connected or coupled to another component. However, unless otherwise limited, it should also be understood that the component can be indirectly connected or coupled to another component via another new component.

[0036] In this disclosure, the term "object" refers to a target to be imaged, including a person, an animal, or a part thereof. For example, an object can include a patient, a part of a patient's body (e.g., an organ), or a phantom.

[0037] In this disclosure, "X-ray" is an electromagnetic wave with a wavelength ranging from 0.01 angstroms (Å) to 100 angstroms (Å), which has the property of penetrating an object and is thus commonly and widely used in medical devices for obtaining internal images of a living body or non-destructive testing devices in general industry.

[0038] In the present disclosure, an X-ray imaging device is a medical imaging device for obtaining an X-ray image of the internal structure of an object (e.g., a patient's body) by causing X-rays to transmit through the object. Compared with other medical imaging devices including MRI devices, CT devices, etc., the X-ray device is easy to use and can obtain a medical image of an object in a short time. Therefore, the X-ray device is widely used for simple chest imaging, simple abdominal imaging, simple bone imaging, simple sinus imaging, simple neck soft tissue imaging, and mammography.

[0039] In the present disclosure, the term "image" or "object image" may refer to data including discrete image elements (e.g., pixels of a two-dimensional (2D) image). In the present disclosure, the term "image" or "object image" refers to an image obtained by a camera having a general image sensor (e.g., CMOS or CCD). In the present disclosure, the term "image" or "object image" refers to an image different from an X-ray image obtained by image processing of detecting X-rays transmitted through an object by an X-ray detector and converting the X-rays into an electrical signal.

[0040] Functions related to artificial intelligence (AI) in the present disclosure are operated by a processor and a memory. The processor may be provided in plural. One or more processors may include: general-purpose processors such as a central processing unit (CPU), an application processor (AP), a digital signal processor (DSP), etc.; a graphics processing unit (GPU); a vision processing unit (VPU), etc.; or a dedicated artificial intelligence (AI) processor such as a neural processing unit (NPU). One or more processors may control the processing of input data according to a pre-defined operation rule or AI model stored in the memory. When one or more processors are dedicated AI processors, the dedicated AI processors may be designed in a hardware structure dedicated to processing a specific AI model.

[0041] A pre-defined operation rule or AI model can be made through learning. Specifically, an AI model made through learning refers to a pre-defined operation rule or an AI model established to perform desired features (or objects) made when a basic AI model is trained by a learning algorithm using a large amount of training data. Such learning may be performed by the device itself that executes AI according to the present disclosure, or by a separate server and / or system. Examples of learning algorithms may include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited thereto.

[0042] In the present disclosure, an AI model may be composed of multiple neural network layers. Each of the multiple neural network layers may have multiple weight values and perform neural network operations or computations through operations or computations between the operation or computation results of the previous layer and the multiple weight values. The multiple weight values owned by the multiple neural network layers may be optimized by learning the results of the AI model. For example, the multiple weight values may be updated to reduce or minimize the loss value or cost value obtained by the AI model during the training process. The artificial neural network model may include a deep neural network (DNN), such as a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q network, but is not limited thereto.

[0043] Embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings so that the embodiments of the present disclosure can be easily practiced by those of ordinary skill in the art. However, as will be discussed herein, the embodiments of the present disclosure can be implemented in many different forms, and the embodiments of the present disclosure are not limited thereto.

[0044] Embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0045] Figure 1 is an external view illustrating the configuration of an X-ray system 1000 according to an embodiment. In Figure 1 an in-room X-ray imaging device (Room DR) will be described as an example.

[0046] Referring to Figure 1 , the X-ray system 1000 may include an X-ray imaging device 100 and a workstation 200. The X-ray imaging device 100 may include: a camera 110 configured to acquire an object image by photographing an object 10; an X-ray irradiator 120 configured to generate X-rays and irradiate the X-rays onto the object 10; an X-ray detector 130 configured to detect the X-rays that have passed through the object; and a user input interface 160. In Figure 1 only the necessary components for describing the operation of the X-ray imaging device 100 are shown, and the components of the X-ray imaging device 100 of the present disclosure are not limited to Figure 1Those components illustrated therein. The workstation 200 can perform data communication with the X-ray imaging device 100 and provide information to the user in response to a command received from the user. In addition, the X-ray system 1000 can also include a controller 220 and a communication interface 210. The controller 220 is used to control the X-ray system 1000 according to commands input through the workstation, and the communication interface 210 is used to communicate with external devices. Some or all components of the communication interface 210 and the controller 220 can be included in the workstation 200 or separately provided from the workstation 200.

[0047] The X-ray irradiator 120 can be equipped with an X-ray source and a collimator. The X-ray source is used to generate X-rays, and the collimator is used to control the irradiation area of the X-rays generated from the X-ray source.

[0048] The guide rail 30 can be installed on the roof of the examination room where the X-ray system 1000 is placed; by connecting the X-ray irradiator 120 to a movable carriage 40 that moves along the guide rail 30, the X-ray irradiator 120 can be moved to a position corresponding to the object 10; the movable carriage 40 and the X-ray irradiator 120 can be connected through a collapsible column frame 50 to adjust the height of the X-ray irradiator 120.

[0049] An input interface 240 for receiving commands from the user and an output interface 250 for displaying information can be arranged on the workstation 200.

[0050] The input interface 240 can receive commands for controlling imaging protocols, imaging conditions, imaging timing, positioning control of the X-ray irradiator 120, etc. In an embodiment of the present disclosure, the input interface 240 can include a keyboard, a mouse, a touch screen, a voice recognizer, etc.

[0051] The output interface 250 can display: a screen for guiding user input, an X-ray image, a screen indicating the status of the X-ray system 1000, etc. In an embodiment of the present disclosure, the output interface 250 can include a display.

[0052] The controller 220 can control the imaging timing, imaging conditions, etc. of the X-ray irradiator 120 according to commands input from the user, and generate an X-ray image by using the image data received from the X-ray detector 130. The controller 220 can also control the position or posture of the mounting portions 14 or 24 where the X-ray irradiator 120 or the X-ray detector 130 is installed according to the imaging protocol and the position of the object 10.

[0053] The controller 220 may include a memory and a processor. The memory is used to store programs for performing the foregoing and following operations, and the processor is used to run the programs. The controller 220 may include a single processor or multiple processors. In the latter case, the multiple processors may be integrated in a single chip or may be physically separated.

[0054] The X-ray system 1000 may be connected to external devices (e.g., an external server 2000, a medical device 3000, and a portable terminal 4000 (e.g., a smart phone, a tablet PC, a wearable device, etc.)) via a communication interface 210 to send or receive data.

[0055] The communication interface 210 may include one or more components for implementing communication with external devices, and may include at least one of, for example, a short-range communication module, a wired communication module, and a wireless communication module.

[0056] The communication interface 210 may receive a control signal from an external device, and may also send the received control signal to the controller 220 for the controller 220 to control the X-ray system 1000 according to the received control signal.

[0057] The processor 220 may also control an external device based on the control signal of the controller 220 by transmitting a control signal to the external device via the communication interface 210. For example, the external device may process data of the external device according to the control signal of the controller 220 received via the communication interface 210.

[0058] The communication interface 210 may also include an internal communication module for implementing communication between components of the X-ray system 1000. A program for controlling the X-ray system 1000 may be installed in an external device, and the program may include instructions for performing some or all operations of the controller 220.

[0059] The program may be pre-installed in the portable terminal 4000, or a user of the portable terminal 4000 may install the program by downloading the program from a server providing the application. The recording medium storing the program may be included in the server providing the application.

[0060] Meanwhile, the X-ray detector 130 can be implemented as a fixed-type X-ray detector 130-1 fixed to the support 20 or the table 12, or can be detachably equipped in the mounting portions 14 or 24; alternatively, the X-ray detector 130 can be implemented as a movable X-ray detector 130-2 or a portable X-ray detector available anywhere. The movable X-ray detector 130-2 or the portable X-ray detector can be implemented as a wired type or a wireless type according to the data transmission method and the power supply method.

[0061] The X-ray detector 130 can be included as an element of the X-ray system 1000, or can not be included as an element of the X-ray system 1000. In the latter case, the X-ray detector 130 can be registered by the user in the X-ray system 1000. In addition, in both cases, the X-ray detector 130 can be connected to the controller 220 through the communication interface 210 to receive control signals or transmit image data.

[0062] The user input interface 160 can be arranged on one side of the X-ray irradiator 120 to provide information to the user and receive commands from the user. The user input interface 160 can be a sub-user interface that performs some or all of the functions performed by the input interface 240 and the output interface 250 of the workstation 200.

[0063] In the case where all or some of the components in the communication interface 210 and the controller 220 are separately arranged from the workstation 200, these components can be included in the user input interface 160 arranged in the X-ray radiator 120.

[0064] In Figure 1 The X-ray system 1000 illustrated in is an in-room X-ray imaging device connected to the ceiling of the examination room, but the X-ray system 1000 can include various structured X-ray devices within the scope obvious to those of ordinary skill in the art, such as C-arm type X-ray devices, movable X-ray devices, etc.

[0065] Figure 2 is an external view of the X-ray detector 130.

[0066] Reference Figure 2 , the X-ray detector 130 can be implemented as a movable X-ray detector. In this case, the X-ray detector 130 can include a battery for power supply and operate in a wireless manner; or as Figure 2 illustrated, the X-ray detector 130 can have a charging port 132 connected to a separate power source via a cable C and operate.

[0067] Within a housing 304 that defines the exterior of the X-ray detector 130, there are a detection element that detects X-rays and converts the X-rays into image data, a memory that temporarily or non-temporarily stores the image data, a communication module that receives control signals from the X-ray system 1000 or transmits the image data to the X-ray system 1000, and a battery. Additionally, the memory can store image correction information of the detector and unique identification information of the X-ray detector 130, and transmits the stored identification information while communicating with the X-ray system 1000.

[0068] Figure 3 FIG. 4 illustrates an X-ray imaging apparatus 100 including a movable X-ray detector 130 according to an embodiment of the present disclosure.

[0069] Reference Figure 3 , the X-ray imaging apparatus 100 may include a movable X-ray detector 130. The movable X-ray detector 130 is a movable or portable type of X-ray detector that can perform X-ray imaging without being restricted by the imaging position. Figure 3 The X-ray imaging apparatus 100 illustrated in Figure 1 may be an embodiment of the X-ray imaging apparatus 100 illustrated in Figure 3 . Among the components included in the X-ray imaging apparatus 100 illustrated in Figure 1 , the same components as those in

[0070] In Figure 3 are denoted by the same reference numerals, and redundant descriptions will not be repeated.

[0071] The main unit 102 may further include an operation unit for providing a user interface to operate the X-ray imaging apparatus 100. Although the operation unit is illustrated as being included in the main unit 102 in Figure 3 , it is not limited thereto. For example, as Figure 1 shows, the input interface 240 and the output interface 250 of the X-ray system 1000 may be arranged on one side of the workstation 200 (see Figure 1 ).

[0072] The X-ray irradiator 120 may include an X-ray source 122 for generating X-rays and a collimator 124 for controlling the irradiation area of the X-rays generated and irradiated by the X-ray source 122 by guiding the path of the X-rays. The main unit 102 may include a high-voltage generator 126 for generating a high voltage to be applied to the X-ray source 122.

[0073] will be combined with Figure 5 The specific functions and / or operations of the X-ray detector 130, the processor 140, the user input interface 160, and the display 172 will be described in detail.

[0074] It is also feasible that the X-ray system 1000 is implemented not only as the above-described ceiling type but also as a movable type. Figure 3 The X-ray detector 130 is illustrated as a table type placed on the table 106, but it is obvious that the X-ray detector 130 can also be implemented as a stand type as a movable type or a portable type.

[0075] Figure 4 is a conceptual diagram for describing the operation of the X-ray imaging device 100 according to an embodiment of the present disclosure for detecting the movement of the object 10 from the image acquired by the camera 110.

[0076] In Figure 4 the X-ray imaging device 100 is illustrated as a ceiling type, but is not limited thereto. In an embodiment of the present disclosure, the X-ray imaging device 100 can be implemented as a movable type.

[0077] Reference Figure 4 , the X-ray imaging device 100 may include a camera 110, an X-ray irradiator 120, an X-ray detector 130, a user input interface 160, and a display 172. In Figure 4 only the minimum components for describing the functions and / or operations of the X-ray imaging device 100 are illustrated, and the components included in the X-ray imaging device 100 are not limited to Figure 4 those illustrated in Figure 5 The components of the X-ray imaging device 100 will be described in detail.

[0078] In operation ①, the X-ray imaging device 100 obtains an object image 402 by photographing an object 10 with a camera 110. When the positioning of the patient in front of the X-ray detector 130 is completed, the X-ray imaging device 100 can capture or take a picture of the object 10 through the camera 110. In an embodiment of the present disclosure, the X-ray imaging device 100 can display a button UI 404 for executing a motion detection mode on a display 172 and receive a touch input from a user who touches the button UI 404. In response to receiving the user's touch input, the X-ray imaging device 100 can execute the motion detection mode and obtain the object image 402 by photographing the object 10 through the camera 110. However, it is not limited thereto, and the X-ray imaging device 100 can automatically execute the motion detection mode. In an embodiment of the present disclosure, the X-ray imaging device 100 can automatically execute the motion detection mode after a preset time has elapsed after the positioning of the patient in front of the X-ray detector 130 is completed, and obtain the object image by photographing the object 10 with the camera 110.

[0079] The obtained object image 402 is a two-dimensional (2D) image obtained by the camera 110, which has a conventional image sensor (e.g., CMOS or CCD), and this two-dimensional image is different from the X-ray image obtained by receiving the X-ray transmitted through the object 10 by the X-ray detector 130 and performing image processing on the X-ray. In an embodiment of the present disclosure, the X-ray imaging device 100 can display the object image 402 on the display 172.

[0080] In operation ②, the X-ray imaging device 100 detects the motion of the object by using an AI model 152. The X-ray imaging device 100 can detect the motion of the object by analyzing the obtained object image 402 by using the AI model 152. In an embodiment of the present disclosure, the X-ray imaging device 100 can determine the first image frame obtained by photographing the object 10 after the positioning of the patient is completed as a reference image, and detect the motion of the object by comparing the image frames obtained by subsequent image shootings with the reference image by using the AI model 152. The AI model 152 can include at least one of a machine learning algorithm and a deep neural network model.

[0081] In an embodiment of the present disclosure, the X-ray imaging device 100 can use self-organizing maps of a machine learning model to cluster the pixels of the object 10 and the background in the reference image and subsequent image frames, and apply weights to the pixels representing the object 10, so as to improve the accuracy of motion detection of the object 10 in a manner that reduces the influence of background noise.

[0082] In an embodiment of the present disclosure, the X-ray imaging device 100 can detect the movement of an object by inputting an object image 402 into a trained deep neural network model and performing inference using the deep neural network model. The X-ray imaging device 100 can extract key points of landmark parts of the object 10 from each of a reference image and subsequent image frames by performing inference using the deep neural network model. The deep neural network model can be a model trained by a supervised learning method that applies a plurality of acquired images as input data and applies the position coordinates of the key points of the landmark parts as ground truth. The deep neural network model can be, for example, a convolutional neural network (CNN) model, but is not limited thereto. The X-ray imaging device 100 can calculate the difference between the key points extracted from the reference image and the key points extracted from the subsequent image frames, and detect the movement of the object by comparing the calculated difference with a threshold value.

[0083] In operation ③, the X-ray imaging device 100 outputs a notification signal indicating the detection result of the movement of the object 10. In an embodiment of the present disclosure, the X-ray imaging device 100 can display a graphical user interface (UI) 406 having a preset color on the display 172 to indicate the movement of the object 10. The graphical UI 406 can be an icon having, for example, an orange (or red) color and forming the shape of a moving person. Although not illustrated in Figure 4 , the X-ray imaging device 100 may further include a speaker 174 configured to output an acoustic signal and output at least one of a voice and a notification sound as the acoustic signal to notify the user of information about the movement of the object 10 through the speaker 174.

[0084] A conventional X-ray imaging device detects the movement of an object 10 (e.g., a patient) by attaching a reference element to a certain body part of the object 10 and analyzing the displacement of the reference element from an image obtained by photographing the object 10 to which the reference element is attached. When no reference element is attached to a certain body part of the object, this conventional technique does not work, and this conventional technique may only detect the movement of the object 10 during the process of acquiring continuous X-ray images. The limitation of the conventional technique is that it is impossible to prevent abnormal images from being acquired by detecting the movement of the object 10 before taking the X-ray image.

[0085] The present disclosure aims to provide an X-ray imaging device 100 and an operation method thereof that detect the movement of an object 10 by using a camera 110 to photograph the object 10 to obtain an object image 402 and analyzing the object image 402 by using an AI model 152.

[0086] In Figure 4 In the illustrated embodiment, different from the conventional technology, after the positioning of the patient is completed and before the actual X-ray imaging is performed, when the movement of the object 10 is greater than the normal movement of the object 10 (e.g., breathing), by outputting a graphical UI 406 or an acoustic signal indicating the movement of the object 10 (e.g., the patient), the X-ray imaging device 100 can assist the user in performing effective and accurate X-ray imaging, and thereby improve user convenience. In an embodiment of the present disclosure, the X-ray imaging device 100 can automatically perform patient monitoring, and acquire an X-ray image of the object 10 when the object 10 is maintained in a position as accurate as the intention of the user (e.g., a radiographer), thereby preventing the deterioration of the image quality of the X-ray image due to the movement of the object 10. In addition, in an embodiment of the present disclosure, the X-ray imaging device 100 provides a technical effect of preventing the risk of an increase in radiographic time and additional radiation exposure occurring during re-shooting caused by the movement of the object 10. Figure 1

[0087] Figure 5 FIG. is a block diagram illustrating components of an X-ray imaging device 100 according to an embodiment of the present disclosure.

[0088] Figure 5 The X-ray imaging device 100 illustrated in may be a movable type device including a movable X-ray detector 130. However, it is not limited thereto, and the X-ray imaging device 100 may be implemented as a ceiling type. The ceiling type X-ray imaging device 100 will be described in detail later in conjunction with Figure 12

[0089] Referring to Figure 5 , the X-ray imaging device 100 may include a camera 110, an X-ray irradiator 120, an X-ray detector 130, a processor 140, a memory 150, a user input interface 160, and an output interface 170. The camera 110, the X-ray irradiator 120, the X-ray detector 130, the processor 140, the memory 150, the user input interface 160, and the output interface 170 may be electrically connected and / or physically connected to each other. In Figure 5 , only the necessary components for describing the operation of the X-ray imaging device 100 are shown, and the components included in the X-ray imaging device 100 are not limited to Figure 5 those illustrated in. In an embodiment of the present disclosure, the X-ray imaging device 100 may further include a communication interface 190 for performing data communication with a workstation 200 (see Figure 12 ), a server 2000 (see Figure 1 ), other medical devices 3000 (see Figure 1 ), or an external portable terminal 4000 (see Figure 1 ) (seeFigure 12 ). In an embodiment of the present disclosure, the X-ray imaging device 100 may further include a high-voltage generator 126 for generating a high voltage to be applied to the X-ray source 122 (see Figure 3 ). In another embodiment of the present disclosure, the output interface 170 of the X-ray imaging device 100 may not include a speaker 174.

[0090] The camera 110 is configured to acquire an object image by photographing an object (e.g., a patient) positioned in front of the X-ray detector 130. In an embodiment of the present disclosure, the camera 110 may include a lens module, an image sensor, and an image processing module. The camera 110 may acquire a still image or a video of the object through the image sensor (e.g., CMOS or CCD). The video may include a plurality of image frames acquired in real time by photographing the object through the camera 110. The image processing module may encode the still image having a single image frame or the video data composed of a plurality of image frames acquired through the image sensor, and send it to the processor 140.

[0091] In an embodiment of the present disclosure, the camera 110 may be implemented as a small form factor to be mounted on one side of the user input interface 160 of the X-ray imaging device 100, and may be a lightweight RGB camera that consumes low power. However, it is not limited thereto, and in another embodiment of the present disclosure, the camera 110 may be implemented as any type of camera, such as an RGB-depth camera including a depth estimation function, a stereo fisheye camera, a grayscale camera, or an infrared camera.

[0092] The X-ray irradiator 120 is configured to generate X-rays and irradiate the X-rays onto an object. The X-ray irradiator 120 may include an X-ray source 122 and a collimator 124. The X-ray source 122 generates X-rays by receiving the high voltage generated from the high-voltage generator 126 (see Figure 3 ) and irradiates the X-rays. The collimator 124 adjusts the X-ray irradiation area by guiding the path of the X-rays irradiated from the X-ray source 122.

[0093] The X-ray source 122 may include an X-ray tube, and the X-ray tube may be implemented as a two-electrode vacuum tube having an anode and a cathode. The interior of the X-ray tube is made into a high vacuum state of about 10 mmHg, and thermoelectrons are generated by heating the cathode filament. For the filament, a tungsten filament may be used, and the filament may be heated by applying a voltage of 10 V and a current of about 3 A to 5 A to the wire connected to the filament. When a high voltage of about 10 kVp to 300 kVp is applied between the cathode and the anode, the thermoelectrons are accelerated and collide with the target material at the anode, thereby generating X-rays. The X-rays may be irradiated to the outside through a window, and a barium film may be used as the material of the window. In this case, most of the energy of the electrons colliding with the target material is consumed as heat, and the remaining energy is converted into X-rays.

[0094] The anode is mainly composed of copper, and the target material is disposed on the side opposite to the cathode, and for the target material, a high-resistivity material such as Cr, Fe, Co, Ni, W, Mo, etc. may be used. The target material may be rotated by a rotating magnetic field, and when the target material rotates, the electron impact area increases compared to the case where the target material is fixed, and the heat accumulation rate per unit area may increase by 10 times or more.

[0095] The voltage applied between the cathode and the anode of the X-ray tube is called the tube voltage, which is applied from the high voltage generator 126, and the amplitude may be expressed as peak kVp. When the tube voltage increases, the speed of the thermoelectrons increases, and as a result, the energy (photon energy) of the X-rays generated from the collision with the target material increases. The current flowing in the X-ray tube is called the tube current, and the tube current may be expressed in average mA, and as the tube current increases, the number of thermoelectrons emitted from the filament increases, and as a result, the dose of the X-rays generated from the collision with the target material (the number of X-ray photons) increases. Therefore, the X-ray energy can be controlled by the tube voltage, and the intensity or dose of the X-rays can be controlled by the tube current and the X-ray exposure time.

[0096] The X-ray detector 130 is configured to detect the X-rays irradiated by the X-ray irradiator 120 and transmitted through the object. In an embodiment of the present disclosure, the X-ray detector 130 may be a digital detector implemented with a charge-coupled device (CCE) or a thin film transistor (TFT). Although the X-ray detector 130 is illustrated as a component included in the X-ray imaging device 100 in Figure 5 it may be a separate device that can be attached to the X-ray imaging device 100 and can be detached from the X-ray imaging device 100.

[0097] The processor 140 may execute one or more instructions of a program stored in the memory 150. The processor 140 may include hardware components for performing arithmetic operations, logical operations, input / output operations, and image processing. The processor 140 is illustrated as one element in Figure 5 but is not limited thereto. In an embodiment of the present disclosure, the processor 140 may be configured with one or more elements. The processor 140 may be: a general-purpose processor, such as a central processing unit (CPU), an application processor (AP), a digital signal processor (DSP), etc.; a dedicated graphics processor, such as a graphics processing unit (GPU), a vision processing unit (VPU), etc.; or a dedicated artificial intelligence (AI) processor, such as a neural processing unit (NPU). The processor 140 may control the processing of input data according to predefined operation rules or AI models. When the processor 140 is a dedicated AI processor, the dedicated AI processor may be designed in a hardware structure dedicated to processing using a specific AI model.

[0098] The memory 150 may include, for example, at least one type of storage medium, including flash memory, a hard disk, a multimedia card micro memory, a card-type memory (e.g., SD or XD memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), or an optical disc.

[0099] Instructions related to the function and / or operation of the X-ray imaging device 100 for detecting the movement of an object from an object image acquired by the camera 110 may be stored in the memory 150. In an embodiment of the present disclosure, the memory 150 may store at least one of an algorithm, a data structure, program code, an application program, and instructions that can be read by the processor 140. The instructions, algorithms, data structures, and program code stored in the memory 150 may be implemented in, for example, a programming or scripting language (such as C, C++, Java, assembly language, etc.).

[0100] In the following embodiments, the processor 140 may be implemented by running the instructions or program code stored in the memory 150.

[0101] The processor 140 may acquire the image data of the object image obtained by photographing the object with the camera 110. The processor 140 may acquire the image data of the object by controlling the camera 110 to photograph the object when the positioning of the patient in front of the X-ray detector 130 is completed. In an embodiment of the present disclosure, the processor 140 may control the camera 110 to photograph the object in response to a user input received through the user input interface 160 for executing the motion detection mode. The user input interface 160 may receive a user touch input selecting the button UI 404 (see Figure 4 )(which is displayed on the display 172), and when the touch input is received, the processor 140 may execute the motion detection mode (manual mode). In an embodiment of the present disclosure, the user input for executing the motion detection mode is not limited to the touch input, but may correspond to an input such as pressing a keyboard, a hardware button, a microswitch, etc.

[0102] However, it is not limited thereto, and the processor 140 may automatically execute the motion detection mode to photograph the object. In an embodiment of the present disclosure, the processor 140 may automatically execute the motion detection mode (automatic mode) after a preset time has elapsed after the positioning of the patient in front of the X-ray detector 130 is completed.

[0103] In an embodiment of the present disclosure, the processor 140 may acquire video data composed of a plurality of image frames acquired in real time by the camera 110.

[0104] The processor 140 may detect the motion of the object according to the image data by analyzing the image data using the artificial intelligence (AI) model 152. The AI model 152 may include at least one of a machine learning algorithm and a deep neural network. In an embodiment of the present disclosure, the AI model 152 may be implemented by instructions, program codes, or algorithms stored in the memory 150, but it is not limited thereto. In an embodiment of the present disclosure, the AI model 152 may not be included in the X-ray imaging device 100. In this case, the AI model 232 (see Figure 12 may be included in the workstation 200 (see Figure 12 ).

[0105] When the motion detection mode is executed after the positioning of the patient is completed, the processor 140 may determine the first image frame obtained by photographing the object using the camera 110 as a reference image, and detect the motion of the object by comparing the image frames obtained by subsequent image shootings with the reference image using the AI model 152. In an embodiment of the present disclosure, the processor 140 may use a self-organizing map (which is a machine learning algorithm of the AI model 152) to cluster the pixels of the object and the pixels of the background from each of the reference image and the subsequent image frames, and apply weights to the pixels representing the object, thereby detecting the motion of the object. By using the self-organizing map, the influence of background noise can be reduced, so that the motion detection accuracy of the object can be improved. Specific embodiments in which the processor 140 detects the motion of the object by using the self-organizing map will be described in detail in Figure 8 The specific embodiments in which the processor 140 detects the motion of the object by using the self-organizing map will be described in detail.

[0106] In an embodiment of the present disclosure, the processor 140 may detect the motion of the object by inputting the object image into the trained deep neural network model of the AI model 152 and performing inference using the deep neural network model. The processor 140 may perform inference using the deep neural network model to extract the key points of the feature landmark parts of the object from each of the reference image and the subsequent image frames. The deep neural network model may be a model trained by a supervised learning method that applies multiple acquired images as input data and applies the position coordinates of the key points of the feature landmark parts as ground truth. The deep neural network model may be, for example, a convolutional neural network (CNN) model, but is not limited thereto. The processor 140 may calculate the difference between the key points extracted from the reference image and the key points extracted from the subsequent image frame, and detect the motion of the object by comparing the calculated difference with a threshold. Specific embodiments in which the processor 140 detects the motion of the object by using the deep neural network model will be described in detail in conjunction with Figure 9 and Figure 10 The specific embodiments in which the processor 140 detects the motion of the object by using the deep neural network model will be described in detail.

[0107] The processor 140 may control the output interface 170 to output a notification signal that notifies the user of the detection result of the motion of the object. In an embodiment of the present disclosure, the processor 140 may control the display 172 to display a graphical UI having a preset color that represents the motion of the object. The graphical UI may be an icon having, for example, an orange (or red) color and forming the shape of a moving person. In an embodiment of the present disclosure, the processor 140 may control the speaker 174 to output at least one acoustic signal of voice and notification sound that notifies the user of information about the motion of the object.

[0108] The processor 140 may be configured to adjust the motion detection sensitivity for adjusting the level of motion detection. The motion detection sensitivity indicates the degree to which motion detection and notification signals are provided based on the degree of motion of an object. If motion is detected and a notification signal is output even when the motion of the object is small, the greater the motion detection sensitivity, and if motion is detected only when the object makes a relatively large motion, the smaller the motion detection sensitivity. In an embodiment of the present disclosure, the processor 140 may set the motion detection sensitivity based on at least one of the source-to-image distance (SID), which is the distance between the object and the X-ray irradiator 120, the size and shape of the object, and the imaging protocol. However, it is not limited thereto, and the processor 140 may set the motion detection sensitivity according to a user input. In this case, the user input interface 160 may receive a user input to set or adjust the motion detection sensitivity, and the processor 140 may set the motion detection sensitivity based on the received user input.

[0109] The user input interface 160 is configured to provide an interface for operating the X-ray imaging device 100. The user input interface 160 may be configured as, for example but not limited to, a control panel including hardware elements such as a keyboard, a mouse, a trackball, a jog dial, a microswitch, or a touchpad. In an embodiment of the present disclosure, the user input interface 160 may be configured as a touch screen that receives touch inputs and displays a graphical user interface (GUI).

[0110] The user input interface 160 may receive an input of a command for operating the X-ray imaging device 100 and various information about X-ray imaging from the user. The user input interface 160 may receive a user input such as a command for, for example, setting the motion detection sensitivity, executing a motion detection mode (manual mode), etc.

[0111] The output interface 170 is configured to output a detection result of the motion of the object under the control of the processor 140. The output interface 170 may include a display 172 and a speaker 174.

[0112] The display 172 may display a GUI representing the detection result of the motion of the object. The display 172 may include a hardware device including, for example, at least one of a CRT display, an LCD display, a PDP display, an OLED display, a FED display, an LED display, a VFD display, a DLP display, a flat panel display, a 3D display, and a transparent display, but is not limited thereto. In an embodiment of the present disclosure, the display 172 may be configured as a touch screen including a touch interface. In the case where the display 172 is configured with a touch screen, the display 172 may be a component integrated with the user input interface 160 including a touch panel.

[0113] Figure 6 FIG. Figure 6 is a flowchart illustrating a method for a X-ray imaging device 100 according to an embodiment of the present disclosure to detect the movement of an object from an image acquired by a camera.

[0114] In operation S610, the X-ray imaging device 100 acquires image data of the object by photographing the object using the camera. The X-ray imaging device 100 may acquire image data by photographing an object (e.g., a patient) positioned in front of the X-ray detector 130 (see Figure 4 ). In an embodiment of the present disclosure, after the positioning of the patient is completed, the X-ray imaging device 100 may receive a user input to select a button UI for executing the motion detection mode, and execute the motion detection mode based on the received user input. When the motion detection mode is executed, the X-ray imaging device 100 may acquire image data of the object through the camera 110 to detect the movement of the object.

[0115] In an embodiment of the present disclosure, after a preset time has elapsed after the positioning of the patient in front of the X-ray detector 130 is completed, the X-ray imaging device 100 may automatically execute the motion detection mode. When the motion detection mode is executed, the X-ray imaging device 100 may photograph the object through the camera 110 to acquire image data.

[0116] The X-ray imaging device 100 may acquire a reference image by photographing the object after the positioning of the patient in front of the X-ray detector 130 is completed, and acquire image frames by photographing subsequent images of the object after acquiring the reference image. In an embodiment of the present disclosure, after acquiring the reference image, the X-ray imaging device 100 may acquire a plurality of image frames by photographing the object's image in real time using the camera.

[0117] In operation S620, the X-ray imaging device 100 may detect the movement of the object according to the image data by analyzing the image data using an AI model. The X-ray imaging device 100 may detect the movement of the object by comparing the object identified from the reference image with the object identified from the subsequently captured or photographed image based on the analysis of the AI model.

[0118] In an embodiment of the present disclosure, the X-ray imaging device 100 may identify the object from each of the reference image and the subsequent image frames using a self-organizing map (which is a machine learning algorithm of the AI model), cluster the pixels representing the identified object and the background respectively, and detect the movement of the object by applying weights to the pixels representing the object.

[0119] In an embodiment of the present disclosure, the X-ray imaging device 100 may input image data into a trained deep neural network model in an AI model, and detect the movement of an object by performing inference using the deep neural network model. The X-ray imaging device 100 may extract key points of characteristic landmark parts of the object from each of a reference image and subsequent image frames by performing inference using the deep neural network model. The X-ray imaging device 100 may calculate a difference between the key points extracted from the reference image and the key points extracted from the subsequent image frames, and detect the movement of the object by comparing the calculated difference with a threshold value.

[0120] In operation S630, the X-ray imaging device 100 outputs a notification signal to notify the user of the detection result of the movement of the object. In an embodiment of the present disclosure, the X-ray imaging device 100 may display a graphical UI having a preset color indicating the movement of the object. In an embodiment of the present disclosure, the X-ray imaging device 100 may output at least one acoustic signal among voice and notification sound, the acoustic signal notifying the user of information about the movement of the object.

[0121] Figure 7 is a schematic diagram for describing an operation of the X-ray imaging device 100 for detecting the movement of an object by comparing a reference image i R with subsequent image frames i1, i2, i3,....

[0122] Reference Figure 7 , the positioning of the patient is completed at time t0, where the object is positioned in front of the X-ray detector 130 (see Figure 4 and Figure 5 ). The X-ray imaging device 100 may obtain a plurality of image frames i1, i2, i3,... by photographing the object using a camera after the positioning of the patient is completed. The X-ray imaging device 100 may determine the image frame obtained at the first time t1 after the positioning of the patient as the reference image i R . The X-ray imaging device 100 may store the reference image i R in a storage space in the memory 150 (see Figure 5 ). The X-ray imaging device 100 may obtain a plurality of image frames i1, i2, i3,... by photographing subsequent images of the object after obtaining the reference image i R . For example, the X-ray imaging device 100 may obtain the first image frame i1 at the second time t2, obtain the second image frame i2 at the third time t3, and obtain the third image frame i3 at the fourth time t4.

[0123] The X-ray imaging device 100 may, based on the AI model 152 (see Figure 4 and Figure 5The analysis of ) starts from the reference image i R Identifies the object 700, and detects the movement of the object by comparing the objects 701, 702, and 703 identified from the multiple image frames i1, i2, i3, …… obtained by subsequent image capture. For example, the X-ray imaging device 100 can identify the object 700 from the reference image i by using the AI model 152 R Identifies the object 700, identifies the object 701 from the first image frame i1 captured or photographed subsequently, and detects the movement of the object by comparing the object identified from the reference image i R Identified object and the object identified from the first image frame i1. Similarly, the X-ray imaging device 100 can compare the object 700 identified from the reference image i R Identified object with each of the objects 702 identified from the second image frame i2 and the object 703 identified from the third image frame i3 to detect the movement of the object.

[0124] Figure 8 Is a flowchart illustrating a method for the X-ray imaging device 100 according to an embodiment of the present disclosure to detect the movement of an object by using a machine learning algorithm.

[0125] In Figure 8 The operations S810 to S830 illustrated are Figure 6 Detailed operations of the operation S620 illustrated in. After performing the operation S610 illustrated in Figure 6 The operation S810 of Figure 8 Can be executed. Figure 6 The operation S630 illustrated in can be after Figure 8 The operation S830 of

[0126] In operation S810, the X-ray imaging device 100 obtains weights from the reference image by using a self-organizing map. The processor 140 of the X-ray imaging device 100 (see Figure 5 ) can identify the object from the reference image by using a self-organizing map in a machine learning algorithm, and apply weights to the pixels representing the identified object. In an embodiment of the present disclosure, the processor 140 can store the image data and weights of the reference image in the storage space in the memory 150 (see Figure 5 ). In an embodiment of the present disclosure, the processor 140 may not apply any weights to the pixels representing the background, or may apply low weights to the pixels representing the background.

[0127] In operation S820, by using weights, the X-ray imaging device 100 detects the movement of an object by comparing the object identified from an image frame captured or photographed subsequently with the object identified from a reference image. In an embodiment of the present disclosure, the processor 140 of the X-ray imaging device 100 may identify an object from an image frame obtained by subsequent image capture after obtaining the reference image, and calculate the difference in image pixel values between the identified objects by comparing the identified object with the object identified from the reference image. When the calculated difference exceeds a preset threshold, the processor 140 may identify the movement of the object. In an embodiment of the present disclosure, the processor 140 may periodically identify an object from subsequent image frames at a preset time interval, and detect the movement of the object by comparing the identified object with the object identified from the reference image.

[0128] In operation S830, the X-ray imaging device 100 updates the reference image and weights by using the detection result. The processor 140 of the X-ray imaging device 100 may update the reference image by using information about pixels of the corresponding objects identified from the reference image and each subsequently captured or photographed image frame. In addition, the processor 140 may update the weights by using information about the pixels of the detected object. The processor 140 may update the reference image and weights by repeatedly performing operations S820 and S830 on a plurality of subsequently captured or photographed image frames.

[0129] In the case of identifying an object based on a self-organizing map, a high weight may be applied to the object in the image, while no weight or a relatively low weight is applied to the background. According to the Figure 8 flowchart illustrated in

[0130] In an embodiment of the present disclosure, the X-ray imaging device 100 may detect the movement of an object by comparing a reference image with subsequent image frames by using a known machine learning algorithm. For example, the X-ray imaging device 100 may analyze the reference image and subsequent image frames by using at least one of a support vector machine (SVM), linear regression, logistic regression, Naive Bayes, random forest, decision tree, or k-nearest neighbor algorithm to detect the movement of the object.

[0131] Figure 9FIG. 0 is a flowchart of a method for a X-ray imaging device 100 to detect the movement of an object by using a trained deep neural network according to an embodiment of the present disclosure.

[0132] In Figure 9 the operations S910 to S950 illustrated in Figure 6 are detailed operations of the operation S620 illustrated in Figure 6 After performing the operation S610 illustrated in Figure 9 the operation S910 of Figure 6 can be performed. The operation S630 illustrated in Figure 9 can be after the operation S940 of

[0133] Figure 10 FIG. 19 is a conceptual diagram for describing the operations of the X-ray imaging device 100 for detecting the movement of an object by using a trained deep neural network model according to an embodiment of the present disclosure.

[0134] Now, the functions and / or operations of the X-ray imaging device 100 for detecting the movement of an object will be described with reference to both Figure 9 and Figure 10 In operation S910, the X-ray imaging device 100 performs inference by using a trained deep neural network model to extract a plurality of first key points of the characteristic landmark part of the object from the reference image. Additionally, referring to

[0135] the X-ray imaging device 100 can obtain the reference image i Figure 10 through the camera 110. The processor 140 of the X-ray imaging device 100 can input the reference image i R into the AI model 152, and can extract a plurality of first key points P R of the characteristic landmark part of the object 1001 from the reference image i R by performing inference using the AI model 152, such as P R_1 P R_2 ...... P R_n。In embodiments of the present disclosure, the positions and quantities of the landmark sites of an object can be determined according to an imaging protocol. For example, in the case of a stitching protocol for capturing or photographing an image of the entire body of an object (e.g., a patient), in the body part, the head, shoulders, elbows, hands, waist, knees, and feet can be determined as landmark sites, and the processor 140 can extract the key points of the landmark sites. For example, in the case of an entire spine protocol, in the body part from the ears to below the pelvis, the head, shoulders, elbows, hands, waist, etc. can be determined as landmark sites, and the processor 140 can extract the key points of the landmark sites. In the case of a limb protocol for the head (skull), hand, or foot, parts with unique body characteristics (such as the face, hand, or foot) can be determined as landmark sites, and the processor 140 can extract the key points of the landmark sites. For ease of explanation, Figure 10 FIG. illustrates a plurality of first key points P extracted according to the entire spine protocol R_1 , P R_2 , ……, P R_n . The plurality of first key points P R_1 , P R_2 , ……, P R_n are not limited to Figure 10 the key points illustrated therein, and can be changed according to the imaging protocol and the required accuracy. In embodiments of the present disclosure, the processor 140 can construct the plurality of first key points P R_1 , P R_2 , ……, P R_n into a data set. The processor 140 can store the data set in the memory 150 (see Figure 5 ).

[0136] In embodiments of the present disclosure, the AI model 152 can be a deep neural network model. The deep neural network model can be a model trained by a supervised learning method that applies multiple acquired images as input data and applies the position coordinates of the key points of the landmark sites as the ground truth. To improve the accuracy of key point extraction, the deep neural network model can be trained in a partially modified form. When the input data (e.g., multiple images) and ground truth data (e.g., the position coordinates of the key points) for training are insufficient, data can also be processed by augmentation to increase the data volume or by applying a fine-tuning method that partially modifies the trained model.

[0137] The deep neural network model can be implemented as a convolutional neural network (CNN) model. The deep neural network model can be, for example, a U-Net. However, it is not limited thereto, and the deep neural network model can be implemented as any pose estimation model known to the public.

[0138] Return to referenceFigure 9 , in operation S920, the X-ray imaging device 100 calculates a difference by comparing a plurality of first key points extracted from a reference image with a plurality of second key points extracted from a subsequently acquired image frame. Also refer to Figure 10 , the processor 140 of the X-ray imaging device 100 can acquire a subsequent image frame i1 by photographing an object with the camera 110, and extract a plurality of second key points P1, P2,..., P of the feature landmark part of the object 1002 from the subsequent image frame I1 by performing inference using the AI model 152 n . The processor 140 can extract a plurality of first key points P from the reference image i R R_1 , R_2 R_n with a plurality of second key points P1, P2,..., P extracted from the subsequent image frame i1 n to calculate the difference.

[0139] Return to reference Figure 9 , in operation S930, the X-ray imaging device 100 can compare the calculated difference with a preset threshold α.

[0140] When, as a result of the comparison, the difference exceeds the threshold α, in operation S940, the X-ray imaging device 100 detects the movement of the object. When the difference exceeds the threshold α, the processor 140 of the X-ray imaging device 100 can determine that the object has moved.

[0141] When, as a result of the comparison, the difference is equal to or less than the threshold α, in operation S950, the X-ray imaging device 100 does not detect the movement of the object. When no movement is detected, the X-ray imaging device 100 can perform the following operations: acquire an image frame (e.g., a second image frame) through subsequent image capture, and return to operation S920 to extract a plurality of third key points from the image frame (e.g., the second image frame), and calculate the difference by comparing the extracted plurality of third key points with a plurality of first key points P R_1 , R_2 R_n for comparison.

[0142] Figure 11 is a diagram illustrating operations of the X-ray imaging device 100 according to an embodiment of the present disclosure for detecting the positioning of an object 10 by using a depth measurement device 180.

[0143] Refer to Figure 11 , the X-ray imaging device 100 may include a camera 110, an X-ray irradiator 120, an X-ray detector 130, and a depth measurement device 180. In Figure 11The camera 110, X-ray irradiator 120, and X-ray detector 130 illustrated therein are the same as the components described in connection with Figure 4 and thus redundant descriptions thereof will not be repeated.

[0144] The depth measurement device 180 is configured to measure the distance between the X-ray irradiator 120 and the object 10. In an embodiment of the present disclosure, the depth measurement device 180 may include at least one of a stereo type camera, a time-of-flight (ToF) camera, and a laser distance measurer. The processor 140 of the X-ray imaging device 100 (see Figure 5 ) may detect the positioning of the patient by measuring the distance between the X-ray irradiator 120 and the object 10 using the depth measurement device 180. After detecting the positioning of the patient, the processor 140 may execute a motion detection mode in response to receiving a user input (manual mode) or after a preset time has elapsed (automatic mode).

[0145] Figure 12 is a block diagram illustrating components of an X-ray imaging device 100 and a workstation 200 according to an embodiment of the present disclosure.

[0146] In Figure 12 the X-ray imaging device 100 illustrated therein may be implemented as a ceiling type. Referring to Figure 12 , the X-ray imaging device 100 may include a camera 110, an X-ray irradiator 120, an X-ray detector 130, a processor 140, a user input interface 160, an output interface 170, and a communication interface 190. In Figure 12 the X-ray imaging device 100 illustrated therein is the same as the X-ray imaging device 100 (see Figure 5 ) except that the former does not include a memory 150 (see Figure 5 ) but further includes a communication interface 190, and thus redundant descriptions will be omitted.

[0147] The communication interface 190 may send data to and receive data from the workstation 200 through a wired communication network or a wireless communication network and process the data. The communication interface 190 may perform data communication with the workstation 200 by using at least one of data communication schemes including, for example, wireless local area network (WLAN), Wi-Fi, Bluetooth, zigbee, WFD, infrared data association (IrDA), Bluetooth low energy (BLE), near field communication (NFC), wireless broadband internet (Wibro), worldwide interoperability for microwave access (WiMAX), shared wireless access protocol (SWAP), wireless gigabit alliance (WiGig), and radio frequency (RF) communication.

[0148] In an embodiment of the present disclosure, under the control of the processor 140, the communication interface 190 may transmit an object image obtained by photographing an object with the camera 110 to the workstation 200, and receive a detection result of the object movement from the workstation 200. The X-ray imaging device 100 may display a notification signal (e.g., a graphical UI) indicating the detection result of the received object movement through the display 172, or output the notification signal as an acoustic signal through the speaker 174.

[0149] The workstation 200 may include a communication interface 210, a memory 230, and a processor 220. The communication interface 210 is for communicating with the X-ray imaging device 100. The memory 230 is for storing at least one instruction or program code. The processor 220 is configured to run the instruction or program code stored in the memory 230. The processor 220 may be a hardware device that constitutes the controller 220 illustrated in Figure 1 (see Figure 1 ).

[0150] The AI model 232 may be stored in the memory 230 of the workstation 200. Except for the storage location, the AI model 232 stored in the workstation 200 is the same as the AI model 152 described and illustrated in Figure 4 and Figure 5 (see Figure 4 and Figure 5 ), and thus redundant descriptions will be omitted. The workstation 200 may receive the image data of the object image from the X-ray imaging device 100 through the communication interface 210. In an embodiment of the present disclosure, the image data transmitted to the workstation 200 may include a reference image and subsequent image frames. The processor 220 of the workstation 200 may detect the movement of the object by comparing the reference image with the subsequent image frames using the AI model 232.

[0151] In an embodiment of the present disclosure, the processor 220 may use self-organizing mapping (which is a machine learning algorithm of the AI model 232) to cluster the pixels representing the object and the pixels representing the background from each of the reference image and the subsequent image frames, and detect the movement of the object by applying weights to the pixels representing the object. In an embodiment of the present disclosure, the processor 220 may input the reference image and the subsequent image frames into the trained deep neural network model of the AI model 232, perform inference using the deep neural network model to extract key points of the feature landmark parts of the object from the reference image and the subsequent image frames, and detect the movement of the object by comparing the extracted key points. The specific method by which the processor 220 uses self-organizing mapping or the deep neural network model to detect the movement of the object is the same as the operation method of the processor 140 of the X-ray imaging device 100 described in conjunction with Figures 8 to 10 and thus redundant descriptions will not be repeated.

[0152] The processor 220 of the workstation 200 may control the communication interface 210 to transmit data of the motion detection result to the X-ray imaging device 100.

[0153] Generally, compared with the workstation 200, the storage capacity of the memory 150 of the X-ray imaging device 100 (see Figure 5 ), and the operation processing speed of the processor 140 may be restricted. Therefore, the workstation 200 may execute operations that require storing a large amount of data and a large amount of calculations (e.g., detecting the motion of an object by performing inference using the AI model 232), and then transmit the required data (e.g., data of the motion detection result of the object) to the X-ray imaging device 100 through the communication network. In this way, even without a large-capacity memory and a processor with high-speed computing capabilities, the X-ray imaging device 100 can receive data of the motion detection result of the object from the workstation 200 and output a notification signal indicating the motion detection result of the object, thereby reducing the processing time spent on detecting the motion of the object and improving the accuracy of the motion detection result.

[0154] Figure 13 is a conceptual diagram for describing the operation of the X-ray imaging device 300 to display a divided imaging area for X-ray imaging on an image acquired by a camera according to an embodiment of the present disclosure.

[0155] In Figure 13 , the X-ray imaging device 300 is illustrated as a roof type, but is not limited thereto. In an embodiment of the present disclosure, the X-ray imaging device 300 may also be implemented as a movable type.

[0156] Referring to Figure 13 , the X-ray imaging device 300 may include a camera 310, an X-ray irradiator 320, an X-ray detector 330, a user input interface 360, and a display 370. In Figure 13 , only the minimum components for describing the functions and / or operations of the X-ray imaging device 300 are illustrated, and the components included in the X-ray imaging device 300 are not limited to those illustrated in Figure 13 . The components of the X-ray imaging device 300 will be described in detail in conjunction with Figure 14 .

[0157] In operation ①, the X-ray imaging device 300 can obtain an object image 1300 by photographing the object 10 with the camera 310. When the positioning of the patient in front of the X-ray detector 330 is completed, the X-ray imaging device 300 can obtain an image of the object 10 through the camera 310. The X-ray imaging device 300 can automatically recognize that the object 10 (e.g., the patient) has been positioned in front of the X-ray detector 330, and in response to recognizing the positioning of the patient, obtain an image of the object 10 by using the camera 310. In an embodiment of the present disclosure, the X-ray imaging device 300 may further include a depth measurement device 380 implemented by using at least one of a stereo type camera, a ToF camera, and a laser range finder (see Figure 19 ), and detect the positioning of the patient by measuring the distance between the X-ray irradiator 320 and the object 10 by using the depth measurement device 380.

[0158] The object image 1300 obtained in operation ① is a 2D image obtained by the camera 310 having a conventional image sensor (e.g., CMOS or CCD), and this 2D image is different from the X-ray image obtained by the X-ray detector 330 receiving the X-rays transmitted through the object 10 and performing image processing on the X-rays. In an embodiment of the present disclosure, the X-ray imaging device 300 can display the object image 1300 on the display 370.

[0159] In operation ②, the X-ray imaging device 300 obtains a plurality of divided imaging regions 1310-1 to 1310-3 for stitching X-ray imaging of an object 10 by using the AI model 352. The X-ray imaging device 300 can input the object image 1300 into the trained AI model 352, and perform inference by using the AI model 352 to obtain a plurality of divided imaging regions 1310-1 to 1310-3 for stitching X-ray imaging of the object 10. In an embodiment of the present disclosure, the AI model 352 may be a deep neural network model trained by a supervised learning method, and this supervised learning method applies a plurality of images as input data, and applies the position coordinates indicating the divided imaging regions stitched according to the imaging protocol as the ground truth. The deep neural network model may be a convolutional neural network (CNN) model, but is not limited thereto. The deep neural network model may be implemented by, for example, CenterNet, but is not limited thereto.

[0160] In the present disclosure, the term "stitching" refers to image processing for obtaining one X-ray image by connecting a plurality of X-ray images of a plurality of divided X-ray imaging regions 1310-1 to 1310-3. Stitching may include: image processing for detecting an overlapping portion between X-ray images obtained for the plurality of divided imaging regions 1310-1 to 1310-3 and connecting the detected overlapping portions. In Figure 13 , the plurality of divided imaging regions 1310-1 to 1310-3 are illustrated as a total of three obtained by dividing a target imaging region of the object 10, but this is merely illustrative and not limited thereto. The number of the divided imaging regions 1310-1 to 1310-3 and the number of times of division imaging may be determined based on at least one of a part of an object to be X-ray imaged, an imaging protocol, the size (e.g., height) or shape (e.g., body type) of the object 10, and may be determined to be two or more.

[0161] In an embodiment of the present disclosure, the X-ray imaging device 300 may receive a user input for selecting the automatic stitching planning UI 1302 through the user input interface 360, and in response to receiving the user input, obtain a plurality of divided imaging regions 1310-1 to 1310-3 for stitching X-ray imaging from the object image 1300 through the camera 310. In an embodiment of the present disclosure, the automatic stitching planning UI 1302 may be a graphical UI displayed on the display 370. In this case, the user input interface 360 and the display 370 may be integrated into a touch screen type.

[0162] In operation ③, the X-ray imaging device 300 displays a graphical UI representing a plurality of divided imaging regions 1310-1 to 1310-3. The X-ray imaging device 300 may display a plurality of guide lines 1320S, 1320-1, 1320-2, and 1320-3 on the display 370 representing the top and bottom of the plurality of divided imaging regions 1310-1 to 1310-3. In an embodiment of the present disclosure, the plurality of guide lines 1320S, 1320-1, 1320-2, and 1320-3 may represent not only the top and bottom of the plurality of divided imaging regions 1310-1 to 1310-3, but also the left and right boundaries. The X-ray imaging device 300 may display the graphical UI by superimposing the graphical UI representing the plurality of guide lines 1320S, 1320-1, 1320-2, and 1320-3 on the plurality of divided imaging regions 1310-1 to 1310-3 in the object image 1300. Among the plurality of guide lines 1320S, 1320-1, 1320-2, and 1320-3 displayed on the display 370, the upper indicator 1320S may be a graphical UI representing the top of the first divided imaging region 1310-1. The first guide line 1320-1 may be a graphical UI representing the bottom of the first divided imaging region 1310-1 and the top of the second divided imaging region 1310-2, the second guide line 1320-2 may be a graphical UI representing the bottom of the second divided imaging region 1310-2 and the top of the third divided imaging region 1310-3, and the third guide line 1320-3 may be a graphical UI representing the bottom of the third divided imaging region 1310-3.

[0163] In an embodiment of the present disclosure, the X-ray imaging device 300 may display a divided imaging count UI 1330 that represents the number of divided imaging times corresponding to the plurality of divided imaging regions 1310-1 to 1310-3. In Figure 13 it, the divided imaging count UI 1330 may display the number of divided imaging times as numbers (e.g., 1, 2, 3,...).

[0164] In an embodiment of the present disclosure, the X-ray imaging device 300 may display a graphical UI including a stitching icon 1340, a reset icon 1342, and a settings icon 1344 on the display 370. The stitching icon 1340 is a graphical UI for receiving a user input to display a plurality of guiding lines by superimposing the plurality of guiding lines 1320S, 1320-1, 1320-2, and 1320-3 on the object image 1300. The reset icon 1342 is a graphical UI for receiving a user input to enter a reset mode for changing at least one of the position, size, and shape of the plurality of divided imaging regions 1310-1 to 1310-3 by changing the positions of the plurality of guiding lines 1320S, 1320-1, 1320-2, and 1320-3. The settings icon 1344 is a graphical UI for receiving a user input to determine the plurality of displayed divided imaging regions 1310-1 to 1310-3 and perform stitched X-ray imaging.

[0165] According to the X-ray imaging device 300 of the embodiment illustrated in Figure 13 It is possible to obtain a plurality of divided imaging regions 1310-1 to 1310-3 for stitched imaging before X-ray imaging by using the AI model 352, and display a plurality of guiding lines 1320S, 1320-1, 1320-2, and 1320-3 indicating the top and bottom of the plurality of divided imaging regions 1310-1 to 1310-3, thereby allowing the user to effectively, appropriately, conveniently, and intuitively understand the number of divided imaging regions. The X-ray imaging device 300 according to an embodiment of the present disclosure may automate the entire stitching process, thereby effectively acquiring an X-ray image and reducing the stitching imaging preparation time. In addition, in an embodiment of the present disclosure, the X-ray imaging device 300 may provide the following technical effects: preventing an increase in the radiographic time of the patient and the risk of additional radiation exposure or excessive radiation that occur during reshooting due to inaccurate imaging region settings.

[0166] Figure 14 is a block diagram illustrating components of the X-ray imaging device 300 according to an embodiment of the present disclosure.

[0167] In Figure 14 The X-ray imaging device 300 illustrated in may be a movable type device including a movable X-ray detector 330. However, it is not limited thereto, and the X-ray imaging device 300 may be implemented as a ceiling type. The ceiling type X-ray imaging device 300 will be described in detail later in conjunction with Figure 20

[0168] Refer to Figure 14 ​, the X-ray imaging device 300 may include a camera 310, an X-ray irradiator 320, an X-ray detector 330, a processor 340, a memory 350, a user input interface 360, and a display 370. The camera 310, the X-ray irradiator 320, the X-ray detector 330, the processor 340, the memory 350, the user input interface 360, and the display 370 may be electrically and / or physically connected to each other. Figure 14 In the figure, only the necessary components for describing the operation of the X-ray imaging apparatus 300 are illustrated, and the components included in the X-ray imaging apparatus 300 are not limited to Figure 14 In the embodiment of the present disclosure, the X-ray imaging device 300 may further include a computer for communicating with the workstation 400 (see Figure 20 ), Server 2000 (see Figure 1 ), other medical equipment 3000 (see Figure 1 ) or external portable terminal 4000 (see Figure 1 ) performs data communication through a communication interface 390 (see Figure 20 ).

[0169] The camera 310, the X-ray irradiator 320 and the X-ray detector 330 are connected to the Figure 5 The camera 110 described and illustrated in Figure 5 )、X-ray irradiator 120 (see Figure 5 ) and the X-ray detector 130 (see Figure 5 ) have the same components and perform the same functions and / or operations of the camera 110, the X-ray irradiator 120 and the X-ray detector 130, and thus a redundant description will not be repeated.

[0170] The processor 340 may execute one or more instructions of a program stored in the memory 350. The processor 340 may include hardware components for performing arithmetic operations, logical operations, and input / output operations and image processing. The processor 340 may be configured to execute one or more instructions of a program stored in the memory 350. Figure 14 is illustrated as an element, but is not limited to this. In an embodiment of the present disclosure, the processor 340 may be configured with one or more elements. The processor 340 may be: a general-purpose processor, such as a central processing unit (CPU), an application processor (AP), a digital signal processor (DSP), etc.; a dedicated graphics processor, such as a graphics processing unit (GPU), a visual processing unit (VPU), etc.; or a dedicated artificial intelligence (AI) processor, such as a neural processing unit (NPU). The processor 340 may control the processing of input data according to predefined operating rules or AI models. When the processor 340 is a dedicated AI processor, the dedicated AI processor may be designed in a hardware structure dedicated to processing using a specific AI model.

[0171] The memory 350 may include, for example, at least one type of storage medium, including flash memory, hard disk, multimedia card micro memory, card-type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), or optical disc.

[0172] The memory 350 may store instructions related to the following functions and / or operations of the X-ray imaging device 300: obtaining a divided imaging area for stitching X-ray imaging from an object image acquired by the camera 310, and displaying a plurality of guide line UIs representing the top, bottom, left boundary, and right boundary of the divided imaging area. In an embodiment of the present disclosure, the memory 350 may store at least one of algorithms, data structures, program codes, application programs, and instructions that can be read by the processor 340. The instructions, algorithms, data structures, and program codes stored in the memory 350 may be implemented in, for example, a programming or scripting language (such as C, C++, Java, assembly language, etc.).

[0173] In the following embodiments, the processor 340 may be implemented by running the instructions or program codes stored in the memory 350.

[0174] The processor 340 may acquire image data of an object image obtained by photographing an object with the camera 310. In response to the completion of the positioning of the patient in front of the X-ray detector 330, the processor 340 may acquire the image data of the object by controlling the camera 310 to acquire an image of the object. In an embodiment of the present disclosure, the processor 340 may receive a user touch input for selecting a button UI for executing the automatic stitching imaging mode through the user input interface 360, and in response to receiving the touch input, control the camera 310 to acquire an image of the object for stitching imaging. The user input for executing the automatic stitching imaging mode is not limited to touch input, but may correspond to an input such as pressing a keyboard, a hardware button, a micro switch, etc.

[0175] However, not limited thereto, and the X-ray imaging device 300 may further include a depth measurement device 380 (see Figure 19 ), and the processor 340 may identify an object positioned in front of the X-ray detector 330 by using the depth measurement device 380, and in response to identifying the object, execute the automatic stitching imaging mode to automatically acquire an image of the object. Specific embodiments in which the processor 340 identifies the position of the object by using the depth measurement device 380 will be described in detail in Figure 19 .

[0176] The processor 340 may obtain multiple divided imaging regions for stitching X-ray imaging by analyzing the inference of the object image using the AI model 352. In an embodiment of the present disclosure, the AI model 352 may be implemented by instructions, program codes, or algorithms stored in the memory 350, but is not limited thereto. In an embodiment of the present disclosure, the AI model 352 may not be included in the X-ray imaging device 300. In this case, the AI model 432 (see Figure 20 ) may be included in the workstation 400 (see Figure 20 ).

[0177] The processor 340 may input the object image into the AI model 352 and obtain multiple divided imaging regions for stitching X-ray imaging by using the inference of the AI model 352. In an embodiment of the present disclosure, the AI model 352 may be a deep neural network model trained by a supervised learning method that applies the obtained multiple object images as input data and applies the divided imaging regions stitched according to the imaging protocol as the ground truth. The ground truth of the divided imaging regions may be determined differently according to the imaging protocol. To improve the accuracy of the divided imaging regions, the deep neural network model may be trained in a partially modified form. When the input data (e.g., multiple images) and the ground truth data (e.g., the position coordinates of the divided imaging regions) for training are insufficient, the data may also be processed by augmentation to increase the data volume or by applying a fine-tuning method that partially modifies the trained model.

[0178] In an embodiment of the present disclosure, the deep neural network model may be a convolutional neural network (CNN) model. The deep neural network model may be implemented by, for example, CenterNet. However, it is not limited thereto, and the deep neural network model may be implemented by, for example, a recurrent neural network, a restricted Boltzmann machine, a deep belief network, a bidirectional recurrent deep neural network, or a deep Q-network.

[0179] The sizes of the multiple divided imaging regions obtained by the AI model 352 may be larger than the size of the region that can be obtained by the X-ray detector 330. In an embodiment of the present disclosure, the processor 340 may adjust the sizes of the multiple divided imaging regions obtained by the AI model 352 to be smaller than the size of the X-ray detector 330.

[0180] The processor 340 may identify the target imaging part from the object image based on the imaging protocol, input the information about the identified target imaging part together with the object image into the AI model 352, and obtain multiple divided imaging regions by using the inference of the AI model 352. It will be described in Figure 16Specific embodiments in which the processor 340 obtains a plurality of divided imaging regions based on the target imaging portion are described in detail.

[0181] The processor 340 may display an object image through the display 370. In an embodiment of the present disclosure, the processor 340 may control the display 370 to display the graphical UI by superimposing the graphical UI representing the plurality of divided imaging regions on the object image.

[0182] The user input interface 360 may receive a user input to adjust the position of at least one of a plurality of guiding lines representing the top, bottom, left boundary, and right boundary of the plurality of divided imaging regions. In an embodiment of the present disclosure, the user input interface 360 may receive a touch input from the user to adjust the position of at least one of the plurality of guiding lines. The processor 340 may change at least one of the position, size, and shape of the plurality of divided imaging regions by adjusting the position of at least one of the plurality of guiding lines based on the user input received through the user input interface 360. Specific embodiments in which the processor 340 changes at least one of the position, size, and shape of the plurality of divided imaging regions based on the user input will be described in detail in conjunction with Figure 17a and Figure 17b Specific embodiments in which the processor 340 changes at least one of the position, size, and shape of the plurality of divided imaging regions based on the user input will be described in detail.

[0183] The user input interface 360 may receive a user input to adjust the size of the margin between the X-ray imaging region and the target imaging region by adjusting the size of the target imaging region of the object. The processor 340 may determine the top margin size, bottom margin size, left margin size, and right margin size of the plurality of divided imaging regions based on the user input received through the user input interface 360. Specific embodiments in which the processor 340 determines or adjusts the margin size of the X-ray imaging region based on the user input will be described in detail in conjunction with Figure 18 Specific embodiments in which the processor 340 determines or adjusts the margin size of the X-ray imaging region based on the user input will be described in detail.

[0184] The processor 340 may obtain at least one divided X-ray imaging image by performing X-ray imaging on the plurality of divided imaging regions. The processor 340 may control the X-ray irradiator 320 to irradiate X-rays onto the object, receive the X-rays transmitted through the object through the X-ray detector, and obtain a plurality of divided X-ray imaging images by converting the received X-rays into electrical signals. The processor 340 may obtain an X-ray image of the target X-ray imaging region by stitching the plurality of divided X-ray imaging images.

[0185] The user input interface 360 is configured to provide an interface for operating the X-ray imaging device 300. The user input interface 360 may be, for example but not limited to, a control panel including hardware elements such as a keyboard, a mouse, a trackball, a jog dial, a micro switch, or a touchpad. In an embodiment of the present disclosure, the user input interface 360 may be configured as a touch screen that receives touch inputs and displays a graphical UI.

[0186] The display 370 may display an object image under the control of the processor 340. In an embodiment of the present disclosure, the display 370 may, under the control of the processor 340, display the graphical UI by superimposing the graphical UI representing a plurality of divided imaging regions on the object image. The display 370 may be configured with a hardware device that includes at least one of, for example but not limited to, a CRT monitor, an LCD monitor, a PDP monitor, an OLED monitor, an FED monitor, an LED monitor, a VFD monitor, a DLP monitor, a flat panel monitor, a 3D monitor, and a transparent monitor. In an embodiment of the present disclosure, the display 370 may include a touch screen having a touch interface. In the case where the display 370 is configured as a touch screen, the display 370 may be a component integrated with the user input interface 360 including a touch panel.

[0187] Although not illustrated in Figure 14 the X-ray imaging device 300 may further include a speaker configured to output an acoustic signal. In an embodiment of the present disclosure, the processor 340 may control the speaker to output information related to the completion of setting a plurality of divided imaging regions in voice or notification sound.

[0188] Figure 15 is a flowchart illustrating a method according to an embodiment of the present disclosure, by which the X-ray imaging device 300 obtains divided imaging regions for stitching X-ray imaging on an image acquired by a camera and displays a graphical user interface (UI) representing the divided imaging regions.

[0189] In operation S1510, the X-ray imaging device 300 obtains an object image by photographing an object positioned in front of the X-ray detector. The X-ray imaging device 300 may obtain image data by photographing an object (e.g., a patient) positioned in front of the X-ray detector 330 (see Figure 13 and Figure 14 ) using a camera. The object image 1300 obtained in operation 1510 is a 2D image obtained by a camera having a conventional image sensor (e.g., CMOS or CCD), which is different from an X-ray image obtained by receiving X-rays transmitted through the object by the X-ray detector 330 and performing image processing thereon.

[0190] In operation S1520, the X-ray imaging device 300 inputs an object image into the trained AI model and obtains multiple divided imaging regions for X-ray imaging by performing inference using the AI model. In an embodiment of the present disclosure, the AI model may be a deep neural network model trained by a supervised learning method that applies the obtained multiple object images as input data and applies the divided imaging regions stitched according to the imaging protocol as the ground truth. The deep neural network model may be a convolutional neural network (CNN) model. The deep neural network model may be implemented by, for example, CenterNet. However, it is not limited thereto, and the deep neural network model may be implemented by, for example, a recurrent neural network, a restricted Boltzmann machine, a deep belief network, a bidirectional recurrent deep neural network, or a deep Q network.

[0191] The X-ray imaging device 300 may identify a target imaging part from the object image based on the imaging protocol and obtain multiple divided imaging regions by inputting the information about the identified target imaging part together with the object image into the inference of the AI model.

[0192] In operation S1530, the X-ray imaging device 300 displays a graphical UI representing the top, bottom, left boundary, and right boundary of the multiple divided imaging regions. In an embodiment of the present disclosure, the X-ray imaging device 300 may display the object image and display the plurality of guide lines by superimposing the plurality of guide lines representing the top, bottom, left boundary, and right boundary of the multiple divided imaging regions on the object image.

[0193] In an embodiment of the present disclosure, the X-ray imaging device 300 may output a voice or a notification sound that provides the user with information related to the completion of setting the multiple divided imaging regions.

[0194] The X-ray imaging device 300 may obtain multiple divided X-ray imaging images by performing X-ray imaging on the multiple divided imaging regions and obtain an X-ray image of the target X-ray imaging region by stitching the obtained multiple divided X-ray imaging images.

[0195] Figure 16 is a schematic diagram illustrating an operation of the X-ray imaging device 300 according to an embodiment of the present disclosure for determining divided imaging regions according to an imaging protocol and displaying a graphical UI representing the determined divided imaging regions.

[0196] The processor 340 of the X-ray imaging device 300 may identify an imaging protocol based on the object image. The imaging protocol may include, for example, a whole spine protocol, a long bone protocol, or a limb protocol, but is not limited thereto. The processor 340 may identify a target imaging part from the object image based on the imaging protocol. For example, in the case of the whole spine protocol, the processor 340 may identify the part from the ears to below the pelvis (e.g., the head, shoulders, elbows, hands, waist, etc.) as the target imaging part from the object image. For example, in the case of the long bone protocol, the processor 340 may identify the part from the waist to the toes as the target imaging part, and in the case of the limb protocol, the processor 340 may identify parts such as the face, hands, or feet as the target imaging part.

[0197] The processor 340 may input information about the identified target imaging part together with the object image into the AI model 352 and perform inference using the AI model 352. The AI model 352 may be a deep neural network model trained by a supervised learning method that applies multiple images of an object as input data and applies the divided imaging regions stitched according to the imaging protocol as the ground truth. The ground truth of the divided imaging regions may be determined differently according to the imaging protocol. For example, in the case of the whole spine protocol in the imaging protocol, the head, shoulders, elbows, hands, waist, etc. in the body part from the ears to below the pelvis may be determined as the divided regions, and in the case of the long bone protocol, the part from the waist to the toes may be determined as the divided region. For example, in the case of the limb protocol for the head (skull), hands, or feet, parts with unique body characteristics such as the face, hands, or feet may be determined as the divided regions. The processor 340 may identify the target imaging part from the object image by performing inference using the deep neural network model and display the divided imaging regions.

[0198] During the process of performing inference using the deep neural network model, when the data of the deep neural network model is encrypted, the processor 340 may decode the data. In the case where multiple candidates for the target imaging region are output as a result of the inference of the deep neural network model, the processor 340 may determine the final target imaging region by selecting one candidate with the highest confidence among the multiple candidates.

[0199] Refer to Figure 16In the illustrated embodiment, the processor 340 may identify a first protocol based on the first object image 1600. The processor 340 may input the first object image 1600 and the identified first protocol into the AI model 352, identify the first target imaging region 1620 through inference using the AI model 352, and obtain multiple guiding lines 1610-1 to 1610-3 that divide the first target imaging region 1620 into multiple divided imaging regions. For example, the first protocol may be an entire spine protocol, and the first target imaging region 1620 may include a portion of the body part from the ears to below the pelvis. The multiple guiding lines 1610-1 to 1610-3 may be graphical UIs indicating the tops and bottoms of multiple divided regions such as the head, shoulders, elbows, waist, etc. in the portion from the ears to below the pelvis. The processor 340 may display the first object image 1600 on the display 370, and display the first target imaging region 1620 and the multiple guiding lines 1610-1 to 1610-3 by superimposing the first target imaging region 1620 and the multiple guiding lines 1610-1 to 1610-3 on the first object image 1600.

[0200] Similarly, the processor 340 may identify a second protocol based on the second object image 1602, identify a second target imaging region 1622 corresponding to the second protocol from the second object image 1602 through inference using the AI model 352, and obtain multiple guiding lines 1612-1 to 1612-4 that divide the second target imaging region 1622 into multiple divided imaging regions. For example, the second protocol may be a long bone protocol, and the second target imaging region 1622 may include a portion of the body part from the waist to the toes. The processor 340 may display the second target imaging region 1622 and the multiple guiding lines 1612-1 to 1612-4 by superimposing the second target imaging region 1622 and the multiple guiding lines 1612-1 to 1612-4 on the second object image 1602 displayed on the display 370. The processor 340 may identify a third protocol based on the third object image 1604, identify a third target imaging region 1624 corresponding to the third protocol from the third object image 1604 through inference using the AI model 352, and obtain multiple guiding lines 1614-1 to 1614-5 that divide the third target imaging region 1624 into multiple divided imaging regions. For example, the third protocol may be a limb protocol, and the third target imaging region 1624 may include a portion of the body part from the shoulders to the fingertips. The processor 340 may display the third target imaging region 1624 and the multiple guiding lines 1614-1 to 1614-5 by superimposing the third target imaging region 1624 and the multiple guiding lines 1614-1 to 1614-5 on the third object image 1604 displayed on the display 370.

[0201] In Figure 16 the illustrated embodiment, the X-ray imaging device 300 can identify a target imaging part for each imaging protocol, and when identifying the target imaging regions 1620, 1622, and 1624 for the corresponding imaging protocol by performing inference using the AI model 352, the X-ray imaging device 300 can automatically stitch the X-ray imaging according to the imaging protocol and detect the accurate and reliable target imaging regions 1620, 1622, and 1624. Accordingly, user convenience can be increased, and the time for stitching the X-ray imaging can be reduced.

[0202] Figure 17a is a schematic diagram illustrating an operation of the X-ray imaging device 300 according to an embodiment of the present disclosure for changing at least one of a position, a size, and a shape of a divided imaging region based on a user input.

[0203] Referring to Figure 17a , an object image 1700a can be displayed on the display 370, and a plurality of guide lines 1710S and 1710-1 to 1710-4 indicating the top and bottom of the plurality of divided imaging regions can be displayed on the object image 1700a. A user can adjust a position of at least one of the plurality of guide lines 1710S and 1710-1 to 1710-4. The user input interface 360 (see Figure 14 ) of the X-ray imaging device 300 can receive a user input for adjusting a position of at least one of the plurality of guide lines 1710S and 1710-1 to 1710-4. In an embodiment of the present disclosure, the user input interface 360 can be configured with a touch screen including a touch pad, and in this case, the user input interface 360 can be a component integrated with the display 370. The user input interface 360 can receive a touch input of a user for adjusting a position of at least one of the plurality of guide lines 1710S and 1710-1 to 1710-4. However, it is not limited thereto, and the user input interface 360 can receive a user input for adjusting a position of at least one of the plurality of guide lines 1710S and 1710-1 to 1710-4 through a keyboard, a hardware button, a mouse, a micro switch, or a micro dial. In Figure 17a the illustrated embodiment, the user input interface 360 can receive a user input for downwardly adjusting a position of the upper indicator 1710S, which indicates the top of the first divided imaging region among the plurality of guide lines 1710S and 1710-1 to 1710-4.

[0204] In response to receiving the user input, the processor 340 (see Figure 14It is possible to change the position of at least one guiding line and, based on the at least one changed guiding line, change at least one of the position, size, and shape of the plurality of divided imaging regions. In Figure 17a In the illustrated embodiment, the processor 340 may change the size and shape of the plurality of divided imaging regions based on the upper indicator 1710a whose position is adjusted by user input. The processor 340 may change the size and position of the plurality of divided imaging regions by evenly dividing the region between the upper indicator 1710a whose position is adjusted and the fourth guiding line 1710-4. The processor 340 may display the changed upper indicator 1710a and the changed plurality of divided imaging regions on the display 370.

[0205] In an embodiment of the present disclosure, when the position of at least one guiding line is adjusted by user input, the processor 340 may change the number of imaging divisions. For example, in the case of receiving user input for downwardly adjusting the position of the upper indicator 1710S, the processor 340 may reduce the number of imaging divisions. For example, in the case of receiving user input for upwardly adjusting the position of the upper indicator 1710S or downwardly adjusting the position of the fourth guiding line 1710-4, the processor 340 may increase the number of imaging divisions. For example, the processor 340 may reduce the number of imaging divisions from four to three and evenly divide the region between the upper indicator 1710a whose position is adjusted by user input and the fourth guiding line 1710-4 into three regions.

[0206] However, not limited thereto, and in an embodiment of the present disclosure, the processor 340 may unevenly divide the region between the upper indicator 1710a and the fourth guiding line 1710-4. For example, when adjusting the position of the upper indicator 1710a, the processor 340 may only change the size and shape of the first divided imaging region.

[0207] Figure 17b is a schematic diagram illustrating an operation of the X-ray imaging device 300 according to an embodiment of the present disclosure for changing at least one of the position, size, and shape of the divided imaging regions based on user input.

[0208] Figure 17b Same as Figure 17a except that one of the plurality of guiding lines 1710S and 1710-1 to 1710-4 whose position is adjusted by user input is the third guiding line 1710-3, and the position of the third guiding line 1710-3 is upwardly adjusted by user input, so overlapping descriptions will be omitted.

[0209] Referring to Figure 17b , the user input interface 360 of the X-ray imaging device 300 (seeFigure 14 It can receive user input for upwardly adjusting the position of the fourth guiding line 1710-4, which represents the bottom of the fourth divided imaging region among the plurality of guiding lines 1710S and 1710-1 to 1710-4. In response to receiving the user input, the processor 340 of the X-ray imaging device 300 (see Figure 14 It can change the size and shape of the plurality of divided imaging regions based on the fourth guiding line 1710b whose position is adjusted by the user input. The processor 340 can change the size and position of the plurality of divided imaging regions by evenly dividing the region between the first guiding line 1710-1 and the fourth guiding line 1710b based on the fourth guiding line 1710b whose position is adjusted.

[0210] However, not limited thereto, and the processor 340 can unevenly divide the region between the first guiding line 1710-1 and the fourth guiding line 1710b. For example, when the position of the third guiding line 1710b is adjusted upward, the processor 340 can only change the size and shape of the fourth divided imaging region.

[0211] In Figure 17a and Figure 17b In the illustrated embodiment, the X-ray imaging device 300 can change at least one of the position, size, and shape of the plurality of divided imaging regions obtained by the AI model 352 (see Figure 13 and Figure 14 ) by adjusting the positions of the plurality of guiding lines 1710S and 1710-1 to 1710-4 by user input. Therefore, in the special case where the AI model 352 inappropriately obtains the divided imaging regions or the user needs to personally change or adjust the divided imaging regions, the X-ray imaging device 300 according to the embodiments of the present disclosure can allow the user to manually adjust the position, size, and shape of the plurality of divided imaging regions, thereby increasing user convenience and achieving accurate X-ray imaging.

[0212] Figure 18 is a schematic diagram illustrating an operation of the X-ray imaging device 300 according to an embodiment of the present disclosure for determining the margin of a divided imaging region based on user input.

[0213] Referring to Figure 18 , the X-ray imaging device 300 can display the object image 1800 on the display 370, and display the target imaging region 1810 by superimposing the target imaging region 1810, which is a graphical UI representing X-ray imaging, on the object image 1800. The user input interface 360 of the X-ray imaging device 300 (see Figure 14 ) can receive for adjusting a plurality of margins d in the up, down, left, and right directionsm1 to d m4 A user input for determining the X-ray imaging area 1820 is obtained based on one of the margins from d to d. In an embodiment of the present disclosure, the user input interface 360 may be configured as a touch screen including a touchpad. In this case, the user input interface 360 may be a component integrated with the display 370. The user input interface 360 may receive a plurality of margins d of the target imaging area 1810 displayed as a graphical UI on the touch screen from the user for adjusting in the up, down, left, and right directions m1 to d m4 A touch input for one of the margins. However, it is not limited thereto. The user input interface 360 may receive a user input for adjusting a plurality of margins d in the up, down, left, and right directions through a keyboard, a hardware button, a mouse, a micro switch, or a micro dial m1 to d m4 A user input for one of the margins.

[0214] The processor 340 of the X-ray imaging device 300 (see Figure 14 ) may adjust the size of one of the plurality of margins d in the up, down, left, and right directions based on the user input received through the user input interface 360. The processor 340 may set the X-ray imaging area 1820 based on the margin whose size is adjusted m1 to d m4 Figure is a schematic diagram illustrating an operation of the X-ray imaging device 300 according to an embodiment of the present disclosure for detecting the positioning of the object 10 by using the depth measurement device 380

[0215] Figure 19 Refer to

[0216] The X-ray imaging device 300 may include a camera 310, an X-ray irradiator 320, an X-ray detector 330, and a depth measurement device 380. The camera 310, the X-ray irradiator 320, and the X-ray detector 330 illustrated in Figure 19 are the same as the camera 110 (see Figure 19 ), the X-ray irradiator 120 (see Figure 4 ), and the X-ray detector 130 (see Figure 4 ), and the X-ray detector 130 (see Figure 4 ), respectively, and thus redundant descriptions will not be repeated

[0217] The depth measurement device 380 is configured to measure the distance between the X-ray irradiator 320 and the object 10. In an embodiment of the present disclosure, the depth measurement device 380 may include at least one of a stereo type camera, a time-of-flight (ToF) camera, and a laser distance measurer. The processor 340 of the X-ray imaging device 300 (see Figure 14The patient positioning can be detected by measuring the distance between the X-ray irradiator 320 and the object 10 using the depth measurement device 380. After detecting the patient positioning, the processor 340 can obtain an object image by photographing the object 10 via the camera 310, and obtain a plurality of divided imaging regions by analyzing the object image using the AI model 352 (see Figure 13 and Figure 14 ).

[0218] In an embodiment of the present disclosure, when the distance between the X-ray irradiator 320 and the object 10 obtained by the depth measurement device 380 (i.e., the source-to-image distance (SID)) is outside the preset range, the processor 340 can determine that the object 10 is abnormally positioned in the imaging position. In this case, the processor 340 can output the divided imaging regions with default settings on the display 370 (see Figure 13 and Figure 14 ), regardless of the imaging protocol.

[0219] Figure 20 is a block diagram illustrating components of the X-ray imaging device 300 and the workstation 400 according to an embodiment of the present disclosure.

[0220] In Figure 20 the illustrated X-ray imaging device 300 can be implemented as a ceiling type. Referring to Figure 20 , the X-ray imaging device 300 can include a camera 310, an X-ray irradiator 320, an X-ray detector 330, a processor 340, a user input interface 360, a display 370, and a communication interface 390. In Figure 20 the illustrated X-ray imaging device 300 is the same as the X-ray imaging device 300 (see Figure 14 ), except that the former does not include the memory 350 (see Figure 14 ), but may further include a communication interface 390, so redundant descriptions will be omitted.

[0221] The communication interface 390 can process data while sending data to the workstation 400 and receiving data from the workstation 400 via a wired communication network or a wireless communication network. The communication interface 390 can perform data communication with the workstation 400 by using at least one of data communication schemes including, for example, wireless local area network (WLAN), Wi-Fi, Bluetooth, zigbee, WFD, infrared data association (IrDA), Bluetooth low energy (BLE), near field communication (NFC), wireless broadband internet (Wibro), worldwide interoperability for microwave access (WiMAX), shared wireless access protocol (SWAP), wireless gigabit alliance (WiGig), and radio frequency (RF) communication.

[0222] In an embodiment of the present disclosure, under the control of the processor 340, the communication interface 390 may transmit an object image obtained by photographing an object via the camera 310 to the workstation 400, and receive data on the divided imaging regions for X-ray imaging from the workstation 400. The X-ray imaging device 300 may display, on the object image via the display 370, a plurality of guiding lines representing the divided imaging regions based on the received data on the divided imaging regions.

[0223] The workstation 400 may include: a communication interface 410 for communicating with the X-ray imaging device 300, a memory 430 for storing at least one instruction or program code, and a processor 420 configured to run the instruction or program code stored in the memory 430.

[0224] The AI model 432 may be stored in the memory 430 of the workstation 400. Except for the storage location, the AI model 432 stored in the workstation 400 is the same as the AI model 352 described and illustrated in Figure 13 and Figure 14 (see Figure 13 and Figure 14 ), so redundant descriptions will be omitted. The workstation 400 may receive the image data of the object image from the X-ray imaging device 300 via the communication interface 410. The processor 420 of the workstation 400 may input the object image into the AI model 432, and obtain a plurality of divided imaging regions for X-ray imaging by performing inference using the AI model 432. In an embodiment of the present disclosure, the workstation 400 may receive information on the imaging protocol from the X-ray imaging device 300 via the communication interface 410, or receive user input for setting the imaging protocol via the input interface 440. The processor 420 may identify the target imaging part from the object image based on the imaging protocol, input the information on the identified target imaging part together with the object image into the AI model 432, and obtain a plurality of divided imaging regions by performing inference using the AI model 432.

[0225] The processor 420 of the workstation 400 may control the communication interface 410 to transmit the data on the divided imaging regions to the X-ray imaging device 300.

[0226] Generally, compared with the workstation 400, the memory 350 of the X-ray imaging device 300 (see Figure 14The storage capacity of ) and the operation processing speed of the processor 340 may be restricted. Accordingly, the workstation 400 may perform operations that require storing a vast amount of data and a large amount of computation (e.g., obtaining the divided imaging regions by performing inference using the AI model 432), and then transmit the required data (e.g., data regarding the divided imaging regions) to the X-ray imaging device 300 through the communication network. In this way, even without a large-capacity memory and a processor with high-speed computing capabilities, the X-ray imaging device 300 may receive data regarding the divided imaging regions from the workstation 400 and display a plurality of guiding lines representing the divided imaging regions, thereby reducing the processing time spent on obtaining the divided imaging regions and improving the accuracy of the divided imaging regions.

[0227] The present disclosure provides an X-ray imaging device 100 for detecting the movement of an object. In an embodiment of the present disclosure, the X-ray imaging device 100 may include: an X-ray irradiator 120 configured to generate X-rays and irradiate the X-rays onto the object; an X-ray detector 130 configured to detect the X-rays irradiated by the X-ray irradiator 120 and transmitted through the object; a camera 110 configured to obtain an object image by photographing an image of the object positioned in front of the X-ray detector 130; a display 172; and at least one processor 140. The at least one processor 140 may be configured to detect the movement of the object based on the object image by analyzing the object image via using an AI model. The at least one processor 140 may be configured to output a notification signal on the display 172 to notify a user of the result of the detection of the movement of the object.

[0228] In an embodiment of the present disclosure, the at least one processor 140 may be configured to: obtain a reference image by photographing an image of the object positioned in front of the X-ray detector 130 after photographing is completed; and obtain an image frame by photographing a subsequent image of the object after obtaining the reference image. The at least one processor 140 may detect the movement of the object by comparing the object identified from the reference image with the object identified from the image frame via analysis based on an AI model.

[0229] In an embodiment of the present disclosure, the at least one processor 140 may use self-organizing mapping in the AI model to obtain weights for pixels representing the object identified from the reference image. The at least one processor 140 may use the weights to detect the movement of the object by comparing the object identified from the reference image with the object identified from the image frame. The at least one processor 140 may use the result of the detection to update the reference image and the weights.

[0230] In an embodiment of the present disclosure, the at least one processor 140 may extract a plurality of first key points of the feature landmark part of the object from the reference image by performing inference using a trained deep neural network model in an AI model. The at least one processor 140 may calculate the difference between the key points by comparing the extracted plurality of first key points with a plurality of second key points of the object extracted from the image frame. The at least one processor 140 may detect the movement of the object by comparing the calculated difference with a preset threshold value.

[0231] In an embodiment of the present disclosure, the deep neural network model may be a model trained by a supervised learning method, and the supervised learning method applies a plurality of acquired images as input data and applies the position coordinates of the key points of the feature landmark part as the ground truth.

[0232] In an embodiment of the present disclosure, the X-ray imaging device 100 may further include a user input interface configured to receive a user input for selecting a motion detection mode after the patient positioning is completed. The at least one processor 140 may perform the motion detection mode based on the received user input and detect the movement of the object in response to the execution of the motion detection mode.

[0233] In an embodiment of the present disclosure, the at least one processor 140 may perform the motion detection mode after a preset time has elapsed after the patient positioning is completed. The at least one processor 140 may detect the movement of the object in response to the execution of the motion detection mode.

[0234] In an embodiment of the present disclosure, the X-ray imaging device 100 may further include a depth measurement device 180, and the depth measurement device 180 includes at least one of a stereo type camera, a time-of-flight (ToF) camera, and a laser range finder. The at least one processor 140 may detect patient positioning by measuring the distance between the X-ray irradiator 120 and the object using the depth measurement device 180.

[0235] In an embodiment of the present disclosure, the at least one processor 140 may set the motion detection sensitivity based on at least one of the source-to-image distance (SID) that is the distance between the object and the X-ray irradiator 120, the size and shape of the object, and the imaging protocol.

[0236] In an embodiment of the present disclosure, the display 172 may display a graphical UI having a preset color indicating the movement of the object.

[0237] In an embodiment of the present disclosure, the X-ray imaging device 100 may further include a speaker 174 configured to output at least one acoustic signal among voice and notification sounds, and the at least one acoustic signal notifies a user of information about the movement of the object.

[0238] The present disclosure provides a method of operating an X-ray imaging device 100. According to an embodiment of the present disclosure, the method of operating the X-ray imaging device 100 may include: obtaining image data of the object by capturing an image of the object with a camera 110 (S610). According to an embodiment of the present disclosure, the method of operating the X-ray imaging device 100 may include: detecting movement of the object based on the image data by analyzing the image data using an AI model (S620). According to an embodiment of the present disclosure, the method of operating the X-ray imaging device 100 may include: outputting a notification signal to notify a user of a result of the detection of the movement of the object (S630).

[0239] In an embodiment of the present disclosure, obtaining the image data (S610) may include: obtaining a reference image by capturing an object positioned in front of the X-ray detector 130 with the camera 110; and obtaining an image frame by capturing a subsequent image of the object after obtaining the reference image. Detecting the movement of the object (S620) may include: detecting the movement of the object by comparing the object identified from the reference image with the object identified from the image frame via analysis based on an AI model.

[0240] In an embodiment of the present disclosure, detecting the movement of the object (S620) may include: obtaining weights from the reference image using a self-organizing map (S810); and using the weights to detect the movement of the object by comparing the object identified from the image frame with the object identified from the reference image (S820). Detecting the movement of the object (S620) may include: updating the reference image and the weights using the result of the detection (S830).

[0241] In an embodiment of the present disclosure, detecting the movement of the object (S620) may include: extracting a plurality of first key points of a feature landmark part of the object from the reference image by performing inference using a trained deep neural network model (S910); calculating a difference between the key points by comparing the extracted plurality of first key points with a plurality of second key points of the object extracted from the image frame (S920); and detecting the movement of the object by comparing the calculated difference with a preset threshold.

[0242] In an embodiment of the present disclosure, the method of operating the X-ray imaging device 100 may further include: receiving a user input to select a motion detection mode after patient positioning is completed. Detecting the motion of the object (S620) may include: performing a motion detection mode based on the user input, and detecting the motion of the object in response to the motion detection mode being performed.

[0243] In an embodiment of the present disclosure, the method of operating the X-ray imaging device 100 may further include: performing the motion detection mode after a preset time has elapsed after patient positioning is completed. Detecting the motion of the object (S620) may include: detecting the motion of the object in response to the motion detection mode being performed.

[0244] In an embodiment of the present disclosure, the method of operating the X-ray imaging device 100 may further include: setting motion detection sensitivity based on at least one of the source-to-image distance (SID) which is the distance between the object and the X-ray irradiator 120, the size and shape of the object, and the imaging protocol.

[0245] In an embodiment of the present disclosure, outputting the notification signal (S630) may include: displaying a graphical UI having a preset color representing the motion of the object.

[0246] In an embodiment of the present disclosure, the method of operating the X-ray imaging device 100 may include: outputting at least one acoustic signal of voice and a notification sound, the at least one acoustic signal notifying the user of information about the motion of the object.

[0247] The present disclosure provides an X-ray imaging device 300 for performing stitching X-ray imaging. In an embodiment of the present disclosure, the X-ray imaging device 300 may include: an X-ray detector 330 for detecting X-rays irradiated by an X-ray irradiator 320 and transmitted through an object; a camera 310 for obtaining an object image by photographing the object positioned in front of the X-ray detector 330; a display 370; and at least one processor 340. The at least one processor 340 may be configured to: input the object image into a trained AI model; and obtain a plurality of divided imaging regions for performing stitching X-ray imaging on the object by performing inference using the AI model. The at least one processor 340 may be configured to: display a plurality of guiding lines on the display 370 to indicate the top, bottom, left boundary, and right boundary of each of the plurality of divided imaging regions.

[0248] In an embodiment of the present disclosure, the AI model may be a deep neural network model trained by a supervised learning method. The supervised learning method uses a plurality of acquired images as input data and uses the divided imaging regions stitched according to the imaging protocol as the ground truth.

[0249] In an embodiment of the present disclosure, the at least one processor 340 may identify a target imaging part from the object image based on the imaging protocol. The at least one processor 340 may input information about the identified target imaging part together with the object image into the AI model, and obtain a plurality of divided imaging regions by performing inference using the AI model.

[0250] In an embodiment of the present disclosure, the at least one processor 340 may adjust the sizes of the plurality of divided imaging regions to be smaller than the size of the X-ray detector 330.

[0251] In an embodiment of the present disclosure, the X-ray imaging device 300 may further include a user input interface 360 configured to receive a user input for adjusting the position of at least one of the plurality of guiding lines. The at least one processor 340 may change at least one of the position, size, and shape of the plurality of divided imaging regions by adjusting the position of at least one of the plurality of guiding lines based on the received user input.

[0252] In an embodiment of the present disclosure, the at least one processor 340 may determine the upper margin size, lower margin size, left margin size, and right margin size of the plurality of divided imaging regions based on the margin information set by the user input.

[0253] In an embodiment of the present disclosure, the at least one processor 340 may control the display 370 to display the graphical UI by superimposing the graphical UI representing the plurality of divided imaging regions on the object image.

[0254] In an embodiment of the present disclosure, the X-ray imaging device 300 may further include a speaker for outputting information related to the completion of setting the plurality of divided imaging regions in voice or notification sound.

[0255] In an embodiment of the present disclosure, the X-ray imaging device 300 may further include a depth measurement device 380, and the depth measurement device 380 includes at least one of a stereo type camera, a time-of-flight (ToF) camera, and a laser range finder. The at least one processor 340 may detect the positioning of an object in front of the X-ray detector 330 by measuring the distance between the X-ray irradiator 320 and the object by using the depth measurement device 330.

[0256] In an embodiment of the present disclosure, the at least one processor 340 may obtain a plurality of divided X-ray imaging images by performing X-ray imaging on the plurality of divided imaging regions. The at least one processor 340 may obtain an X-ray image of a target X-ray imaging region by stitching the plurality of divided X-ray imaging images.

[0257] The present disclosure provides a computer program product including a computer-readable storage medium. The storage medium may include instructions that are readable for an X-ray imaging device 100 to perform: obtaining an object image by photographing an image of an object with a camera; detecting the movement of the object based on the object image by analyzing the object image by using an AI model; and outputting a notification signal that notifies a user of a result of the detection of the movement of the object.

[0258] A program executed by the X-ray imaging device 100 as described in the present disclosure may be implemented in a hardware element, a software element, and / or a combination thereof. The program may be executed by any system capable of executing computer-readable instructions.

[0259] Software may include a computer program, code, instructions, or one or more combinations thereof, and the software may configure a processing device to operate the processing device as needed, or independently or cooperatively instruct the processing device.

[0260] Software may be implemented by a computer program including instructions stored in a computer-readable record (or storage) medium. Examples of the computer-readable record medium include: a magnetic storage medium (e.g., a read-only memory (ROM), a floppy disk, a hard disk, etc.) and an optical recording medium (e.g., a compact disc ROM (CD-ROM) or a digital versatile disc (DVD)). The computer-readable record medium may also be distributed over a network-connected computer system such that the computer-readable code is stored and run in a distributed manner. The medium may be read by a computer, stored in a memory, and run by a processor.

[0261] It can be a computer-readable storage medium in the form of a non-transitory storage medium. The term "non-transitory" only means that the storage medium is tangible and does not include signals, but does not help to distinguish any data stored semi-permanently or temporarily in the storage medium. For example, a non-transitory storage medium can include a buffer that temporarily stores data.

[0262] In addition, a program according to the disclosed embodiments of the present specification can be provided in the form of a computer program product. The computer program product can be a commercial product that can be traded between a seller and a buyer.

[0263] The computer program product can include a software program and a computer-readable storage medium on which the software program is stored. For example, the computer program product can include a product in the form of a software program (e.g., a downloadable application), which is electronically distributed by the manufacturer of the X-ray imaging device or an electronic marketplace (e.g., Samsung Galaxy store®). For electronic distribution, at least a part of the software program can be stored in the storage medium or arbitrarily created. In this case, the storage medium can be either the server of the manufacturer of the X-ray imaging device 100 or a relay server that temporarily stores the software program.

[0264] The computer program product can include the storage medium of the server or the storage medium of the X-ray imaging device 100 included in the system of the X-ray imaging device 100 and / or the server. Alternatively, in the presence of a third device (e.g., the workstation 200 (see Figure 12 )) that is communicatively connected to the X-ray imaging device 100, the computer program product can include the storage medium of the third device. In another example, the computer program product can be transmitted from the X-ray imaging device 100 to the third device, or can include the software program itself transmitted from the third device to an electronic device.

[0265] In this case, one of the X-ray imaging device 100 or the third device (e.g., the workstation 200 (see Figure 12 )) can run the computer program product to execute the method according to the disclosed embodiments. Alternatively, at least one of the X-ray imaging device 100 and the third device can run the computer program product to execute the method according to the disclosed embodiments in a distributed manner.

[0266] For example, the X-ray imaging device 100 can run the computer program product stored in the memory 150 (see Figure 5 )) to control another electronic device communicatively connected to the X-ray imaging device 100 to execute the method according to the disclosed embodiments.

[0267] In another example, a third device may run a computer program product to control an electronic device communicatively connected to the third device to perform the method according to the disclosed embodiments.

[0268] In the case where the third device runs the computer program product, the third device may download the computer program product from the X-ray imaging device 100 and run the downloaded computer program product. Alternatively, the third device may run a pre-loaded computer program product to perform the method according to the disclosed embodiments.

[0269] Although the present disclosure has been described with reference to some embodiments and the accompanying drawings as described above, it will be apparent to those of ordinary skill in the art that various modifications and changes can be made to the embodiments. For example, the above methods may be performed in a different order, and / or the above components (such as computer systems or modules) may be combined in a form different from the above description, and / or the above components (such as computer systems or modules) may be replaced or substituted by other components or their equivalents, so as to obtain appropriate results.

Claims

1. An X-ray imaging device (100) for detecting the movement of an object, the X-ray imaging device (100) comprising: An X-ray irradiator (120) configured to generate X-rays and irradiate the X-rays onto the object; An X-ray detector (130) configured to detect X-rays irradiated by the X-ray irradiator (120) and transmitted through the object; A camera (110) configured to acquire an object image by photographing the object positioned in front of the X-ray detector (130); A display (172); and At least one processor (140) configured to: detect the movement of the object based on the object image by analyzing the object image using an artificial intelligence (AI) model, and output a notification signal on the display (172), the notification signal notifying the user of the result of the detection of the movement of the object.

2. The X-ray imaging device (100) according to claim 1, wherein, The at least one processor (140) is configured to: Acquire a reference image by photographing the object with the camera (110) when the object finishes positioning in front of the X-ray detector (130); After acquiring the reference image, acquire an image frame by subsequently photographing the object; And Detect the movement of the object by comparing the object identified from the reference image with the object identified from the image frame through analysis using the AI model.

3. The X-ray imaging device (100) according to claim 2, wherein, The at least one processor (140) is configured to: Acquire weights for pixels representing the object identified from the reference image by using a self-organizing map of the AI model; Detect the movement of the object by comparing the object identified from the image frame with the object identified from the reference image by using the weights; And Update the reference image and the weights by using the result of the detection.

4. The X-ray imaging device (100) according to claim 2, wherein, The at least one processor (140) is configured to: Extract a plurality of first key points of the characteristic landmark part of the object from the reference image by performing inference using a trained deep neural network model of the AI model; Calculate the difference between the key points by comparing the extracted plurality of first key points with a plurality of second key points of the object extracted from the image frame; And Detect the movement of the object by comparing the calculated difference with a preset threshold.

5. The X-ray imaging device (100) according to claim 4, wherein, The deep neural network model is a model trained by a supervised learning method that applies a plurality of acquired images as input data and applies the position coordinates of the key points of the characteristic landmark part as the ground truth.

6. The X-ray imaging device (100) according to any one of claims 1 to 5, further comprising: A depth measurement device (180) including at least one of a stereo type camera, a time-of-flight (ToF) camera, and a laser distance measurer, wherein the at least one processor (140) is configured to: Detect patient positioning by measuring the distance between the X-ray irradiator (120) and the object by using the depth measurement device (180).

7. The X-ray imaging device (100) according to any one of claims 1 to 6, wherein, The at least one processor (140) is configured to: Set the motion detection sensitivity based on at least one of the source-to-image distance (SID) which is the distance between the object and the X-ray irradiator (120), the size and shape of the object, and the imaging protocol.

8. The X-ray imaging device (100) according to any one of claims 1 to 7, wherein, The display (172) displays a graphical user interface (UI) with a preset color representing the motion of the object.

9. The X-ray imaging device (100) according to any one of claims 1 to 8, further comprising: A speaker (174) configured to output at least one acoustic signal among voice and notification sound, the at least one acoustic signal notifying the user of information about the motion of the object.

10. A method of operating an X-ray imaging device (100), the method comprising: Obtain image data of the object by photographing the object with a camera (110) (S610); Detect the motion of the object based on the image data by analyzing the image data using an artificial intelligence (AI) model (S620); and Output a notification signal notifying the user of the detection of the motion of the object (S630).

11. The method according to claim 10, wherein, Obtaining the image data (S610) includes: When the object completes positioning in front of the X-ray detector (130), obtaining a reference image by photographing the object using the camera (110); and After obtaining the reference image, obtaining image frames by subsequently photographing the object; Detecting the motion of the object includes: detecting the motion of the object by comparing the object identified from the reference image with the object identified from the image frame through analysis via using the AI model.

12. The method according to claim 11, wherein, Detecting the motion of the object (S620) includes: Obtaining weights from the reference image by using a self-organizing map (S810); Detecting the motion of the object by comparing the object identified from the image frame with the object identified from the reference image via using the weights (S820); and Updating the reference image and the weights by using the result of the detection (S830).

13. The method according to claim 11, wherein, Detecting the motion of the object (S620) includes: Extracting a plurality of first key points of the feature landmark part of the object from the reference image by performing inference using a trained deep neural network model (S910); Calculating the difference between the key points by comparing the extracted plurality of first key points with the plurality of second key points of the object extracted from the image frame (S920); and Detecting the motion of the object by comparing the calculated difference with a preset threshold.

14. The method according to any one of claims 10 to 13, further comprising: Set the motion detection sensitivity based on at least one of the source-to-image distance (SID) which is the distance between the object and the X-ray irradiator (120), the size and shape of the object, and the imaging protocol.

15. The method according to any one of claims 10 to 14, wherein, Outputting the notification signal (S630) includes: displaying a graphical user interface (UI) with a preset color representing the motion of the object.

Images