Image processing device

Abstract

The present application discloses an image processing method and an image processing apparatus. The image processing method includes extracting first coordinates of spinal features in an image pixel coordinate system from a back image; converting the first coordinates into second coordinates in the end coordinate system of a guiding device based on a preset transformation matrix; smoothing the trajectory of the spinal features based on the second coordinates of the spinal features to obtain a smooth three-dimensional spinal feature trajectory; acquiring spinal ultrasound images obtained by an ultrasound scanning device on the guiding device scanning along the three-dimensional spinal feature trajectory; identifying abnormal deformations of the spine from the spinal ultrasound images; drawing a minimum bounding rectangle for each abnormal deformation, calculating an angle between one of the upper and lower edges of the minimum bounding rectangle of each abnormal deformation and one of the upper and lower edges of the minimum bounding rectangle of other abnormal deformations, and taking out the largest angle. The present invention conveniently and effectively calculates the Cobb angle, facilitating medical personnel to judge scoliosis.

Description

Technical Field

[0001] The present application relates to an image processing method and an image processing apparatus. Background Art

[0002] Currently, there are mainly three general survey modes for scoliosis at home and abroad. The first is to directly use X-ray films for scoliosis detection and screening. This method is the most accurate one and basically there will be no missed diagnosis or misdiagnosis. However, the subjects will be exposed to X-ray radiation, especially in children and adolescents who need frequent monitoring, and the detection cost is relatively high. The second is to first use artificial physical methods for detection and then use X-ray films for diagnosis. This method adds physical detection. If the physical detection is positive, it can be regarded as an indication for X-ray examination. Although this method reduces unnecessary radiation, it is affected by factors such as the body fat percentage of the patient, the detection accuracy is poor, and misdiagnosis is likely to occur. The third is to use artificial physical methods, moiré photography, and X-ray films for detection in sequence. This method can screen out suspected scoliosis patients through artificial physical detection methods and moiré image detection, but its steps are numerous, time-consuming, and costly.

[0003] Therefore, the current methods for detecting scoliosis are not very satisfactory. Summary of the Invention

[0004] Embodiments of the present application provide an image processing method, an image processing apparatus, a computer-readable storage medium, and an electronic device.

[0005] In a first aspect, embodiments of the present application provide an image processing method for an electronic device, including:

[0006] An extraction step of extracting a first coordinate of a spinal feature in an image pixel coordinate system from a collected back image;

[0007] A conversion step of converting the first coordinate of the spinal feature into a second coordinate in the end coordinate system of a guiding device based on a preset transformation matrix;

[0008] A processing step of smoothing the trajectory of the spinal feature based on the second coordinate of the spinal feature to obtain a smooth three-dimensional spinal feature trajectory;

[0009] An acquisition step of acquiring a spinal ultrasound image obtained by an ultrasound scanning device on the guiding device scanning along the three-dimensional spinal feature trajectory;

[0010] An identification step of identifying an abnormal deformation of the spine from the spinal ultrasound image;

[0011] Calculation steps: draw the minimum circumscribed rectangle of each vertebral body, calculate the angle between one of the upper and lower edges of the minimum circumscribed rectangle of each abnormal deformation and one of the upper and lower edges of the minimum circumscribed rectangle of other abnormal deformations, and take out the largest angle.

[0012] In a second aspect, an embodiment of the present invention provides an image processing device, including:

[0013] An extraction unit that extracts the first coordinates of the spinal column features in the image pixel coordinate system from the acquired back image;

[0014] A conversion unit that converts the first coordinates of the spinal column features into second coordinates in the end coordinate system of the guiding device based on a preset transformation matrix;

[0015] A processing unit that smooths the trajectory of the spinal column features based on the second coordinates of the spinal column features to obtain a smooth three-dimensional spinal column feature trajectory;

[0016] An acquisition unit that acquires spinal column ultrasound images obtained by scanning the three-dimensional spinal column feature trajectory by an ultrasound scanning device on the guiding device;

[0017] An identification unit that identifies abnormal deformations of the spinal column from the spinal column ultrasound images;

[0018] A calculation unit that draws the minimum circumscribed rectangle of each abnormal deformation, calculates the angle between one of the upper and lower edges of the minimum circumscribed rectangle of each abnormal deformation and one of the upper and lower edges of the minimum circumscribed rectangle of other abnormal deformations, and takes out the largest angle.

[0019] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed on a computer, the computer executes the image processing method described in the first aspect.

[0020] In a fourth aspect, an embodiment of the present application provides an electronic device, including: one or more processors; one or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the image processing method described in the first aspect.

[0021] In the present invention, spinal features are extracted from a back image and converted into a three-dimensional spinal feature trajectory, enabling an ultrasonic scanning device to perform ultrasonic scanning imaging based on the three-dimensional spinal feature trajectory. Then, abnormal deformations are identified in the scanned spinal ultrasonic image, and the Cobb angle is calculated. Thus, the present invention facilitates and effectively calculates the Cobb angle through fully automated image recognition, making it convenient for medical staff to judge scoliosis and effectively solving the dependence on the experience of medical staff in scoliosis screening.

[0022] The present invention integrates technologies such as digital image processing, computer vision, and ultrasonic imaging, enabling radiation-free processing throughout the process and avoiding significant radiation damage to the human body.

[0023] In the traditional ultrasonic imaging method, medical staff hold the device by hand to scan the spinal position, and the hand tremors can cause significant errors in the ultrasonic imaging results, resulting in poor imaging quality and meaningless images. The present invention integrates the image pixel coordinate system and the end coordinate system of the robot, and the robot operation can be standardized, so accurate ultrasonic imaging of the human spine can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 According to an embodiment of the present application, a schematic diagram of an image processing method is shown;

[0025] Figure 2 According to an embodiment of the present application, a schematic diagram showing a guiding device controlling an ultrasonic scanning device to scan on the back of a human body is shown;

[0026] Figure 3 According to an embodiment of the present application, a structural diagram of an image processing device is shown;

[0027] Figure 4 According to an embodiment of the present application, a block diagram of an electronic device is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] Exemplary embodiments of the present application include, but are not limited to, an image processing method and an image processing device.

[0029] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0030] Figure 1 A flowchart of an image processing method according to an embodiment of the present application is shown. It can be understood that the image processing method of the present invention can be implemented in the Figure 4 electronic device shown. In the extraction step S101, the first coordinates of the spinal features in the image pixel coordinate system are extracted from the collected back image.

[0031] The back image is a two-dimensional or three-dimensional image acquired by an imaging device. The imaging device is, for example, a high-resolution two-dimensional camera. The back image of the naked back of the human body is obtained using the two-dimensional camera. This back image is a two-dimensional image. Then, a target detection model in deep learning is used to extract the back spine region from the back image, and digital image processing techniques such as image enhancement, binarization, and morphological analysis are employed to extract the first coordinate P of the spine feature in the image pixel coordinate system. p . It can be understood that here, the first coordinate P of the spine feature p is a two-dimensional coordinate.

[0032] The imaging device can also be, for example, a three-dimensional laser scanner. The back image of the naked back of the human body is obtained using the three-dimensional laser scanner. This back image is three-dimensional data, that is, three-dimensional point cloud data. Then, the three-dimensional point cloud data is subjected to point cloud segmentation to extract the back spine region, and the first coordinate P of the spine feature in the image pixel coordinate system is extracted through algorithms such as spine curvature and region growing. c . It can be understood that here, the first coordinate P of the spine feature c is a three-dimensional coordinate.

[0033] In transformation step S102, based on a preset transformation matrix, the first coordinate of the spine feature is transformed into a second coordinate in the end coordinate system of the guiding device.

[0034] Figure 2 Figure 18 shows the guiding device 20. The guiding device 20 can be a collaborative robot or a robotic arm, etc. An ultrasonic scanning device 21 is installed at the end of the guiding device 20. The guiding device 20 can control the ultrasonic scanning device 21 to scan on the back of the human body. The ultrasonic scanning device 21 is, for example, an ultrasonic scanner.

[0035] When the back image is a two-dimensional image, the preset transformation matrix is represented by formula (1):

[0036] Formula (1)

[0037] where, P p is the two-dimensional first coordinate of the spine feature, P e is the three-dimensional second coordinate of the spine feature, K -1 is the inverse matrix of the internal parameter matrix of the imaging device, T cw is the transformation matrix between the coordinate system of the imaging device and the world coordinate system, T we is the transformation matrix between the world coordinate system and the end coordinate system of the guiding device, and s is the transformation scale coefficient.

[0038] It can be understood that through the preset transformation matrix shown in formula (1), the two-dimensional first coordinate P of the spine feature in the image pixel coordinate systemp Convert the three-dimensional second coordinate P of the spine feature to the end coordinate system of the guiding device e .

[0039] In addition, when the back image is a three-dimensional image, the preset transformation matrix is represented by formula (2):

[0040] Formula (2)

[0041] Where P c is the three-dimensional first coordinate of the spine feature, P e is the three-dimensional second coordinate of the spine feature, T cw is the transformation matrix between the coordinate system of the imaging device and the world coordinate system, and T we is the transformation matrix between the world coordinate system and the end coordinate system of the guiding device.

[0042] It can be understood that through the preset transformation matrix shown in formula (2), the three-dimensional first coordinate P of the spine feature in the image pixel coordinate system c can be converted to the three-dimensional second coordinate P of the spine feature in the end coordinate system of the guiding device e .

[0043] In processing step 103, based on the second coordinate of the spine feature, smooth the trajectory of the spine feature to obtain a smooth three-dimensional spine feature trajectory.

[0044] Specifically, perform a least squares fitting on the spine feature in the end coordinate system of the guiding device 20 obtained as described above to achieve smoothing and obtain a smooth three-dimensional spine feature trajectory, that is, fit it into a smooth curve. The smooth three-dimensional spine feature trajectory will be sent to the controller (not shown in the figure) in the guiding device 20.

[0045] In acquisition step 104, acquire the spine ultrasound image obtained by the ultrasound scanning device on the guiding device scanning along the three-dimensional spine feature trajectory.

[0046] After the controller in the guiding device 20 receives the three-dimensional spine feature trajectory, it will control (guide) the ultrasound scanning device 21 to scan along the three-dimensional spine feature trajectory on the back of the human body, so that the ultrasound scanning device 21 can correspond to the position of the spine feature, as Figure 2 shown.

[0047] Specifically, the guiding device 20 will combine the coordinate position (i.e., pose information) of the human body in the world coordinate system obtained from the back image to guide the ultrasound scanning device 21 to scan along the three-dimensional spine feature trajectory on the back of the human body.

[0048] It can be understood that the coordinate position of the human body in the world coordinate system is obtained from the collected back image. Specifically, when the back image is a two-dimensional image, the coordinate position of the back image of the human body in the image pixel coordinate system is converted into the coordinate position of the human body in the world coordinate system and transmitted to the controller of the guiding device 20 in real time. When the back image is a three-dimensional image, the coordinate position of the human body in the world coordinate system can be directly obtained and transmitted to the controller of the guiding device 20 in real time.

[0049] During the scanning process of the ultrasonic scanning device 21, according to the coordinate position of the end of the guiding device 20, the ultrasonic image scanned by the ultrasonic scanning device is corrected to obtain a spinal ultrasonic image.

[0050] It can be understood that during the scanning process of the ultrasonic scanning device 21, the coordinate position of the end of the guiding device 20 is recorded in real time.

[0051] After the scanning is completed according to the spinal feature trajectory, the ultrasonic image can be corrected according to the running trajectory of the end of the guiding device 20 (the coordinate position of the end coordinate system of the guiding device 20), so as to obtain an accurate three-dimensional spinal ultrasonic image. It can be understood that when correcting the ultrasonic image, combining the position and attitude information of the end of the guiding device 20 during scanning and the scanned ultrasonic image, the pixel points in the two-dimensional ultrasonic image are converted into three-dimensional space coordinates, and the ultrasonic image is corrected three-dimensionally to obtain a spinal ultrasonic image. In this way, the ultrasonic measurement error caused by jitter and contact can be eliminated.

[0052] Recognition step S105: Identify abnormal deformations of the spine from the spinal ultrasonic image.

[0053] Specifically, after preprocessing the spinal ultrasonic image, image segmentation is performed, and abnormal deformations are identified from the segmented image.

[0054] Preprocessing the spinal ultrasonic image includes operations such as denoising, contrast enhancement, and image normalization to improve the clarity and contrast of the image. Then, for the preprocessed spinal ultrasonic image, image segmentation techniques such as threshold segmentation, edge detection, and region growing are used to separate different structures of the spine (including abnormal deformations, etc.) from the background.

[0055] Identify abnormal deformations from the segmented image. The abnormal deformations are, for example, obvious protruding structures in the image, and the abnormal deformations are identified by detecting these protruding parts in the three-dimensional data. In addition, the abnormal deformations are, for example, the protruding parts on the side of the spine, and the position of the abnormal deformations can be identified by analyzing the cross-section or oblique section in the three-dimensional data. It is also possible to use a deep learning object recognition model to identify abnormal deformations from the segmented spinal ultrasonic image.

[0056] In calculation step S106, the minimum bounding rectangle of each abnormal deformation is drawn, and the angle between one of the upper and lower edges of the minimum bounding rectangle of each abnormal deformation and one of the upper and lower edges of the minimum bounding rectangle of other abnormal deformations is calculated, and the largest angle is taken.

[0057] Specifically, for each abnormal deformation, the minimum bounding rectangle is drawn, the angle between the upper or lower edge of the minimum bounding rectangle of each abnormal deformation and the upper or lower edge of the minimum bounding rectangle of other abnormal deformations is calculated, and the largest angle is used as the Cobb angle. It can be understood that the upper and lower edges of the minimum bounding rectangle are two parallel sides.

[0058] It can be understood that the Cobb angle is a method for measuring the lateral bending angle and is used to evaluate the severity of scoliosis. The X-ray film used for measurement is the anteroposterior view of the full length of the spine. The measurement method is usually: on the anteroposterior X-ray film of the spine, first draw a horizontal line at the upper edge of the upper end vertebra, then draw another horizontal line along the lower edge of the lower end vertebra, and finally draw a vertical line of these two horizontal lines. The intersection angle of the two vertical lines is the Cobb angle, representing the degree of spinal scoliosis. Usually, if the Cobb angle is within 10°, it indicates normal; if it is greater than 10°, it indicates scoliosis. It can be understood that medical staff can determine whether there is scoliosis in the spine and the degree of spinal scoliosis based on the Cobb angle.

[0059] In the present invention, spine features are extracted from the back image and transformed into a three-dimensional spine feature trajectory, so that the ultrasonic scanning device performs ultrasonic scanning imaging according to the three-dimensional spine feature trajectory, and then abnormal deformations in the scanned spine ultrasonic image are identified, thereby calculating the Cobb angle. In this way, the present invention realizes fully automated image recognition, conveniently and effectively calculates the Cobb angle, facilitates medical staff to judge scoliosis, and effectively solves the dependence on the experience of medical staff for scoliosis screening.

[0060] The present invention integrates technologies such as digital image processing technology, computer vision technology, and ultrasonic imaging, and can achieve radiation-free processing throughout the process, avoiding large radiation damage to the human body.

[0061] The traditional ultrasonic imaging method uses the hand-held method by medical staff to scan the spine position. The shaking of the hand will cause large errors in the ultrasonic imaging results, resulting in poor imaging quality and the image having no reference significance. The present invention integrates the image pixel coordinate system and the end coordinate system of the robot, and the robot operation can be standardized, so accurate ultrasonic imaging of the human spine can be performed.

[0062] The present invention also provides an image processing device 30, as Figure 3As shown in the figure, the image processing device 30 includes: an extraction unit 301 that acquires the captured back image and extracts the first coordinates of the spine feature in the image pixel coordinate system from the back image; a conversion unit 302 that converts the first coordinates of the spine feature into the second coordinates in the end coordinate system of the guiding device based on a preset transformation matrix; a processing unit 303 that smoothes the trajectory of the spine feature based on the second coordinates of the spine feature to obtain a smooth three-dimensional spine feature trajectory; an acquisition unit 304 that acquires the spine ultrasound image obtained by the ultrasound scanning device on the guiding device scanning along the three-dimensional spine feature trajectory; an identification unit 305 that identifies the abnormal deformation of the spine from the spine ultrasound image; and a calculation unit 306 that draws the minimum bounding rectangle of each abnormal deformation, calculates the angle between one of the upper and lower edges of the minimum bounding rectangle of each abnormal deformation and one of the upper and lower edges of the minimum bounding rectangle of other abnormal deformations, and selects the largest angle.

[0063] It can be understood that the extraction unit 301, the conversion unit 302, the processing unit 303, the acquisition unit 304, the identification unit 305, and the calculation unit 306 can be implemented by the processor 102 having the functions of these modules or units in the electronic device 100. The implementation manners disclosed in the foregoing are method implementation manners corresponding to this implementation manner, and this implementation manner can be implemented in cooperation with the above implementation manners. The relevant technical details mentioned in the above implementation manners are still valid in this implementation manner. To avoid repetition, they are not described herein again. Correspondingly, the relevant technical details mentioned in this implementation manner can also be applied to the above implementation manners.

[0064] Now refer to Figure 4 , Figure 4 FIG. schematically shows an example electronic device 1400 according to an embodiment of the present invention. In one embodiment, the electronic device 1400 may include one or more processors 1404, a system control logic unit 1408 connected to at least one of the processors 1404, a system memory 1412 connected to the system control logic unit 1408, a non-volatile memory (NVM) 1416 connected to the system control logic unit 1408, and a network interface 1420 connected to the system control logic unit 1408.

[0065] In some embodiments, the processor 1404 may include one or more single-core or multi-core processors. In some embodiments, the processor 1404 may include any combination of a general-purpose processor and a dedicated processor (e.g., a graphics processor, an application processor, a baseband processor, etc.). In embodiments where the electronic device 1400 employs an eNB (Evolved Node B) or a RAN (Radio Access Network) controller, the processor 1404 may be configured to execute various conforming embodiments, e.g., the embodiments as Figure 1 shown.

[0066] In some embodiments, the system control logic unit 1408 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1404 and / or any suitable device or component communicating with the system control logic unit 1408.

[0067] In some embodiments, the system control logic unit 1408 may include one or more memory controllers to provide an interface to the system memory 1412. The system memory 1412 may be used to load and store data and / or instructions. In some embodiments, the system memory 1412 of the electronic device 1400 may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

[0068] The non-volatile memory 1416 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 1416 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a HDD (Hard Disk Drive), a CD (Compact Disc) drive, and a DVD (Digital Versatile Disc) drive.

[0069] The non-volatile memory 1416 may include a part of the storage resources installed on the device of the electronic device 1400, or it may be accessible by the electronic device but not necessarily part of the electronic device. For example, the non-volatile memory 1416 may be accessed via the network interface 1420 through a network.

[0070] Specifically, the system memory 1412 and the non-volatile memory 1416 may respectively include a temporary copy and a permanent copy of the instructions 1424. The instructions 1424 may include: when executed by at least one of the processors 1404, causing the electronic device 1400 to implement as Figure 1Instructions of the method shown. In some embodiments, instructions 1424, hardware, firmware, and / or their software components may alternatively / additionally be placed in system control logic unit 1408, network interface 1420, and / or processor 1404.

[0071] Network interface 1420 may include a transceiver for providing a radio interface for electronic device 1400 to communicate with any other suitable device (such as a front-end module, antenna, etc.) via one or more networks. In some embodiments, network interface 1420 may be integrated with other components of electronic device 1400. For example, network interface 1420 may be integrated with at least one of processor 1404, system memory 1412, non-volatile memory 1416, and a firmware device with instructions (not shown). When at least one of the instructions is executed by processor 1404, electronic device 1400 implements the method as Figure 1 shown.

[0072] Network interface 1420 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, network interface 1420 may be a network adapter, wireless network adapter, telephone modem, and / or wireless modem.

[0073] In one embodiment, at least one of processors 1404 may be packaged with the logic of one or more controllers for system control logic unit 1408 to form a system-in-package (SiP). In one embodiment, at least one of processors 1404 may be integrated with the logic of one or more controllers for system control logic unit 1408 on the same die to form a system-on-chip (SoC).

[0074] Electronic device 1400 may further include: an input / output (I / O) device 1432. I / O device 1432 may include a user interface that enables a user to interact with electronic device 1400; the design of the peripheral component interface enables peripheral components to also interact with electronic device 1400. In some embodiments, electronic device 1400 further includes sensors for determining at least one of environmental conditions and location information related to electronic device 1400.

[0075] In some embodiments, the user interface may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., a light-emitting diode flash), and a keyboard.

[0076] Embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as a computer program or program code executed on a programmable system that includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0077] The program code can be applied to input instructions to perform the various functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0078] The program code can be implemented in a high-level procedural language or an object-oriented programming language in order to communicate with the processing system. When needed, the program code can also be implemented in assembly language or machine language. In fact, the mechanisms described in this application are not limited to the scope of any particular programming language. In any case, the language can be a compiled language or an interpreted language.

[0079] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments can also be implemented as instructions carried or stored on one or more transient or non-transient machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, the instructions can be distributed via a network or via other computer-readable media. Thus, a machine-readable medium can include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to, a floppy disk, a compact disc, a CD-ROM, a magneto-optical disc, a ROM, a RAM, an EPROM, an EEPROM, a magnetic or optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) in electrical, optical, acoustic, or other forms using the Internet. Thus, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0080] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or ordering may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Additionally, the inclusion of a structural or method feature in a particular figure does not imply that such a feature is required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0081] It should be noted that each unit / module mentioned in the device embodiments of this application is a logical unit / module. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or can be implemented as a combination of multiple physical units / module. The physical implementation manner of these logical units / module themselves is not the most important. The combination of the functions implemented by these logical units / module is the key to solving the technical problems proposed in this application. In addition, to highlight the innovative part of this application, the above device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that there are no other units / modules in the above device embodiments.

[0082] It should be noted that in the examples and the specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one" does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0083] Although this application has been illustrated and described by reference to certain preferred embodiments thereof, those of ordinary skill in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. An image processing device, characterized in that: include: An extraction unit extracts a first coordinate of a spine feature in an image pixel coordinate system from the acquired back image; A conversion unit, based on a preset transformation matrix, converts the first coordinate of the spinal feature into a second coordinate in a terminal coordinate system of the guiding device; A processing unit, based on the second coordinate of the spinal column feature, performs smoothing processing on the trajectory of the spinal column feature to obtain a smooth three-dimensional spinal column feature trajectory; An acquisition unit, which acquires an ultrasonic image of the spine scanned by the ultrasonic scanning device on the guiding device along the three-dimensional spine characteristic trajectory; an identification unit, which identifies abnormal deformation of the spine from the spine ultrasound image; The calculation unit draws the minimum circumscribed rectangle of each abnormal deformation, calculates the angle between one of the upper edge and the lower edge of each minimum circumscribed rectangle of the abnormal deformation and one of the upper edge and the lower edge of the minimum circumscribed rectangle of other abnormal deformations, and takes the largest angle. The back image is a two-dimensional image or a three-dimensional image acquired by an imaging device. When the back image is a two-dimensional image, the preset transformation matrix is ​​expressed by formula (1): Formula (1) Among them, P p is the first coordinate of the two-dimensional spine feature, P e is the second coordinate of the three-dimensional spine feature, K -1 is the inverse matrix of the intrinsic parameter matrix of the imaging device, T cw is the transformation matrix between the coordinate system of the imaging device and the world coordinate system, T we is the transformation matrix between the world coordinate system and the terminal coordinate system of the guiding device, s is the transformation scale coefficient, When the back image is a three-dimensional image, the preset transformation matrix is ​​expressed by formula (2): Formula (2) Among them, P c is the first coordinate of the three-dimensional spine feature, P e is the second coordinate of the three-dimensional spine feature, T cw is the transformation matrix between the coordinate system of the imaging device and the world coordinate system, T we is the transformation matrix between the world coordinate system and the terminal coordinate system of the guiding device.

2. The image processing device according to claim 1, characterized in that The ultrasonic scanning device is installed at the end of the guiding device, and the guiding device controls the ultrasonic scanning device to scan along the three-dimensional spine characteristic trajectory. Wherein, the guiding device is a collaborative robot or a robotic arm.

3. The image processing device according to claim 2, characterized in that: The ultrasonic image scanned by the ultrasonic scanning device is corrected according to the coordinate position of the end of the guiding device to obtain the spinal ultrasound image.

4. The image processing device according to claim 1, characterized in that: In the recognition unit, the spinal ultrasound image is preprocessed, then image segmentation is performed, and the abnormal deformation is identified from the segmented image.

Images