Image processing method and image processing device

Abstract

The invention discloses an image processing method and an image processing device. The image processing method comprises the following steps: extracting a first coordinate of a spine feature in an image pixel coordinate system from a back image; based on a preset transformation matrix, converting the first coordinate into a second coordinate under a tail end coordinate system of the guide equipment; on the basis of the second coordinates of the spine features, smoothing the track of the spine features to obtain a smooth three-dimensional spine feature track; acquiring a spinal ultrasonic image scanned by an ultrasonic scanning device on the guide device along the three-dimensional spinal feature trajectory; abnormal deformation of the spine is recognized from the spine ultrasonic image; and drawing a minimum enclosing rectangle of each abnormal deformation, calculating an included angle between one of the upper edge and the lower edge of the minimum enclosing rectangle of each abnormal deformation and one of the upper edge and the lower edge of the minimum enclosing rectangle of other abnormal deformation, and taking out a maximum included angle. According to the method, the Cobb angle can be conveniently and effectively calculated, and medical personnel can conveniently judge scoliosis.

Description

Technical Field

[0001] The present application relates to an image processing method and an image processing device. Background Art

[0002] At present, there are three main scoliosis screening modes at home and abroad. The first is to directly use X-rays for scoliosis detection and screening. This method is the most accurate method and basically will not result in missed diagnosis or misdiagnosis. However, the examinee will be exposed to X-ray radiation, especially in children and adolescents who need frequent monitoring, and the detection cost is relatively high. The second is to use artificial physical methods to detect first and then use X-rays to confirm the diagnosis. This method adds physical method detection. If the physical test is positive, it can be regarded as an indication for X-ray examination. Although this method reduces unnecessary radiation, it is affected by the patient's body fat rate, etc., and the detection accuracy is poor, which is easy to cause misdiagnosis. The third is to use artificial physical methods, moiré photography, and X-rays for detection in succession. This method can screen out suspected scoliosis patients through artificial physical detection methods and moiré image detection, but it has many steps, takes a long time, and is expensive.

[0003] Therefore, current methods of detecting scoliosis are less than ideal. Summary of the invention

[0004] Embodiments of the present application provide an image processing method, an image processing device, a computer-readable storage medium, and an electronic device.

[0005] In a first aspect, an embodiment of the present application provides an image processing method for an electronic device, comprising: An extraction step of extracting the first coordinate of the spine feature in the image pixel coordinate system from the acquired back image; A conversion step, based on a preset transformation matrix, converting the first coordinates of the spinal feature into second coordinates in the terminal coordinate system of the guiding device; A processing step, based on the second coordinate of the spinal column feature, smoothing the trajectory of the spinal column feature to obtain a smooth three-dimensional spinal column feature trajectory; An acquisition step, acquiring an ultrasonic image of the spine scanned by an ultrasonic scanning device on the guiding device along the three-dimensional spine characteristic trajectory; an identification step of identifying abnormal deformation of the spine from the spine ultrasound image; The calculation step is to draw the minimum circumscribed rectangle of each vertebra, and calculate the angle between one of the upper edge and the lower edge of each abnormally deformed minimum circumscribed rectangle and one of the upper edge and the lower edge of other abnormally deformed minimum circumscribed rectangles, and take the largest angle.

[0006] In a second aspect, an embodiment of the present invention provides an image processing device, including: An extraction unit extracts a first coordinate of a spine feature in an image pixel coordinate system from the acquired back image; A conversion unit, based on a preset transformation matrix, converts the first coordinate of the spinal feature into a second coordinate in a terminal coordinate system of the guiding device; A processing unit, based on the second coordinate of the spinal column feature, performs smoothing processing on the trajectory of the spinal column feature to obtain a smooth three-dimensional spinal column feature trajectory; An acquisition unit, which acquires an ultrasonic image of the spine scanned by the ultrasonic scanning device on the guiding device along the three-dimensional spine characteristic trajectory; an identification unit, which identifies abnormal deformation of the spine from the spine ultrasound image; The calculation unit draws the minimum circumscribed rectangle of each abnormal deformation, calculates the angle between one of the upper edge and the lower edge of each minimum circumscribed rectangle of the abnormal deformation and one of the upper edge and the lower edge of other minimum circumscribed rectangles of the abnormal deformation, and takes the largest angle.

[0007] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed on a computer, the computer executes the image processing method described in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; one or more memories; the one or more memories storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the image processing method described in the first aspect.

[0009] In the present invention, the back image is used to extract the spinal features, and the spinal features are converted into a three-dimensional spinal feature trajectory, so that the ultrasonic scanning device performs ultrasonic scanning imaging according to the three-dimensional spinal feature trajectory, and then the abnormal deformation of the scanned spinal ultrasound image is identified, thereby calculating the Cobb angle. In this way, the present invention can conveniently and effectively calculate the Cobb angle by realizing full-process automated image recognition, which is convenient for medical personnel to judge scoliosis and effectively solves the reliance of scoliosis screening on the experience of medical personnel.

[0010] The present invention integrates digital image processing technology, computer vision technology, ultrasonic imaging technology and other technologies, and can achieve radiation-free processing throughout the entire process, thus avoiding greater radiation damage to the human body.

[0011] Traditional ultrasound imaging uses medical staff to scan the spine in a handheld manner. The shaking of the hand will cause a large error in the ultrasound imaging results, resulting in poor imaging quality and no reference value for the image. The present invention combines the image pixel coordinate system and the robot's terminal coordinate system, and the robot operation can be standardized, so it can accurately perform ultrasound imaging of the human spine. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 According to an embodiment of the present application, a schematic diagram of an image processing method is shown; Figure 2 According to an embodiment of the present application, a schematic diagram is shown in which a guiding device controls an ultrasonic scanning device to scan the back of a human body; Figure 3 According to an embodiment of the present application, a structural diagram of an image processing device is shown; Figure 4 According to an embodiment of the present application, a block diagram of an electronic device is shown. DETAILED DESCRIPTION

[0013] The illustrative embodiments of the present application include, but are not limited to, an image processing method and an image processing device.

[0014] The embodiments of the present application will be described in further detail below in conjunction with the accompanying drawings.

[0015] Figure 1 FIG. 1 is a flowchart of an image processing method according to an embodiment of the present application. It can be understood that the image processing method of the present invention can be used in Figure 4 In the extraction step S101, the first coordinate of the spine feature in the image pixel coordinate system is extracted from the acquired back image.

[0016] The back image is a two-dimensional image or a three-dimensional image collected by an imaging device. The imaging device, for example, uses a high-resolution two-dimensional camera to obtain a back image of the naked back of a human body. The back image is a two-dimensional image. Then, the target detection model in deep learning is used to extract the back spine area from the back image. Then, digital image processing techniques such as image enhancement, binarization, and morphological analysis are used to extract the first coordinate P of the spine feature in the image pixel coordinate system. p It can be understood that here, the first coordinate P of the spine feature p are two-dimensional coordinates.

[0017] The imaging device may also be a 3D laser scanner, for example, which is used to obtain a back image of the naked back of a human body. The back image is 3D data, that is, 3D point cloud data. The 3D point cloud data is then segmented to extract the back spine region. The first coordinate P of the spine feature in the image pixel coordinate system is then extracted by using algorithms such as spine curvature and region growing. c It can be understood that here, the first coordinate P of the spine feature c are three-dimensional coordinates.

[0018] In the conversion step S102, based on a preset transformation matrix, the first coordinates of the spinal feature are converted into second coordinates in the terminal coordinate system of the guiding device.

[0019] Figure 2 The guide device 20 is shown, and the guide device 20 may be a collaborative robot or a robot arm, etc. An ultrasonic scanning device 21 is installed at the end of the guide device 20, and the guide device 20 may control the ultrasonic scanning device 21 to scan the back of the human body. The ultrasonic scanning device 21 is, for example, an ultrasonic scanner.

[0020] When the back image is a two-dimensional image, the preset transformation matrix is ​​expressed by formula (1): Formula (1) Among them, P p is the first coordinate of the 2D spine feature, P e is the second coordinate of the three-dimensional spine feature, K -1 is the inverse matrix of the intrinsic parameter matrix of the imaging device, T cw is the transformation matrix between the imaging device coordinate system and the world coordinate system, T we is the transformation matrix between the world coordinate system and the end coordinate system of the guiding device, and s is the transformation scale coefficient.

[0021] It can be understood that the first two-dimensional coordinate P of the spine feature in the image pixel coordinate system can be transformed into p The second three-dimensional coordinate P converted to the terminal coordinate system of the guiding device e .

[0022] In addition, when the back image is a three-dimensional image, the preset transformation matrix is ​​expressed by formula (2): Formula (2) Among them, P c is the first coordinate of the three-dimensional spine feature, P e is the second coordinate of the three-dimensional spine feature, T cw is the transformation matrix between the imaging device coordinate system and the world coordinate system, T weIt is the transformation matrix between the world coordinate system and the end coordinate system of the guiding device.

[0023] It can be understood that the first three-dimensional coordinate P of the spine feature in the image pixel coordinate system can be transformed into c The second three-dimensional coordinate P converted to the terminal coordinate system of the guiding device e .

[0024] In processing step 103, the trajectory of the spine feature is smoothed based on the second coordinate of the spine feature to obtain a smooth three-dimensional spine feature trajectory.

[0025] Specifically, the spinal features in the terminal coordinate system of the guiding device 20 obtained as described above are fitted by the least square method to achieve smoothing and obtain a smooth three-dimensional spinal feature trajectory, that is, fit into a smooth curve. The smooth three-dimensional spinal feature trajectory will be sent to the controller in the guiding device 20 (not shown).

[0026] In the acquisition step 104, an ultrasonic image of the spine scanned by the ultrasonic scanning device on the guiding device along the three-dimensional spine characteristic trajectory is acquired.

[0027] After receiving the three-dimensional spine feature trajectory, the controller in the guidance device 20 controls (guides) the ultrasound scanning device 21 to scan along the three-dimensional spine feature trajectory on the back of the human body, so that the ultrasound scanning device 21 can correspond to the position of the spine feature, such as Figure 2 shown.

[0028] Specifically, the guiding device 20 will guide the ultrasonic scanning device 21 to scan along the three-dimensional spine feature trajectory on the back of the human body in combination with the coordinate position of the human body in the world coordinate system (ie, posture information) obtained from the back image.

[0029] It is understandable that the coordinate position of the human body in the world coordinate system is obtained from the captured back image. Specifically, when the back image is a two-dimensional image, the coordinate position of the back image of the human body in the image pixel coordinate system is converted to the coordinate position of the human body in the world coordinate system, and transmitted to the controller of the guidance device 20 in real time. When the back image is a three-dimensional image, the coordinate position of the human body in the world coordinate system can be directly obtained and transmitted to the controller of the guidance device 20 in real time.

[0030] During the scanning process of the ultrasonic scanning device 21, the ultrasonic image scanned by the ultrasonic scanning device is corrected according to the coordinate position of the end of the guiding device 20 to obtain a spinal ultrasonic image.

[0031] It can be understood that during the scanning process of the ultrasonic scanning device 21, the coordinate position of the end of the guiding device 20 is recorded in real time.

[0032] After scanning is completed according to the characteristic trajectory of the spine, the ultrasound image can be corrected according to the running trajectory of the end of the guiding device 20 (the coordinate position of the coordinate system of the end of the guiding device 20), so as to obtain an accurate three-dimensional ultrasound image of the spine. It can be understood that when correcting the ultrasound image, the position and posture information of the end of the guiding device 20 during scanning and the scanned ultrasound image are combined, and the pixel points in the two-dimensional ultrasound image are converted to three-dimensional space coordinates, and the ultrasound image is corrected in three dimensions to obtain the ultrasound image of the spine. In this way, the ultrasound measurement error caused by jitter and contact can be eliminated.

[0033] In the identification step S105, abnormal deformation of the spine is identified from the spine ultrasound image.

[0034] Specifically, after the spinal ultrasound image is preprocessed, image segmentation is performed, and abnormal deformation is identified from the segmented image.

[0035] The spinal ultrasound image is preprocessed by operations such as denoising, contrast enhancement, and image standardization to improve the clarity and contrast of the image. Then, the preprocessed spinal ultrasound image is subjected to image segmentation techniques using threshold segmentation, edge detection, and region growing to separate different structures of the spine (including abnormal deformation, etc.) from the background.

[0036] Abnormal deformation is identified from the segmented image. For example, abnormal deformation is manifested as obvious protruding structures in the image. The abnormal deformation is identified by detecting these protruding parts in the three-dimensional data. In addition, abnormal deformation is, for example, a lateral protrusion of the spine. The location of the abnormal deformation can be identified in the three-dimensional data by analyzing the cross section or oblique section. The abnormal deformation can also be identified from the segmented spinal ultrasound image through a deep learning target recognition model.

[0037] In calculation step S106, the minimum circumscribed rectangle of each abnormal deformation is drawn, and the angle between one of the upper edge and the lower edge of the minimum circumscribed rectangle of each abnormal deformation and one of the upper edge and the lower edge of the minimum circumscribed rectangle of other abnormal deformations is calculated, and the largest angle is obtained.

[0038] Specifically, for each abnormal deformation, a minimum bounding rectangle is drawn, and the angle between the upper edge or lower edge of the minimum bounding rectangle of each abnormal deformation and the upper edge or lower edge of the minimum bounding rectangle of other abnormal deformations is calculated, and the largest angle is taken as the cobb angle. It can be understood that the upper edge and the lower edge of the minimum bounding rectangle are two parallel sides.

[0039] It is understandable that the Cobb angle is a method of measuring the angle of lateral curvature, which is used to assess the severity of scoliosis. The X-ray used for measurement is the standard full-length anteroposterior phase of the spine. The measurement method is usually: on the anteroposterior spinal X-ray, first draw a horizontal line at the upper edge of the upper cone, then draw another horizontal line along the lower edge of the lower vertebral body, and finally draw a vertical line between the two horizontal lines. The intersection of the two vertical lines is the Cobb angle, which represents the degree of scoliosis. Usually, if the Cobb angle is within 10°, it is normal; if it is greater than 10°, it means scoliosis. It is understandable that medical personnel can determine whether there is scoliosis and the degree of scoliosis of the spine based on the Cobb angle.

[0040] In the present invention, the back image is used to extract the spinal features, and the spinal features are converted into a three-dimensional spinal feature trajectory, so that the ultrasonic scanning device performs ultrasonic scanning imaging according to the three-dimensional spinal feature trajectory, and then the abnormal deformation of the scanned spinal ultrasound image is identified, thereby calculating the Cobb angle. In this way, the present invention can conveniently and effectively calculate the Cobb angle by realizing full-process automated image recognition, which is convenient for medical personnel to judge scoliosis and effectively solves the reliance of scoliosis screening on the experience of medical personnel.

[0041] The present invention integrates digital image processing technology, computer vision technology, ultrasonic imaging technology and other technologies, and can achieve radiation-free processing throughout the entire process, thus avoiding greater radiation damage to the human body.

[0042] Traditional ultrasound imaging uses medical staff to scan the spine in a handheld manner. The shaking of the hand will cause a large error in the ultrasound imaging results, resulting in poor imaging quality and no reference value for the image. The present invention combines the image pixel coordinate system and the robot's terminal coordinate system, and the robot operation can be standardized, so it can accurately perform ultrasound imaging of the human spine.

[0043] The present invention also provides an image processing device 30, such as Figure 3As shown, the image processing device 30 includes: an extraction unit 301, which obtains the collected back image and extracts the first coordinate of the spinal feature in the image pixel coordinate system from the back image; a conversion unit 302, which converts the first coordinate of the spinal feature into the second coordinate in the terminal coordinate system of the guiding device based on a preset transformation matrix; a processing unit 303, which smoothes the trajectory of the spinal feature based on the second coordinate of the spinal feature to obtain a smooth three-dimensional spinal feature trajectory; an acquisition unit 304, which obtains a spinal ultrasound image scanned by the ultrasound scanning device on the guiding device along the three-dimensional spinal feature trajectory; an identification unit 305, which identifies abnormal deformation of the spine from the spinal ultrasound image; a calculation unit 306, which draws the minimum circumscribed rectangle of each abnormal deformation, and calculates the angle between the upper edge and the lower edge of the minimum circumscribed rectangle of each abnormal deformation and the upper edge and the lower edge of the minimum circumscribed rectangle of other abnormal deformations, and takes the largest angle.

[0044] It is understandable that the extraction unit 301, the conversion unit 302, the processing unit 303, the acquisition unit 304, the identification unit 305, and the calculation unit 306 can be implemented by the processor 102 having the functions of these modules or units in the electronic device 100. The implementation methods disclosed in the foregoing are method implementation methods corresponding to the present implementation method, and the present implementation method can be implemented in conjunction with the above-mentioned implementation methods. The relevant technical details mentioned in the above-mentioned implementation methods are still valid in the present implementation methods, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in the present implementation methods can also be applied in the above-mentioned implementation methods.

[0045] Reference now Figure 4 , Figure 4 An example electronic device 1400 according to an embodiment of the present invention is schematically shown. In one embodiment, the electronic device 1400 may include one or more processors 1404, a system control logic unit 1408 connected to at least one of the processors 1404, a system memory 1412 connected to the system control logic unit 1408, a non-volatile memory (NVM) 1416 connected to the system control logic unit 1408, and a network interface 1420 connected to the system control logic unit 1408.

[0046] In some embodiments, the processor 1404 may include one or more single-core or multi-core processors. In some embodiments, the processor 1404 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments where the electronic device 1400 employs an eNB (Evolved Node B, enhanced base station) or RAN (Radio Access Network, radio access network) controller, the processor 1404 may be configured to execute various embodiments, such as Figure 1 The embodiment shown.

[0047] In some embodiments, system control logic unit 1408 may include any suitable interface controller to provide any suitable interface to at least one of processors 1404 and / or any suitable device or component in communication with system control logic unit 1408 .

[0048] In some embodiments, the system control logic unit 1408 may include one or more memory controllers to provide an interface to the system memory 1412. The system memory 1412 may be used to load and store data and / or instructions. In some embodiments, the system memory 1412 of the electronic device 1400 may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

[0049] The non-volatile memory 1416 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 1416 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a HDD (Hard Disk Drive), a CD (Compact Disc) drive, and a DVD (Digital Versatile Disc) drive.

[0050] The non-volatile memory 1416 may include a portion of storage resources on a device on which the electronic device 1400 is installed, or it may be accessible to the electronic device but is not necessarily a portion of the electronic device. For example, the non-volatile memory 1416 may be accessed over a network via the network interface 1420 .

[0051] In particular, the system memory 1412 and the non-volatile memory 1416 may include a temporary copy and a permanent copy of the instructions 1424, respectively. The instructions 1424 may include instructions that, when executed by at least one of the processors 1404, cause the electronic device 1400 to implement the following: Figure 1In some embodiments, instructions 1424, hardware, firmware, and / or software components thereof may additionally / alternatively be located in system control logic unit 1408, network interface 1420, and / or processor 1404.

[0052] The network interface 1420 may include a transceiver for providing a radio interface for the electronic device 1400, thereby communicating with any other suitable device (such as a front-end module, an antenna, etc.) through one or more networks. In some embodiments, the network interface 1420 may be integrated with other components of the electronic device 1400. For example, the network interface 1420 may be integrated with at least one of the processor 1404, the system memory 1412, the non-volatile memory 1416, and a firmware device (not shown) having instructions. When at least one of the processors 1404 executes the instructions, the electronic device 1400 implements the following. Figure 1 The method shown.

[0053] The network interface 1420 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 1420 may be a network adapter, a wireless network adapter, a telephone modem and / or a wireless modem.

[0054] In one embodiment, at least one of the processors 1404 may be packaged together with logic for one or more controllers of the system control logic unit 1408 to form a system in package (SiP). In one embodiment, at least one of the processors 1404 may be integrated on the same die with logic for one or more controllers of the system control logic unit 1408 to form a system on chip (SoC).

[0055] The electronic device 1400 may further include an input / output (I / O) device 1432. The I / O device 1432 may include a user interface to enable a user to interact with the electronic device 1400; the design of the peripheral component interface enables the peripheral components to interact with the electronic device 1400. In some embodiments, the electronic device 1400 further includes a sensor for determining at least one of an environmental condition and location information related to the electronic device 1400.

[0056] In some embodiments, the user interface may include, but is not limited to, a display (e.g., an LCD display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., an LED flash), and a keyboard.

[0057] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0058] Program code can be applied to input instructions to perform each function described in the present application and generate output information. Output information can be applied to one or more output devices in a known manner. For the purpose of the present application, the processing system includes any system with a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC) or a microprocessor.

[0059] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0060] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including, but not limited to, a floppy disk, an optical disk, an optical disc, a read-only memory (CD-ROM), a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic card or an optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., a carrier wave, an infrared signal, a digital signal, etc.) using the Internet in an electrical, optical, acoustic, or other form of propagation signal. Therefore, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0061] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0062] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0063] It should be noted that, in the examples and descriptions, relational terms such as first and second, etc., are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise one" do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0064] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims

1. An image processing method, used in electronic equipment, characterized in that: include: An extraction step of extracting the first coordinate of the spine feature in the image pixel coordinate system from the acquired back image; A conversion step, based on a preset transformation matrix, converting the first coordinates of the spinal feature into second coordinates in the terminal coordinate system of the guiding device; A processing step, based on the second coordinate of the spinal column feature, smoothing the trajectory of the spinal column feature to obtain a smooth three-dimensional spinal column feature trajectory; An acquisition step, acquiring an ultrasonic image of the spine scanned by an ultrasonic scanning device on the guiding device along the three-dimensional spine characteristic trajectory; an identification step of identifying abnormal deformation of the spine from the spine ultrasound image; The calculation step is to draw the minimum circumscribed rectangle of each abnormal deformation, and calculate the angle between one of the upper edge and the lower edge of the minimum circumscribed rectangle of each abnormal deformation and one of the upper edge and the lower edge of the minimum circumscribed rectangle of other abnormal deformations, and take the largest angle.

2. The image processing method according to claim 1, characterized in that: The back image is a two-dimensional image or a three-dimensional image collected by an imaging device.

3. The image processing method according to claim 2, characterized in that: When the back image is a two-dimensional image, the preset transformation matrix is ​​expressed by formula (1): Formula (1) Among them, P p is the first coordinate of the two-dimensional spine feature, P e is the second coordinate of the three-dimensional spine feature, K -1 is the inverse matrix of the intrinsic parameter matrix of the imaging device, T cw is the transformation matrix between the coordinate system of the imaging device and the world coordinate system, T we is the transformation matrix between the world coordinate system and the terminal coordinate system of the guiding device, and s is the transformation scale coefficient.

4. The image processing method according to claim 2, characterized in that: When the back image is a three-dimensional image, the preset transformation matrix is ​​expressed by formula (2): Formula (2) Among them, P c is the first coordinate of the three-dimensional spine feature, P e is the second coordinate of the three-dimensional spine feature, T cw is the transformation matrix between the coordinate system of the imaging device and the world coordinate system, T we is the transformation matrix between the world coordinate system and the terminal coordinate system of the guiding device.

5. The image processing method according to claim 1, characterized in that: The ultrasonic scanning device is installed at the end of the guiding device, and the guiding device controls the ultrasonic scanning device to scan along the three-dimensional spine characteristic trajectory. Wherein, the guiding device is a collaborative robot or a robotic arm.

6. The image processing method according to claim 5, characterized in that: The ultrasonic image scanned by the ultrasonic scanning device is corrected according to the coordinate position of the end of the guiding device to obtain the spinal ultrasound image.

7. The image processing method according to claim 1, characterized in that: The vertebral body comprises an abnormal deformation, Wherein, in the identification step, the spinal ultrasound image is preprocessed, and then image segmentation is performed, and the abnormal deformation is identified from the segmented image.

8. An image processing device, characterized in that: include: An extraction unit extracts a first coordinate of a spine feature in an image pixel coordinate system from the acquired back image; A conversion unit, based on a preset transformation matrix, converts the first coordinate of the spinal feature into a second coordinate in a terminal coordinate system of the guiding device; A processing unit, based on the second coordinate of the spinal column feature, performs smoothing processing on the trajectory of the spinal column feature to obtain a smooth three-dimensional spinal column feature trajectory; An acquisition unit, which acquires an ultrasonic image of the spine scanned by the ultrasonic scanning device on the guiding device along the three-dimensional spine characteristic trajectory; an identification unit, which identifies abnormal deformation of the spine from the spine ultrasound image; The calculation unit draws the minimum circumscribed rectangle of each abnormal deformation, calculates the angle between one of the upper edge and the lower edge of each minimum circumscribed rectangle of the abnormal deformation and one of the upper edge and the lower edge of other minimum circumscribed rectangles of the abnormal deformation, and takes the largest angle.

9. A computer-readable storage medium, characterized in that: The storage medium stores instructions, which, when executed on a computer, cause the computer to execute the image processing method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the image processing method described in any one of claims 1 to 7.

Images