Image processing method, electronic equipment, readable storage medium and program product

Abstract

The invention discloses an image processing method, electronic equipment, a readable storage medium and a program product. The image processing method comprises the steps of firstly obtaining original volume data of an initial target object and a camera first image containing an effective area and a black edge area, then carrying out translation and rotation calibration on three-dimensional volume data based on the first image to obtain a second image and a third image, then storing a transformation matrix according to the third image and carrying out data separation, and collecting data training by using an AI automatic calibration model, and finally obtaining a processed image. According to the technical scheme, through innovative interactive design, parameter separation architecture and AI fusion, a leap-through breakthrough is achieved on core indexes such as operation efficiency, data safety and calibration precision, and a set of efficient, accurate and reliable systematic solution is provided for the field of medical image analysis.

Description

Technical Field

[0001] The present invention relates to the field of image processing, specifically an image processing method, an electronic device, a readable storage medium, and a program product. Background Art

[0002] The three-dimensional volume data of a CT is obtained by reconstructing the multi-angle images of the object to be photographed by a CT using a reconstruction algorithm, and is generally used for medical diagnosis. A medical CT device reconstructs a three-dimensional image of a specific part of a patient, that is, three-dimensional volume data, and analyzes the condition of the patient by analyzing the characteristics of the three-dimensional image. When taking a CT image, the patient needs to be positioned, that is, the part of the patient to be photographed is fixed at a specific position and angle, so as to obtain clearer and easier-to-analyze three-dimensional volume data after shooting. The current CT images have high requirements for positioning during shooting. Slight deviation in positioning will lead to a decrease in the quality of CT images, affecting subsequent analysis and diagnosis. For example, in the field of orthodontics, patients are often required to face forward, not lower or raise their heads, and their heads should not be tilted left or right. Otherwise, the final three-dimensional volume data will cause great trouble to doctors' diagnosis.

[0003] Current Technology and Disadvantages: The traditional processing method is that for three-dimensional volume data with poor shooting quality such as unqualified positioning, patients are required to re-position and shoot. This not only wastes the time of patients and doctors, but also patients have to endure electromagnetic radiation multiple times, which is not good for patients' health. Summary of the Invention

[0004] The purpose of the present invention is to provide an image processing method to solve the problems raised in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solutions: An image processing method includes: obtaining the original volume data of the initial target object and the first image captured by a camera, where the first image includes a valid region and a black edge region other than the valid region; based on the first image, calibrating the three-dimensional volume data, including translational calibration and rotational calibration, where translational calibration: adjusting the position of the volume data in three dimensions of the coronal plane, sagittal plane, and transverse plane to obtain a corrected second image, rotational calibration: rotating the volume data around the x, y, and z axes to correct the angular deviation and obtain a corrected third image; storing the transformation matrix and separating the data according to the third image, saving the translational offset and rotation angle, where the translational offset includes the offsets in the x-axis, y-axis, and z-axis directions in three-dimensional space, and the corresponding rotation angles are α, β, and γ, and integrating the translational and rotational parameters into a 4×4 transformation matrix according to the translational offset and rotation angle; data separation: keeping the original DICOM data unchanged, adjusting the parameters to be stored in the form of an additional file, and dynamically applying the transformation matrix during loading; AI automatic calibration model: collecting the original volume data of the positioning deviation and the corresponding calibration parameters, using a convolutional neural network, with the input being the original volume data and the output being the predicted transformation parameters, and the AI model automatically analyzing the newly input three-dimensional volume data and outputting the calibration parameters; obtaining the processed image.

[0006] According to at least one embodiment of the present disclosure, after adjusting the position of the volume data in three dimensions of the coronal plane, sagittal plane, and transverse plane in the translational calibration, the corrected second image is obtained, including: main view area: displaying the CT tomographic image and the superimposed cross positioning lines, where the cross positioning lines are composed of a red horizontal line and a vertical line intersecting orthogonally; auxiliary view area: including a coronal plane cutting display window and a transverse plane cutting display window; interaction status indicator: an annular area with a diameter of 15-20 pixels, which is default hidden at the intersection point of the cross positioning lines.

[0007] According to at least one embodiment of the present disclosure, the interaction control method of the interaction status indicator includes: Step S1: Cursor sensing and status activation, when the pointer of the input device enters a circular activation area with the center of the cross positioning lines as the center and a radius of 5-8 pixels; a semi-transparent blue ring is displayed; the operation status of the cross positioning lines is changed from the static mode to the to-be-dragged mode; Step S2: Dragging operation and coordinate mapping, when a left mouse button press event is detected; establishing a mapping relationship between the screen coordinates and the medical coordinate system: converting the current cursor position ( ) to the reference point ( ) of the medical coordinate system; starting a real-time position tracking thread with a sampling frequency not lower than 60Hz; changing the color of the cross positioning lines to orange and increasing the line width to 2 pixels; Step S3: Dynamic synchronous update, during the dragging process, the system performs multi-view linkage according to the following rules: calculating the new center point coordinates in real time according to the displacement amount ( , , where , is the conversion coefficient between pixels and millimeters; the blue line cutting position is longitudinally displaced according to the coordinate value, and the displacement speed is linearly related to the dragging speed; the yellow line cutting position is horizontally displaced according to the coordinate value, and the displacement delay is controlled within 16 milliseconds; the double-buffered drawing technology is used to eliminate view flickering. Step S4: Operation termination and status restoration. When the left mouse button release event is detected; submit the final coordinates to the DICOM data processing module; restore the cross positioning line to the initial red state; hide the blue activation indicator ring; update the reference plane data of the 3D reconstruction engine.

[0008] According to at least one embodiment of the present disclosure, the rotation calibration rotates the volume data around the x, y, and z axes. After correcting the angle deviation, the corrected third image is obtained, including: Rotation control line: Display a horizontal reference line and a vertical reference line in the orthogonal view, with a line width of 1-2 pixels and colors of green and yellow respectively; Rotation center locking module: Use the position determined by the last drag of the cross positioning line as the current rotation center point ( , , ); Multi-plane rendering engine: Includes a sagittal plane processor, a cross-sectional processor, and a 3D volume renderer.

[0009] According to at least one embodiment of the present disclosure, the rotation calibration control method includes: Step T1: Rotation operation activation. When the input device pointer touches within the range of ±3 pixels of the horizontal / vertical line: Visual feedback: Generate a circular highlight area with a diameter of 8 pixels at the touch point; Mode switch: Activate the 3D rotation operation mode; Step T2: Rotation parameter calculation. When the mouse drag event is detected; Displacement conversion: Convert the screen coordinate displacement (Δx, Δy) into a 3D rotation angle. The rotation angle around the X axis is θ = * ( = 0.5° / pixel), and the rotation angle around the Y axis is = * ( = 0.5° / pixel); Construct the rotation matrix: R = (θ) * ( ), where Rx and Ry are the standard homogeneous matrices for rotation around the X / Y axis; Step T3: Dynamic rendering and synchronization. Perform real-time updates during rotation: 3D volume data transformation, apply the transformation to all vertices of the volume data: = R * (V - ) + , where V: The original vertex coordinates (position before rotation), : Coordinates of the rotation center point (determined by dragging the crosshairs, e.g., ), : Coordinates of the transformed vertices (new positions after rotation). Sagittal plane update: Generate a cutting plane in the YZ plane, with a rendering interval ≤ 10 ms; Transverse plane update: Generate a cutting plane in the XY plane, using bilinear interpolation algorithm; Visual assistance: Display a rotation trackball in the 3D view; Step T4: Reset the state. After detecting the mouse release event: Freeze the current rotation angle and submit it to the DICOM metadata; Hide the highlighted area; Update the coordinate system display of all views.

[0010] According to at least one embodiment of the present disclosure, the method for storing the transformation matrix for the third image is as follows: Parameter storage module: Independently store the translation offset and rotation angle; Matrix generation engine: Convert the stored parameters into a 4×4 transformation matrix in real time; Dynamic loader: Fuse and render the transformation matrix with the original DICOM data during runtime.

[0011] According to at least one embodiment of the present disclosure, the 4×4 transformation matrix is: T = * (γ)* (β)* (α), where γ is the rotation angle around the Z axis, β is the rotation angle around the Y axis, α is the rotation angle around the X axis, and T is the composite rotation transformation matrix, representing three rotations in sequence around the Z, Y, and X axes, and and are homogeneous matrices used to describe spatial transformation, representing translation and rotation around the X, Y, and Z axes respectively.

[0012] According to at least one embodiment of the present disclosure, the method for data separation includes: Original data protection: Store the DICOM file in a read-only / memory, and establish a hash verification mechanism; Dynamic loading process: Load the original volume data into the video memory; Read the current transformation parameters from the parameter library; The GPU compute shader generates the transformation matrix in real time; Apply the matrix to the vertex shader stage; Output the fused image to the rendering pipeline.

[0013] An electronic device, characterized in that it includes: a memory that stores execution instructions; and a processor that executes the execution instructions stored in the memory, such that the processor executes the image processing method.

[0014] A readable storage medium, characterized in that a computer program is stored in the readable storage medium, and when the computer program is executed by a processor, it is used to implement the image processing method.

[0015] A computer program product, characterized in that the computer program product includes a computer program, and when the computer program is executed by a processor, it is at least used to implement the image processing method.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Improve the diagnostic efficiency and save time for both doctors and patients. By using AI or adjusting the three-dimensional volume data, the three-dimensional volume data can be quickly made to meet the diagnostic standard, without requiring the patient to reposition and take pictures again, thus improving the efficiency.

[0017] 2. Do not suffer from electromagnetic radiation multiple times, which is beneficial to the physical and mental health of patients. Since the three-dimensional volume data can be adjusted through software and AI, without the need for the patient to reposition and take pictures multiple times, the electromagnetic radiation suffered by the patient is reduced, which is beneficial to their physical and mental health. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 FIG. is a schematic flowchart of an image processing method according to an embodiment of the present disclosure.

[0019] Figure 2 FIG. is a schematic flowchart of translation calibration in image processing according to an embodiment of the present disclosure.

[0020] Figure 3 FIG. is a schematic diagram of volume data translation calibration in image processing according to an embodiment of the present disclosure.

[0021] Figure 4 FIG. is a schematic flowchart of rotation calibration in image processing according to an embodiment of the present disclosure.

[0022] Figure 5 FIG. is a schematic diagram of volume data rotation calibration in image processing according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The technical solutions of the present disclosure will be further described in detail below in conjunction with the specific embodiments.

[0024] Please refer to Figures 1-5, Image processing method. The image correction process includes: Data acquisition: Obtain the original volume data of the initial target object and the first image captured by the camera. The first image has a valid area and a black edge area. For example, in orthodontic CT imaging of the oral cavity, the patient's face forms the valid area, and the surrounding irrelevant parts form the black edge area. The original volume data is the basis for subsequent processing, and the first image is used for auxiliary calibration. Three-dimensional volume data calibration: Translation calibration: Adjust the position of the volume data in three dimensions: the coronal plane, the sagittal plane, and the transverse plane. Taking a brain CT image as an example, if the patient's head is shifted left or right, or forward or backward during shooting, the image can be adjusted to the standard position through translation calibration to obtain the corrected second image. Rotation calibration: Rotate the volume data around the x, y, and z axes to correct the angular deviation and obtain the corrected third image. For example, if the patient's head is tilted during shooting, rotation calibration can correct the angle to make the image meet the diagnostic requirements. Transformation matrix storage and data separation: Save the translation offset and rotation angle, and integrate them into a 4×4 transformation matrix. The translation offset includes the offsets in the x, y, and z directions in three-dimensional space, and the rotation angles are α, β, and γ. The original DICOM data remains unchanged, and the adjustment parameters are stored in the form of an additional file and applied dynamically when loaded. This ensures the security of the original data and at the same time facilitates real-time adjustment of the image according to the calibration parameters. AI automatic calibration model: Collect the original volume data of the positioning deviation and the corresponding calibration parameters, and use a convolutional neural network. Input the original volume data and output the predicted transformation parameters to automatically analyze the newly input three-dimensional volume data and output the calibration parameters. After training with a large number of lung CT images, the model can quickly provide accurate calibration parameters for new lung CT images, improving the calibration efficiency.

[0025] M100: Obtain the original volume data of the initial target object and the first image captured by the camera, where the first image includes a valid area and a black edge area other than the valid area.

[0026] M200: Calibrate the three-dimensional volume data based on the first image, including translation calibration and rotation calibration, where translation calibration: Adjust the position of the volume data in three dimensions: the coronal plane, the sagittal plane, and the transverse plane, to obtain the corrected second image, rotation calibration: Rotate the volume data around the x, y, and z axes to correct the angular deviation and obtain the corrected third image.

[0027] Interface elements: Main view area: Display the CT tomographic image and the red orthogonal crosshair to help the doctor locate the region of interest. In a liver CT image, the doctor can determine the location of the lesion in the liver through the crosshair.

[0028] Auxiliary view area: Include the coronal plane and transverse plane cutting display windows to show the volume data from different angles, and cooperate with the main view area to comprehensively present the lesion information.

[0029] Interaction status indicator: By default, it is hidden at the intersection of the crosshair. It has a diameter of 15 - 20 pixels. When the pointer enters the activation area with a radius of 5 - 8 pixels centered at the center of the crosshair, a semi - transparent blue ring is displayed to activate the interaction status.

[0030] Interaction steps: Cursor sensing and status activation: When the pointer enters the activation area, a blue ring is displayed, and the crosshair changes to the drag - waiting mode, indicating that the doctor can perform operations.

[0031] Drag operation and coordinate mapping: When the left mouse button is pressed, the mapping relationship between the screen coordinates and the medical coordinate system is established, and a real - time position tracking thread with a sampling frequency of not less than 60 Hz is started. The crosshair changes color and becomes thicker. For example, when dragging the crosshair to mark the position of the kidney in a kidney CT image, the system tracks in real - time.

[0032] Dynamic synchronous update: During dragging, the new center point coordinates are calculated based on the displacement amount ( , ), where , are the translation amounts along the X and Y axes. The cutting positions of the blue line and the yellow line are displaced longitudinally and horizontally according to the new coordinates respectively, and the displacement delay is controlled within 16 milliseconds. The double - buffer drawing technology is used to eliminate flicker.

[0033] Operation termination and status restoration: When the left mouse button is released, the coordinates are submitted to the DICOM data processing module, the crosshair and the indicator ring are restored to their initial states, and the reference plane data of the 3D reconstruction engine is updated.

[0034] After the translation calibration adjusts the position of the volume data in the three dimensions of the coronal plane, sagittal plane, and transverse plane, the corrected second image is obtained, including: Main view area: Displays the CT tomographic image and the superimposed crosshair, which is composed of a red horizontal line and a vertical line intersecting orthogonally; Auxiliary view area: Includes a coronal plane cutting display window and a transverse plane cutting display window; Interaction status indicator: An annular area with a diameter of 15 - 20 pixels, which is hidden at the intersection of the crosshair by default.

[0035] The interaction control method of the interaction status indicator includes: S1: Cursor sensing and status activation, when the pointer of the input device enters the circular activation area with a radius of 5 - 8 pixels centered at the center of the crosshair; A semi - transparent blue ring is displayed; The operation status of the crosshair is changed from the static mode to the drag - waiting mode; S2: Drag operation and coordinate mapping, when the left mouse button press event is detected; Establish the mapping relationship between screen coordinates and the medical coordinate system: Convert the current cursor position ( ) to the reference point ( ) in the medical coordinate system; Start the real-time position tracking thread with a sampling frequency of not less than 60 Hz; Change the color of the crosshair to orange and increase the line width to 2 pixels; S3: Dynamic synchronous update. During the dragging process, the system performs multi-view linkage according to the following rules: Calculate the new center point coordinates in real time according to the displacement amount ( ), , where , is the conversion coefficient between pixels and millimeters.

[0036] Translation calibration formula for the new center point coordinates: , , where and are the initial reference point coordinates, is the cursor displacement amount, and are the conversion coefficients, which are derived from the DICOM pixel spacing. This formula is based on the principle of similar triangles. The pixel displacement on the screen is proportional to the actual displacement in the medical coordinate system, and the conversion between the two is achieved through the conversion coefficient to ensure accurate coordinate calculation.

[0037] The cutting position of the blue line is longitudinally displaced according to the coordinate value, and the displacement speed is linearly related to the dragging speed; The cutting position of the yellow line is horizontally displaced according to the coordinate value, and the displacement delay is controlled within 16 milliseconds; Use double-buffered drawing technology to eliminate view flickering; S4: Operation termination and state restoration. When the left mouse button release event is detected; Submit the final coordinates to the DICOM data processing module; Restore the crosshair to its initial red state; Hide the blue activation indicator ring; Update the reference plane data of the 3D reconstruction engine.

[0038] Spatial coordinate system mapping model: Define the conversion relationship between the medical coordinate system (X, Y, Z) and the screen coordinate system (u, v): where W is the screen width (pixels), H is the screen height (pixels), and is the conversion coefficient, which is derived from the DICOM pixel pitch. θ is the coronal plane tilt angle. is the sagittal plane projection angle. For example, θ = 15° is the coronal plane tilt angle. = 10° is the sagittal plane projection angle. = 0.1953 mm / pixel. = 0.2034 mm / pixel.

[0039] Translation calibration mathematical model: The displacement - coordinate mapping relationship is as follows. Define the translation transformation matrix: Among them, is the screen abscissa of the starting point of the translation operation, and u is the screen abscissa of the ending point of the translation operation. is the conversion coefficient, which is derived from the DICOM pixel pitch. Combined with to achieve accurate mapping from pixels to physical space. and are the translation amounts along the X, Y, and Z axes. , = , FOV = 300 mm. = 512, = 0.5859 mm / pixel.

[0040] Error propagation analysis, using first - order Taylor expansion: Calculated = 0.048 mm 。

[0041] S1: Cursor sensing and status activation Principle: When the pointer of the input device enters the circular activation area with the center of the cross - hair as the center and a radius of 5 - 8 pixels, a semi - transparent blue ring is displayed, and the operation status of the cross - hair is changed from the stationary mode to the drag - waiting mode. This process realizes the sensing of the user's operation intention, informs the user that the system is ready to accept the drag operation through visual cues (display of the blue ring), and at the same time changes the operation status of the cross - hair to prepare for the subsequent drag operation.

[0042] Example: When the doctor uses the mouse to move the pointer to the activation area near the center of the cross - hair, the semi - transparent blue ring will be immediately displayed, indicating that the doctor can start dragging the cross - hair. At this time, the cross - hair is in the drag - waiting mode, waiting for further operation instructions from the doctor.

[0043] S2: Drag operation and coordinate mapping Principle: When the left mouse button press event is detected, establish the mapping relationship between the screen coordinates and the medical coordinate system, convert the current cursor position into the reference point in the medical coordinate system, start the real-time position tracking thread (sampling frequency not less than 60Hz), and change the color of the crosshair to orange and increase the line width to 2 pixels. This step realizes the accurate mapping of the user's operation (mouse dragging) on the screen into the medical coordinate system, providing an accurate coordinate basis for subsequent multi-view linkage. The high sampling frequency of the real-time position tracking thread ensures that the movement of the mouse can be captured in time, making the operation smoother.

[0044] Example: When the doctor presses the left mouse button and starts dragging the crosshair, the system will quickly establish the mapping relationship between the screen coordinates and the medical coordinate system, and convert the current mouse position into the reference point in the medical coordinate system. For example, in a lung CT image, when the doctor drags the crosshair to mark the position of a lung nodule, the system will accurately record the position information of the nodule in the medical coordinate system according to the mapping relationship. At the same time, the color of the crosshair changes to orange and the line width increases to highlight the current dragging operation state.

[0045] S3: Dynamic synchronous update Principle: During the dragging process, the system performs multi-view linkage according to certain rules. Calculate the new center point coordinates in real time according to the displacement amount, the vertical displacement of the blue line cutting position according to the coordinate value (the displacement speed is linearly related to the dragging speed), the horizontal displacement of the yellow line cutting position according to the coordinate value (the displacement delay is controlled within 16 milliseconds), and use the double-buffer drawing technology to eliminate view flickering. This process realizes that when the user drags the crosshair, multiple views can be updated in real time and synchronously, ensuring that the doctor can see the changes in each view in time during the operation, improving the accuracy and efficiency of the operation. The double-buffer drawing technology effectively solves the problem of view flickering and improves the user experience.

[0046] Example: When the doctor drags the crosshair in the main view area, the blue line in the coronal plane cutting display window will perform vertical displacement according to the coordinate value, and the displacement speed is linearly related to the dragging speed, enabling the doctor to intuitively see the position change of the lesion in the coronal direction. At the same time, the yellow line in the transverse plane cutting display window will perform horizontal displacement, and the displacement delay is controlled within 16 milliseconds, ensuring the real-time nature of the operation. For example, when observing a liver lesion, when the doctor drags the crosshair, each view can quickly and accurately reflect the change in the lesion position, helping the doctor to understand the lesion situation more comprehensively.

[0047] S4: Operation termination and state restoration Principle: When the left mouse button release event is detected, submit the final coordinates to the DICOM data processing module, restore the crosshair to its initial red state, hide the blue activation indicator ring, and update the reference plane data of the 3D reconstruction engine. This step completes the entire interaction operation, submitting the user's operation result (final coordinates) to the data processing module for further processing, while restoring the system to its initial state to prepare for the next operation.

[0048] Example: When the doctor finishes marking the lesion location and releases the left mouse button, the system submits the final coordinates to the DICOM data processing module for subsequent diagnosis and analysis. At the same time, the crosshair returns to its initial red state, the blue activation indicator ring is hidden, and the reference plane data of the 3D reconstruction engine is updated so that the doctor can perform other operations or observe different image regions.

[0049] Interface elements: Rotation control line: Displays green, yellow horizontal and vertical reference lines with a line width of 1 - 2 pixels in the orthogonal view as a reference for rotation operations.

[0050] Rotation center locking module: Uses the position determined by the last drag of the crosshair as the rotation center point to ensure a fixed reference for rotation operations.

[0051] Multi - plane rendering engine: Includes sagittal plane, cross - sectional processor, and 3D volume renderer to render the rotated image in real - time.

[0052] Interaction steps: Activation of rotation operation: When the pointer touches within a range of ±3 pixels of the control line, a circular highlight area is generated to activate the 3D rotation operation mode.

[0053] Calculation of rotation parameters: When the mouse is dragged, convert the screen coordinate displacement into 3D rotation angles (rotation angle around the X - axis θ = * , rotation angle around the Y - axis = * , = = 0.5° / pixel), construct the rotation matrix R = (θ)* ( ), where and are homogeneous matrices used to describe spatial transformation, representing rotations around the X, Y, and Z axes respectively.

[0054] Dynamic rendering and synchronization: Apply the transformation to the volume data vertices, update the sagittal and cross - sectional views in real - time, and display a rotation trackball for auxiliary observation.

[0055] Status reset: When the mouse is released, freeze the rotation angle and submit it to the DICOM metadata, hide the highlighted area, and update the coordinate system display.

[0056] The rotation calibration rotates the volume data around the x, y, and z axes. After correcting the angle deviation, the corrected third image is obtained, including: Rotation control line: Display a horizontal reference line and a vertical reference line in the orthogonal view. The line width is 1-2 pixels, and the colors are green and yellow respectively; Rotation center locking module: Use the position determined by dragging the crosshair for the last time as the current rotation center point ( , , ); Multi-plane rendering engine: Includes a sagittal plane processor, a transverse plane processor, and a 3D volume renderer.

[0057] The rotation calibration control method includes: T1: When the rotation operation is activated and the input device pointer touches within the range of ±3 pixels of the horizontal / vertical line: Visual feedback: Generate a circular highlighted area with a diameter of 8 pixels at the contact point; Mode switch: Activate the 3D rotation operation mode; T2: When a mouse drag event is detected during rotation parameter calculation; Displacement conversion: Convert the screen coordinate displacement to a 3D rotation angle. The rotation angle around the X axis is θ = * ( = 0.5° / pixel), and the rotation angle around the Y axis is φ = * ( = 0.5° / pixel); Rotation calibration rotation angle formula: The rotation angle around the X axis is θ = * , and the rotation angle around the Y axis is φ = * . 、 are conversion coefficients (0.5° / pixel). According to the trigonometric function relationship, the screen coordinate displacement is related to the rotation angle. The conversion coefficients are determined through experiments and theoretical derivations to achieve accurate rotation operations; Construct a rotation matrix: R = (θ)* ( ), where Rx and Ry are the standard homogeneous matrices for rotation around the X / Y axis; T3: Dynamic rendering and synchronization, perform real-time updates during rotation: 3D volume data transformation, applying the transformation to all vertices of the volume data: = R * (V - ) + , where V: original vertex coordinates (position before rotation), : coordinates of the rotation center point (determined by dragging the crosshair, e.g., ), : transformed vertex coordinates (new position after rotation).

[0058] Sagittal plane update: generating a cutting plane in the YZ plane, rendering interval ≤ 10 ms; Transverse plane update: generating a cutting plane in the XY plane, using bilinear interpolation algorithm; Visual assistance: displaying a rotation trackball in the 3D view; T4: state reset, after detecting the mouse release event: Freezing the current rotation angle and submitting it to the DICOM metadata; Hiding the highlighted area; Updating the coordinate system display of all views.

[0059] M300: storing the transformation matrix and separating data according to the third image, saving the translation offset and rotation angle, where the translation offset includes the offsets in the x-axis, y-axis, and z-axis directions in 3D space, and the corresponding rotation angles are α, β, γ. Integrating the translation and rotation parameters into a 4×4 transformation matrix according to the translation offset and rotation angle; data separation: the original DICOM data remains unchanged, adjusting the parameters to be stored in the form of an additional file, and dynamically applying the transformation matrix during loading.

[0060] The method for storing the transformation matrix for the third image is as follows: parameter storage module: independently storing the translation offset and rotation angle; matrix generation engine: converting the stored parameters into a 4×4 transformation matrix in real time; dynamic loader: fusing and rendering the transformation matrix with the original DICOM data during runtime.

[0061] The 4×4 transformation matrix is: T = * (γ)* (β)* (α), where γ is the rotation angle around the Z axis, β is the rotation angle around the Y axis, α is the rotation angle around the X axis, and T is the composite rotation transformation matrix, indicating three rotations in sequence according to the Z, Y, X order, and and are homogeneous matrices used to describe spatial transformation, representing translation and rotation around the X, Y, Z axes respectively.

[0062] The method for data separation includes: Original data protection: The DICOM files are stored in read-only / memory, and a hash verification mechanism is established; Dynamic loading process: Load the original volume data into the video memory; Read the current transformation parameters from the parameter library; The GPU computing shader generates the transformation matrix in real time; Apply the matrix to the vertex shader stage; Output the fused image to the rendering pipeline.

[0063] M400: AI automatic calibration model: Collect the original volume data of the positioning deviation and the corresponding calibration parameters, use a convolutional neural network, with the original volume data as the input and the predicted transformation parameters as the output. The AI model automatically analyzes the newly input three-dimensional volume data and outputs the calibration parameters.

[0064] M500: Obtain and process images.

[0065] The present invention also provides an electronic device, including: a memory that stores execution instructions; and a processor or other hardware module that executes the execution instructions stored in the memory, so that the processor or other hardware module executes the image processing method of the above embodiment.

[0066] The present invention also provides a readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the method of any of the above embodiments.

[0067] For the purposes of this specification, a "readable storage medium" can be any device that can contain, store, communicate, propagate, or transport a program for use in or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of the readable storage medium include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM).

[0068] The present invention also provides a computer program product. The method of the present invention can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, the process or function of the present invention is executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, an OAM, or other programmable devices.

[0069] The computer program or instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer program or instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired or wireless manner. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc; or it can be a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or can include both volatile and non-volatile types of storage media.

[0070] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, system, or computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0071] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device generate means for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or multiple flows and / or blocks

[0072] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one or more of the flows Figure 1 or multiple flows and / or blocks

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process Figure 1 one process or more processes and / or blocks Figure 1 steps for implementing the functions specified in one block or more blocks.

[0074] In the description of this specification, the descriptions with reference to the terms "one embodiment / way", "some embodiments / ways", "specific examples", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment / way or example are included in at least one embodiment / way or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily the same embodiment / way or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments / ways or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments / ways or examples described in this specification and the features of different embodiments / ways or examples.

[0075] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0076] Those skilled in the art should understand that the above embodiments are only for clearly explaining the present invention and are not intended to limit the scope of the present invention. For those skilled in the art, other changes or modifications can be made on the basis of the above disclosure, and these changes or modifications are still within the scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire original volume data of an initial target object and a first image captured by a camera, wherein the first image includes a valid area and a black border area except the valid area; Based on the first image, calibrate the three-dimensional volume data, including translation calibration and rotation calibration, wherein the translation calibration is to adjust the volume data position in the coronal plane, sagittal plane, and transverse plane to obtain a corrected second image, and the rotation calibration is to rotate the volume data around the x, y, and z axes to correct the angle deviation and obtain a corrected third image; According to the third image, the transformation matrix is ​​stored and the data is separated, and the translation offset and rotation angle are saved, wherein the translation offset includes the offset along the x-axis, y-axis and z-axis in the three-dimensional space, and the corresponding rotation angles are α, β and γ. According to the translation offset and the rotation angle, the translation and rotation parameters are integrated into a 4×4 transformation matrix; Data separation: the original DICOM data remains unchanged, the adjustment parameters are stored in the form of an additional file, and the transformation matrix is ​​dynamically applied when loading; AI automatic calibration model: collects the original volume data of positioning deviation and the corresponding calibration parameters, uses convolutional neural network, inputs the original volume data, and outputs the predicted transformation parameters. The AI ​​model automatically analyzes the newly input 3D volume data and outputs the calibration parameters. Get the processed image.

2. The image processing method according to claim 1, characterized in that: After the translation calibration adjusts the volume data position in the coronal plane, sagittal plane, and transverse plane, a corrected second image is obtained, including: Main view area: displays CT tomographic images and superimposed crosshairs, which are composed of orthogonal red horizontal and vertical lines; Auxiliary viewing area: including coronal cutting display window and cross-sectional cutting display window; Interaction status indicator: A 15-20 pixel diameter circular area, hidden by default at the intersection of the crosshairs.

3. The image processing method according to claim 2, characterized in that: The interactive control method of the interactive status indicator comprises: Step S1: Cursor sensing and state activation, when the pointer of the input device enters a circular activation area with a radius of 5-8 pixels and a center of the crosshair positioning line; a) Displays a semi-transparent blue ring; b) changing the crosshair operation state from static mode to waiting mode; Step S2: drag operation and coordinate mapping, when the left mouse button is pressed; a) Establish the mapping relationship between screen coordinates and medical coordinate system: Set the current cursor position Fiducial points converted to medical coordinate system ; b) Start the real-time location tracking thread with a sampling frequency of no less than 60 Hz; c) Change the crosshair color to orange and increase the line width to 2 pixels; Step S3: Dynamic synchronous update. During the dragging process, the system performs multi-view linkage according to the following rules: a) According to the displacement Calculate the new center point coordinates in real time , ,in , is the conversion factor between pixels and millimeters; b) Blue line cutting position press The coordinate value is displaced longitudinally, and the displacement speed is linearly related to the drag speed; c) Yellow line cutting position The coordinate value is displaced laterally, and the displacement delay is controlled within 16 milliseconds; d) Use double buffering drawing technology to eliminate view flicker; Step S4: Operation termination and state restoration, when a left mouse button release event is detected; a) Submit the final coordinates to the DICOM data processing module; b) Restore the crosshair to its initial red state; c) Hide the blue activation indicator ring; d) Update the reference plane data of the 3D reconstruction engine.

4. The image processing method according to claim 1, characterized in that: The rotation calibration rotates the volume data around the x, y, and z axes, and obtains a corrected third image after correcting the angle deviation, including: Rotation control lines: Display horizontal and vertical reference lines in the orthogonal view, with a line width of 1-2 pixels and colors of green and yellow respectively; The rotation center lock module uses the position determined by the most recent drag of the crosshair as the current rotation center point , , ; Multi-plane rendering engine: includes sagittal plane processor, cross-sectional plane processor and 3D volume renderer.

5. The image processing method according to claim 4, characterized in that: The rotation calibration comprises: Step T1: The rotation operation is activated, when the input device pointer touches the horizontal / vertical line within ±3 pixels: a) Visual feedback: Generate a circular highlight area with a diameter of 8 pixels at the contact point; b) Mode switching: Activate the three-dimensional rotation operation mode; Step T2: Rotation parameter calculation, when a mouse drag event is detected; a) Displacement conversion: Convert the screen coordinate displacement Δx, Δy into a three-dimensional rotation angle, the rotation angle around the X-axis θ= * ,in =0.5° / pixel, rotation angle around Y axis φ= * ,in = 0.5° / pixel; b) Construct the rotation matrix: R = (θ)* ( ),in (θ), ( ) is the standard homogeneous matrix for rotation around the X / Y axis; Step T3: Dynamic rendering and synchronization, performing real-time updates during rotation: a) 3D volume data transformation, apply transformation to all vertices of volume data: =R*(V- )+ , where V is the original vertex coordinate, is the coordinate of the rotation center point; b) Sagittal plane update: Generate cutting plane in YZ plane, rendering interval ≤ 10ms; c) Cross-section update: Generate cutting surface in XY plane, using bilinear interpolation algorithm; d) Visual aid: Display the rotating trackball in the 3D view; Step T4: State reset, after detecting the mouse release event: a) Freeze the current rotation angle and submit it to DICOM metadata; b) Hide the highlighted area; c) Update the coordinate system display of all views.

6. The image processing method according to claim 1, characterized in that: The method for storing the transformation matrix of the third image is: Independent storage of translation offset and rotation angle; Convert the stored parameters into a 4×4 transformation matrix in real time; The transformation matrix is ​​fused with the original DICOM data for rendering at runtime.

7. The image processing method according to claim 6, characterized in that: The 4×4 transformation matrix is: T= * (γ)* (β)* (α), where γ is the rotation angle around the Z axis, β is the rotation angle around the Y axis, α is the rotation angle around the X axis, and T is the composite rotation transformation matrix, which represents three rotations in the order of Z, Y, and X. and and It is a homogeneous matrix used to describe spatial transformation, representing translation and rotation around the X, Y, and Z axes respectively.

8. The image processing method according to claim 1, characterized in that: The data separation method comprises: Original data protection: DICOM files are stored in read-only / memory and a hash verification mechanism is established; Dynamic loading process: load the original volume data into the video memory; read the current transformation parameters from the parameter library; the GPU compute shader generates the transformation matrix in real time; apply the matrix to the vertex shader stage; output the fused image to the rendering pipeline.

9. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instruction stored in the memory, so that the processor executes the image processing method according to any one of claims 1 to 8.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program is used to implement the image processing method according to any one of claims 1 to 8.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program is used to implement at least the image processing method according to any one of claims 1 to 8.

Images