Image processing method, device, equipment and storage medium

Abstract

The present application discloses an image processing method, apparatus, device, and storage medium that can be applied to various scenarios such as artificial intelligence, medical technology, cloud technology, blockchain, and image processing. The method comprises: inputting an acquired image to be processed into a target image processing model to obtain a plurality of target feature information to be processed corresponding to the image to be processed, the plurality of target feature information to be processed including: target surgical reference surface feature information, target tissue region feature information, and target key point feature information; determining surgical incision location information based on the plurality of target feature information to be processed; generating corresponding surgical incision location prompt information based on the surgical incision location information; acquiring on-site image information of the target object; and displaying the surgical incision location prompt information and the on-site image information based on the plurality of target feature information to be processed, the on-site image information, and the surgical incision location information to improve the efficiency of determining the surgical incision location for the target object.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] In related fields, before doctors perform neurosurgery, they need to read the patient's CT (Computed Tomography) images or MRI (Magnetic Resource Imaging) images, calculate the distance between the lesion and landmarks such as the ear canthus line, the midline of the brain, and the root of the nose, find the corresponding anatomical landmarks on the patient's head, draw auxiliary lines, and finally determine the location of the surgical incision. The process of determining the location of the surgical incision is time-consuming and the efficiency of determining the location of the surgical incision is low. Summary of the Invention

[0003] The embodiments of the present application provide an image processing method, apparatus, device, and storage medium, which can save time in determining the surgical incision position for the target object and improve the efficiency of determining the surgical incision position for the target object.

[0004] In one aspect, an embodiment of the present application provides an image processing method, comprising:

[0005] Obtaining an image of the target object to be processed;

[0006] Inputting the image to be processed into a target image processing model to obtain a plurality of target feature information to be processed corresponding to the image to be processed, wherein the plurality of target feature information to be processed includes: target surgical reference surface feature information, target tissue region feature information, and target key point feature information;

[0007] Determining surgical incision position information based on the multiple target feature information to be processed;

[0008] generating corresponding surgical incision position prompt information according to the surgical incision position information, wherein the surgical incision position prompt information is used to prompt the surgical incision position for the target object;

[0009] Acquiring on-site image information of the target object;

[0010] The surgical incision position prompt information and the on-site image information are displayed based on the multiple target feature information to be processed, the on-site image information and the surgical incision position information.

[0011] On the other hand, an embodiment of the present application provides a data processing method, including:

[0012] Get a sample image;

[0013] The initial semantic feature extraction unit in the initial image processing model performs feature extraction on the sample image to obtain a plurality of semantic feature information corresponding to the sample image, wherein the plurality of semantic feature information includes: plane semantic feature information, regional semantic feature information and key point semantic feature information;

[0014] Processing the multiple semantic feature information by an initial feature processing unit in the initial image processing model to obtain multiple predicted feature information to be processed corresponding to the multiple semantic feature information, the multiple predicted feature information to be processed including: predicted surgical reference surface feature information, predicted tissue region feature information, and predicted key point feature information;

[0015] Obtaining multiple target feature label information corresponding to the sample image;

[0016] The initial image processing model is trained according to the multiple predicted feature information to be processed and the multiple target feature label information to obtain a target image processing model, wherein the target image processing model includes a target semantic feature extraction unit and a target feature processing unit, which is used to obtain multiple target feature information to be processed corresponding to the image to be processed based on the acquired image to be processed, wherein the multiple target feature information to be processed includes: target surgical reference surface feature information, target tissue region feature information, and target key point feature information;

[0017] The target image processing model is the target image processing model in the aforementioned image processing method.

[0018] In another aspect, an embodiment of the present application provides an image processing device, comprising:

[0019] An acquisition unit, configured to acquire an image to be processed of a target object;

[0020] An input unit, configured to input the image to be processed into a target image processing model to obtain a plurality of target feature information to be processed corresponding to the image to be processed, wherein the plurality of target feature information to be processed includes: target surgical reference surface feature information, target tissue region feature information, and target key point feature information;

[0021] a determining unit, configured to determine surgical incision position information based on the plurality of target feature information to be processed;

[0022] a generating unit, configured to generate corresponding surgical incision position prompt information according to the surgical incision position information, wherein the surgical incision position prompt information is used to prompt the surgical incision position for the target object;

[0023] The acquisition unit is further configured to acquire on-site image information of the target object;

[0024] A display unit is used to display the surgical incision position prompt information and the on-site image information based on the multiple target feature information to be processed, the on-site image information and the surgical incision position information.

[0025] On the other hand, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the method described in any of the above embodiments.

[0026] On the other hand, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the method described in any of the above embodiments by calling the computer program stored in the memory.

[0027] On the other hand, an embodiment of the present application provides a computer program product, including computer instructions, which, when executed by a processor, implement the method described in any of the above embodiments.

[0028] The embodiment of the present application obtains an image to be processed of a target object; inputs the image to be processed into a target image processing model to obtain a plurality of target feature information to be processed corresponding to the image to be processed, the plurality of target feature information to be processed including: target surgical reference surface feature information, target tissue region feature information and target key point feature information; determines surgical incision position information based on the plurality of target feature information to be processed; generates corresponding surgical incision position prompt information according to the surgical incision position information, the surgical incision position prompt information being used to prompt the surgical incision position for the target object; obtains on-site image information of the target object; and displays the surgical incision position prompt information and the on-site image information according to the plurality of target feature information to be processed, the on-site image information and the surgical incision position information. The method can analyze the features in the image to be processed according to the obtained image to be processed of the target object, determine the surgical incision position information, and display the surgical incision position prompt information and the on-site image information according to the on-site image information of the target object, thereby saving time consumption in determining the surgical incision position for the target object and improving the efficiency of determining the surgical incision position for the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 A schematic diagram of the structure of the image processing system provided in an embodiment of the present application.

[0031] Figure 2a A flowchart of the image processing method provided in an embodiment of the present application.

[0032] Figure 2b A schematic diagram of a scenario for determining the processing result corresponding to the target feature information to be processed provided in an embodiment of the present application.

[0033] Figure 2c A schematic diagram of the positions of some key points on the human head provided in an embodiment of the present application.

[0034] Figure 2d A schematic diagram of a scenario of a method for determining surgical incision location information provided in an embodiment of the present application.

[0035] Figure 2e A schematic diagram of the surgical incision location information provided in an embodiment of the present application.

[0036] Figure 2f Schematic diagram of the relationship between the skin flap and the bone flap provided in an embodiment of the present application.

[0037] Figure 2g A schematic diagram of a scene of an image processing method provided in an embodiment of the present application.

[0038] Figure 3 A flowchart of a data processing method provided in an embodiment of the present application.

[0039] Figure 4 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application.

[0040] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0042] The embodiments of the present application can be applied to various scenarios such as artificial intelligence, medical technology, cloud technology, blockchain, and image processing.

[0043] The embodiments of the present application provide an image processing method, apparatus, device and storage medium. Specifically, the various methods of the embodiments of the present application can be executed by a computer device, wherein the computer device can be a terminal or a server and other devices. The terminal can be a smart phone, a tablet computer, a laptop computer, an intelligent voice interaction device, a smart home appliance, a wearable smart device, an aircraft, an intelligent vehicle terminal and other devices. The terminal can also include a client, which can be a video client, a browser client or an instant messaging client, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.

[0044] For example, when the method is run on a terminal, the terminal may download and install related applications. When the terminal actually runs the aforementioned method, it is used to display a graphical user interface and interact with the user through the graphical user interface. Specifically, the terminal may display the graphical user interface to the user in a variety of ways, for example, it may be rendered and displayed on the terminal's display screen, or the graphical user interface may be presented through holographic projection. For example, the terminal may include a touch screen and a processor, the touch screen being used to present the graphical user interface and receive operation instructions generated by the user acting on the graphical user interface, the processor being used to run the aforementioned image processing method or data processing method, and may also be used to generate a graphical user interface, respond to operation instructions, and control the display of the graphical user interface on the touch screen.

[0045] First, some nouns or terms that appear in the description of the embodiments of this application are explained as follows:

[0046] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0047] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to enable data computing, storage, processing, and sharing. Cloud technology is a general term for network, information technology, integration technology, management platform technology, and application technology, all based on the cloud computing business model. It can form a resource pool that can be used on demand with flexibility and convenience. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identification mark and will need to be transmitted to backend systems for logical processing. Data of varying levels will be processed separately, and data from all industries will require a strong system backend, which can only be achieved through cloud computing.

[0048] A blockchain system can be a distributed system consisting of clients and multiple nodes (any type of computing device connected to the network, such as servers and user terminals) connected through network communications. The nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol that runs on top of the Transmission Control Protocol (TCP). In a distributed system, any machine, such as a server or terminal, can join and become a node. Nodes include hardware, middleware, operating system, and application layers.

[0049] Deep learning (DL), a branch of machine learning, is an algorithm that attempts to achieve high-level abstraction of data using multiple processing layers containing complex structures or multiple nonlinear transformations. Deep learning learns the inherent patterns and representational hierarchies of training sample data. The information gained from this learning process is highly helpful in interpreting data such as text, images, and sound. The ultimate goal of deep learning is to enable machines to acquire human-like analytical learning capabilities and recognize data such as text, images, and sound. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far surpass previous technologies.

[0050] Neural Network (NN): A deep learning model in the field of machine learning and cognitive science that imitates the structure and function of biological neural networks.

[0051] CT: Computed Tomography, also known as electronic computer tomography, is a medical imaging modality that uses precisely collimated X-ray beams, gamma rays, ultrasound, etc., together with extremely sensitive detectors to perform one-by-one cross-sectional scans around a certain part of the human body. It has the characteristics of fast scanning time and clear images, and can be used to detect a variety of diseases.

[0052] MRI (Magnetic Resource Imaging) is a medical imaging modality based on the principle that atomic nuclei with magnetic moments can undergo energy level transitions under the influence of a magnetic field. MRI helps examine brain energy status and cerebral blood flow in patients with epilepsy and is highly valuable in diagnosing degenerative diseases. MRI uses an external high-frequency magnetic field to generate signals from internal substances radiating energy into the surrounding environment. The imaging process is similar to image reconstruction and CT, but MRI does not rely on external radiation, absorption, and reflection, nor on gamma radiation from radioactive substances in the body. Instead, it uses the interaction between the external magnetic field and the object to create images. High-energy magnetic fields are harmless to the human body.

[0053] Skin flap: It is formed by the skin with blood supply and the attached subcutaneous fat tissue.

[0054] Bone flap: The bone formed during craniotomy is called a bone flap, which includes free bone flap and pedicled bone flap.

[0055] The canthus-earline, also known as the ear canthus line, is the line connecting the outer canthus of the eye and the center of the external auditory canal. CT scans of the brain are generally performed in cross-section, with the canthus-earline as the baseline.

[0056] The operative approach refers to the path chosen to expose the primary surgical area, avoiding critical structures and minimizing damage. In neurosurgery, the surgical approach involves progressively exposing the scalp, muscles, skull, and dura mater. The lesion is then reached by creating a fistula through normal brain tissue or through natural interstitial spaces. The incision and bone flap require careful design based on the location and size of the lesion.

[0057] Lesion: The part of the body where a disease occurs.

[0058] The term "midline" is used in diagnostic imaging and is generally used to diagnose brain disorders. Normally, the midline is centered. If it deviates, it could indicate the presence of a space-occupying lesion or elevated intracranial pressure. The brain is typically divided into two hemispheres, the left and right, with the midline between them being the center. Because the two hemispheres are separated by structures such as the falx cerebri and the septum pellucidum between the ventricles, the cerebral hemispheres are symmetrical, and therefore a centered midline is normal.

[0059] Image segmentation: It is the technology and process of dividing an image into several specific regions with unique properties and proposing targets of interest.

[0060] 3D point cloud registration is a key research issue in computer vision and has important applications in a wide range of engineering fields, such as reverse engineering, SLAM, image processing, and pattern recognition. The goal of point cloud registration is to determine the transformation matrix for point clouds with different poses under the same coordinate system. This matrix can be used to accurately align multi-view scanned point clouds, ultimately yielding a complete 3D digital model or scene.

[0061] The U-Net network structure is symmetrical and is named U-Net because it resembles a U-shape. Overall, U-Net is an encoder-decoder structure.

[0062] AR: Augmented Reality, is a technology that cleverly integrates virtual information with the real world. It widely uses a variety of technical means such as multimedia, three-dimensional modeling, registration, intelligent interaction, and sensing to simulate computer-generated virtual information such as text, images, three-dimensional models, music, and videos, and apply them to the real world.

[0063] MarchingCubes: Marching Cubes algorithm. There are two main categories of methods for 3D reconstruction of medical images: 3D surface rendering and 3D volume rendering. Volume rendering can more realistically represent object structure, but due to its high computational complexity, even high-performance computers cannot meet the interactive needs of practical applications. Therefore, surface rendering is currently the mainstream algorithm for 3D reconstruction of medical images. The MarchingCubes algorithm is a classic among surface rendering algorithms and a voxel-level reconstruction method. Due to its simple principle and easy implementation, it has been widely used.

[0064] Please refer to Figure 1 , Figure 1 The image processing system provided in the embodiment of the present application is a schematic diagram of the structure of the image processing system. The image processing system includes a terminal 10 and a server 20, wherein the terminal 10 and the server 20 are connected via a network, such as a wired or wireless network connection.

[0065] The terminal 10 can be used to display a graphical user interface. The terminal 10 can be used to interact with the user through the graphical user interface, for example, by downloading and installing a corresponding client and running it, by calling and running a corresponding applet, or by logging into a website to present a corresponding graphical user interface.

[0066] Optionally, the terminal 10 may be configured to execute the image processing method, specifically:

[0067] Obtaining an image of the target object to be processed;

[0068] Inputting the image to be processed into a target image processing model to obtain a plurality of target feature information to be processed corresponding to the image to be processed, wherein the plurality of target feature information to be processed includes: target surgical reference surface feature information, target tissue region feature information, and target key point feature information;

[0069] Determining surgical incision position information based on the multiple target feature information to be processed;

[0070] generating corresponding surgical incision position prompt information according to the surgical incision position information, wherein the surgical incision position prompt information is used to prompt the surgical incision position for the target object;

[0071] Acquiring on-site image information of the target object;

[0072] The surgical incision position prompt information and the on-site image information are displayed based on the multiple target feature information to be processed, the on-site image information and the surgical incision position information.

[0073] Accordingly, the server 20 may be configured to execute the aforementioned data processing method, specifically to:

[0074] Get a sample image;

[0075] The initial semantic feature extraction unit in the initial image processing model performs feature extraction on the sample image to obtain a plurality of semantic feature information corresponding to the sample image, wherein the plurality of semantic feature information includes: plane semantic feature information, regional semantic feature information and key point semantic feature information;

[0076] Processing the multiple semantic feature information by an initial feature processing unit in the initial image processing model to obtain multiple predicted feature information to be processed corresponding to the multiple semantic feature information, the multiple predicted feature information to be processed including: predicted surgical reference surface feature information, predicted tissue region feature information, and predicted key point feature information;

[0077] Obtaining multiple target feature label information corresponding to the sample image;

[0078] The initial image processing model is trained according to the multiple predicted feature information to be processed and the multiple target feature label information to obtain a target image processing model, wherein the target image processing model includes a target semantic feature extraction unit and a target feature processing unit, which is used to obtain multiple target feature information to be processed corresponding to the image to be processed based on the acquired image to be processed, wherein the multiple target feature information to be processed includes: target surgical reference surface feature information, target tissue region feature information, and target key point feature information;

[0079] The target image processing model is a target image processing model in an image processing method.

[0080] In some optional embodiments of the present application, the image processing system may only include the terminal 10, and the terminal 10 may be used to execute the aforementioned data processing method while having the function of executing the aforementioned image processing method.

[0081] In some optional embodiments of the present application, after the server 20 obtains the target image processing model through training, the server 20 may send the target image processing model to the terminal 10 .

[0082] In other optional embodiments of the present application, the terminal 10 may send the acquired image to be processed to the server 20, so that the server 20 inputs the image to be processed into a target image processing model to obtain a plurality of target feature information to be processed corresponding to the image to be processed; and feeds back the obtained plurality of target feature information to be processed to the terminal 10.

[0083] Optionally, the sample image may be sent from the terminal 10 to the server 20 .

[0084] It should be noted that the order of description of the following embodiments does not limit the priority order of the embodiments.

[0085] Each embodiment of the present application provides an image processing method, which can be executed by a terminal or a server, or by both the terminal and the server. The embodiments of the present application are described using execution by a terminal as an example.

[0086] Figure 2a This is a flow chart of an image processing method provided in an embodiment of the present application, which includes the following steps S201-S206:

[0087] S201, obtaining an image to be processed of a target object;

[0088] Optionally, the aforementioned target object may be a human or other animal, such as a cat, a dog, etc.

[0089] The image to be processed is a CT image or an MRI image. Specifically, the image to be processed is a CT image or an MRI image of the target object's head.

[0090] S202, inputting the image to be processed into a target image processing model to obtain a plurality of target feature information to be processed corresponding to the image to be processed, wherein the plurality of target feature information to be processed includes: target surgical reference surface feature information, target tissue region feature information, and target key point feature information;

[0091] The target image processing model may be a neural network model obtained through machine learning training.

[0092] Specifically, the structure of the target image processing model may include Figure 2b The target semantic feature extraction unit and multiple target feature processing units, such as target feature processing unit 1, target feature processing unit 2, and target feature processing unit 3. Figure 2b The target semantic feature extraction unit in the image extracts a plurality of semantic feature information corresponding to the image to be processed, wherein the plurality of semantic feature information includes: plane semantic feature information, regional semantic feature information and key point semantic feature information.

[0093] Optionally, the multiple semantic feature information is processed by the target feature processing unit in the target image processing model to obtain multiple target feature information to be processed corresponding to the multiple semantic feature information, and the multiple target feature information to be processed includes: target surgical reference surface feature information, target tissue area feature information and target key point feature information.

[0094] Optionally, the target surgical reference surface feature information is used to indicate image information related to the surgical reference surface contained in the image to be processed; the target tissue area feature information is used to indicate image information related to the tissue area contained in the image to be processed; and the target key point feature information is used to indicate image information related to the key point contained in the image to be processed.

[0095] Optionally, the aforementioned surgical reference planes include any one or more of the coronal plane, the sagittal plane, and the auricular canthal plane.

[0096] Optionally, the aforementioned tissue region includes one or more of a brain region, a ventricular region, a lesion region, a blood vessel region, and a skin surface region.

[0097] Optionally, the aforementioned key points are one or more of the following points: 4 orbital points, 4 canthal points, the root of the nose, the tip of the nose, 2 corners of the mouth, 2 midpoints of the zygomatic arches, 2 midpoints of the mandibular condyles, 2 upper edge points of the external auditory canal, 2 midpoints of the external auditory canal, 2 posterior mastoid points, 2 star points, and the external occipital protuberance point.

[0098] Among them, one kind of semantic feature information corresponds to one kind of target feature information to be processed.

[0099] Optionally, various target feature information to be processed may include one or more feature information matrices, each feature information matrix is ​​a three-dimensional matrix, and the value of each matrix element in the three-dimensional matrix is ​​a feature value.

[0100] Optionally, the matrix element comprises a value of the matrix element.

[0101] Optionally, the matrix element includes the value of the matrix element and the position information of the matrix element.

[0102] When the target feature information to be processed is the target surgical reference surface feature information, the feature information matrix included in the target surgical reference surface feature information may be any one or more of the following: a coronal plane feature information matrix, a sagittal plane feature information matrix, and an auricular canthal plane feature information matrix.

[0103] When the target feature information to be processed is target tissue area feature information, the feature information matrix contained in the target tissue area feature information can be any one or more of the following: brain feature information matrix, blood vessel feature information matrix, ventricle feature information matrix, lesion feature information matrix, and skin surface feature information matrix.

[0104] When the target feature information to be processed is target key point feature information, the feature information matrix contained in the target key point feature information can be the key point feature information matrix of any one of the following 25 key points: 4 orbital points, 4 canthal points, nasal root, nose tip, 2 corners of the mouth, 2 zygomatic arch midpoints, 2 mandibular joint condyle midpoints, 2 external auditory canal upper edge points, 2 external auditory canal midpoints, 2 mastoid posterior points, 2 star points, and external occipital protuberance point.

[0105] Optionally, Figure 2c This is a schematic diagram of the positions of some of the aforementioned 25 key points on the human head.

[0106] For the eigenvalue of each matrix element in a characteristic information matrix, the eigenvalue corresponds to position information of the eigenvalue in the characteristic information matrix.

[0107] Optionally, the image to be processed is a three-dimensional image, and for each feature information matrix, voxels in the image to be processed correspond one-to-one to matrix elements in the feature information matrix.

[0108] Optionally, the coordinate system of the image to be processed is a third coordinate system, and each voxel in the image to be processed corresponds to its coordinate position information in the third coordinate system; the coordinate system of each feature information matrix is ​​the first coordinate system, and the coordinate position information of each voxel in the image to be processed in the third coordinate system is the same as the coordinate position information of the position information of the matrix element corresponding to the voxel in the feature information matrix in the first coordinate system, wherein the third coordinate system and the first coordinate system are the same coordinate system.

[0109] Optionally, the characteristic value of each matrix element in the aforementioned characteristic information matrix is ​​a probability value. Specifically, the characteristic information matrix may be three-dimensional heat map information. The characteristic value of the matrix element in the aforementioned characteristic information matrix is ​​the value of the voxel in the three-dimensional heat map information. The characteristic value of the matrix element may refer to the probability value of the matrix element belonging to the preset object corresponding to the characteristic information matrix. Optionally, the larger the characteristic value of the matrix element in the characteristic information matrix, the larger the heat map color value. The matrix elements in the characteristic information matrix are voxels in the three-dimensional heat map information.

[0110] Optionally, when the feature information matrix is ​​the feature information matrix in the target surgical reference plane feature information, the preset object corresponding to the feature information matrix is ​​a plane. For example, when the feature information matrix is ​​the coronal plane feature information matrix, the preset object corresponding to the feature information matrix is ​​the coronal plane. When the feature information matrix is ​​the sagittal plane feature information matrix, the preset object corresponding to the feature information matrix is ​​the sagittal plane. When the feature information matrix is ​​the canthus plane feature information matrix, the preset object corresponding to the feature information matrix is ​​the canthus plane.

[0111] Optionally, when the feature information matrix is ​​a feature information matrix in the target tissue region feature information, the preset object corresponding to the feature information matrix is ​​the tissue region. For example, when the feature information matrix is ​​a brain feature information matrix, the preset object corresponding to the feature information matrix is ​​the brain region. When the feature information matrix is ​​a ventricle feature information matrix, the preset object corresponding to the feature information matrix is ​​the ventricle region. When the feature information matrix is ​​a lesion feature information matrix, the preset object corresponding to the feature information matrix is ​​the lesion region. When the feature information matrix is ​​a vascular feature information matrix, the preset object corresponding to the feature information matrix is ​​the vascular region. When the feature information matrix is ​​a skin surface feature information matrix, the preset object corresponding to the feature information matrix is ​​the skin surface region.

[0112] Optionally, when the feature information matrix is ​​a feature information matrix in the target key point feature information, the preset object corresponding to the feature information matrix is ​​the key point; when the feature information matrix is ​​a left eye orbital point feature information matrix, the preset object corresponding to the feature information matrix is ​​the left eye orbital point; when the feature information matrix is ​​a right eye orbital point feature information matrix, the preset object corresponding to the feature information matrix is ​​the right eye orbital point, and so on.

[0113] S203, determining surgical incision location information based on the multiple target feature information to be processed;

[0114] Optionally, in the aforementioned S203, determining the surgical incision location information based on the multiple target feature information to be processed includes the following S2031-S2032:

[0115] S2031. Preprocessing the plurality of target feature information to be processed to obtain a plurality of processing results, wherein the plurality of processing results include: surgical reference surface parameter information, tissue region information, and key point information; the tissue region information includes: skin surface region information and skin internal region information; the skin internal region information includes: blood vessel region information and lesion region information;

[0116] Optionally, one type of target feature information to be processed corresponds to one processing result.

[0117] Specifically, in the aforementioned S2031, the pre-processing of the various target feature information to be processed is performed to obtain various processing results, which may include:

[0118] For each type of target feature information to be processed among the multiple types of target feature information to be processed, preprocessing is performed on the target feature information to be processed to obtain a processing result corresponding to the target feature information to be processed.

[0119] When the target feature information to be processed is target surgical reference surface feature information, the processing result corresponding to the target surgical reference surface feature information is surgical reference surface parameter information.

[0120] When the target characteristic information to be processed is target tissue region characteristic information, the processing result corresponding to the target tissue region characteristic information is tissue region information.

[0121] When the target feature information to be processed is target key point feature information, the processing result corresponding to the target key point feature information is the key point information.

[0122] For details, please refer to Figure 2b As shown, Figure 2b A schematic diagram of a scenario for determining a processing result corresponding to target feature information to be processed provided in an embodiment of the present application.

[0123] Optionally, when a type of target feature information to be processed includes a feature information matrix, the target feature information to be processed is preprocessed to obtain a processing result corresponding to the target feature information to be processed, including: preprocessing the feature information matrix included in the target feature information to be processed to obtain a processing result corresponding to the feature information matrix; and using the processing result corresponding to the feature information matrix as the processing result corresponding to the target feature information to be processed.

[0124] Optionally, when the preset object corresponding to the characteristic information matrix is ​​a plane, or the preset object corresponding to the characteristic information matrix is ​​a tissue region, preprocessing the characteristic information matrix included in the target characteristic information to be processed to obtain a processing result corresponding to the characteristic information matrix includes:

[0125] Obtaining, in a feature information matrix included in the target feature information to be processed, a first portion of matrix elements whose feature values ​​are greater than a second preset threshold corresponding to a preset object corresponding to the feature information matrix;

[0126] A processing result corresponding to the feature information matrix is ​​determined according to the first part of matrix elements.

[0127] Optionally, when the preset object corresponding to the characteristic information matrix is ​​a plane, such as a coronal plane, a sagittal plane, or auricular canthal plane, determining the processing result corresponding to the characteristic information matrix according to the first part of the matrix elements includes:

[0128] Fitting the set of coordinate position information of the first part of matrix elements based on the least squares method to obtain plane parameter information of the predicted object corresponding to the set of coordinate position information;

[0129] The plane parameter information of the prediction object is used as the processing result.

[0130] The surgical reference plane parameter information includes plane parameter information of the predicted object, and the predicted object may include a coronal plane, a sagittal plane, or auricular canthal plane.

[0131] The plane parameter information includes the origin information of the plane and the angle information of the plane relative to the preset coordinate axis in the aforementioned first coordinate system, wherein the preset coordinate axis may include at least one of the horizontal axis, the longitudinal axis and the vertical axis.

[0132] Optionally, when the preset object corresponding to the feature information matrix is ​​a tissue region, such as a brain region, a ventricle region, a lesion region, a blood vessel region, or a skin surface region, determining the processing result corresponding to the feature information matrix according to the first part of the matrix elements includes: using a set of coordinate position information of the first part of the matrix elements as region information of the predicted object corresponding to the feature information matrix;

[0133] The region information of the prediction object is used as a processing result.

[0134] The tissue region information includes region information of the prediction object.

[0135] Among them, when the prediction object is the skin surface area, the regional information of the prediction object is the skin surface area information; when the prediction object is the vascular area, the regional information of the prediction object is the vascular area information; when the prediction object is the lesion area, the regional information of the prediction object is the lesion area information; when the prediction object is the brain area, the regional information of the prediction object is the brain area information; when the prediction object is the ventricular area, the regional information of the prediction object is the ventricular area information.

[0136] Optionally, the set of coordinate position information of the first part of matrix elements is a set of coordinate position information of each matrix element in the first part of matrix elements.

[0137] Optionally, when the preset object corresponding to the characteristic information matrix is ​​a tissue area, the processing result corresponding to the characteristic information matrix may also include the characteristic values ​​of each matrix element in the first part of the matrix elements, and the processing result corresponding to the characteristic information matrix may include the characteristic values ​​of each matrix element in the first part of the matrix elements, which may also be a preset value.

[0138] Optionally, when the preset object corresponding to the feature information matrix is ​​a key point, such as a left orbital point, a right orbital point, etc., preprocessing the feature information matrix included in the target feature information to be processed to obtain a processing result corresponding to the feature information matrix includes:

[0139] The target coordinate position information of the target matrix element with the largest corresponding characteristic value among the multiple coordinate position information corresponding to all matrix elements in the characteristic information matrix is ​​used as the coordinate position information of the predicted object corresponding to the characteristic information matrix; and the coordinate position information of the predicted object is used as the processing result corresponding to the characteristic information matrix.

[0140] Among them, when the feature information matrix is ​​three-dimensional heat map information, the corresponding target matrix element with the largest feature value is the three-dimensional heat map information, that is, the voxel with the peak value in the probability map information.

[0141] Optionally, the multiple coordinate position information corresponding to all matrix elements is a collection of coordinate position information of each matrix element in all matrix elements.

[0142] Optionally, when the preset object corresponding to the feature information matrix is ​​a key point, the processing result corresponding to the feature information matrix may further include the feature values ​​of the aforementioned target matrix elements.

[0143] The aforementioned key point information includes the coordinate position information of the predicted object.

[0144] The preset object corresponding to the characteristic information matrix is ​​consistent with the type of the predicted object corresponding to the characteristic information matrix. If the preset object corresponding to the characteristic information matrix is ​​the coronal plane, then the predicted object corresponding to the characteristic information matrix is ​​also the coronal plane.

[0145] Optionally, when a type of target feature information to be processed includes multiple feature information matrices, the target feature information to be processed is preprocessed to obtain a processing result corresponding to the target feature information to be processed, including: for each feature information matrix in the multiple feature information matrices included in the target feature information to be processed, preprocessing the feature information matrix to obtain a processing result corresponding to the feature information matrix, and then obtaining a processing result corresponding to the target feature information to be processed.

[0146] Optionally, each feature information matrix corresponds to a preset object.

[0147] Among them, the method of preprocessing the characteristic information matrix to obtain the processing result corresponding to the characteristic information matrix can be found in the above content and will not be repeated here.

[0148] S2032. Determine surgical incision location information based on the multiple processing results.

[0149] Optionally, in the aforementioned S2032, determining the surgical incision location information according to the multiple processing results includes the following S321-S324:

[0150] S321, determining the centroid information of the lesion area information according to the lesion area information;

[0151] The centroid information may be the coordinate position information of the centroid of the lesion region corresponding to the lesion region information, and / or the matrix element at the coordinate position information of the centroid of the lesion region corresponding to the lesion region information.

[0152] The method of determining the centroid information of the lesion area information based on the lesion area information can be found in the relevant technology and will not be repeated here.

[0153] S322, determining a plurality of surgical reference plane information according to the surgical reference plane parameter information and the key point information, wherein the plurality of surgical reference plane information includes: coronal plane information, sagittal plane information, and auricular canthal plane information;

[0154] Optionally, the surgical reference plane parameter information includes sagittal plane parameter information. In S322, determining a plurality of surgical reference plane information according to the surgical reference plane parameter information and the key point information includes the following steps S3221-S3225:

[0155] S3221. Determine sagittal plane information according to the sagittal plane parameter information;

[0156] Optionally, the corresponding preset object may be a feature information matrix of a sagittal plane, and the set of matrix elements of the sagittal plane corresponding to the parameter information of the sagittal plane, whose coordinate position information is included in the sagittal plane, may be used as the sagittal plane information.

[0157] S3222: Determine the parameter information and the ear canthal surface information of the ear canthus surface according to the position information of the middle point of the external auditory canal and the position information of the external canthus point in the key point information;

[0158] Specifically, the ear canthus surface can be obtained based on the least squares fitting according to the position information of the two external auditory canal midpoints and the position information of the two external canthus points in the key point information, and the parameter information of the ear canthus surface can be determined.

[0159] Optionally, the corresponding preset object can be a feature information matrix of the canthus surface, and the coordinate position information of the matrix elements of the canthus surface corresponding to the parameter information of the canthus surface can be used as the canthus surface information.

[0160] S3223: Determine the coronal plane parameter information based on the external auditory canal midpoint position information in the key point information and the canthal plane parameter information;

[0161] Among them, the position information passing through the midpoints of the two external auditory canals, that is, the coordinate position information of the midpoints of the two external auditory canals, and the plane perpendicular to the canthus plane can be used as the coronal plane to determine the parameter information of the coronal plane.

[0162] S3224, determining coronal plane information according to the coronal plane parameter information;

[0163] S3225. Use the sagittal plane information, the canthal plane information, and the coronal plane information as multiple surgical reference plane information.

[0164] Optionally, the corresponding preset object may be a feature information matrix of a coronal plane, and the set of matrix elements of the coronal plane corresponding to the parameter information of the coronal plane, whose coordinate position information is included in the coronal plane, may be used as the coronal plane information.

[0165] S323, determining a plurality of first candidate incision key point information based on the centroid information and the plurality of surgical reference surface information;

[0166] Optionally, in S323, determining a plurality of first candidate incision key point information based on the centroid information and the plurality of surgical reference surface information includes the following S3231-S3234:

[0167] S3231, translating the sagittal plane information along the sagittal plane normal direction, and when the sagittal plane information is translated to include the centroid information, using the intersection line of the sagittal plane information and the skin surface area information as sagittal plane intersection parallel line information;

[0168] S3232, translating the coronal plane information along the coronal plane normal direction, and when the coronal plane information is translated to include the centroid information, using the intersection line of the coronal plane information and the skin surface area information as the coronal plane intersection parallel line information;

[0169] S3233: translating the auricular canthal surface information along the normal direction of the auricular canthal surface. When the auricular canthal surface information is translated to include the centroid information, using the intersection line of the auricular canthal surface information and the skin surface area information as the auricular canthal surface intersection line parallel line information.

[0170] S3234. Determine the multiple first candidate incision key point information based on the sagittal plane intersection parallel line information, the coronal plane intersection parallel line information, and the auricular canthus plane intersection parallel line information.

[0171] Optionally, in the aforementioned S3234, determining the plurality of first candidate incision key point information based on the sagittal plane intersection parallel line information, the coronal plane intersection parallel line information, and the auricular canthal plane intersection parallel line information includes the following S241-S244:

[0172] S241, taking the intersection of the coronal plane intersection parallel line information and the sagittal plane intersection parallel line information as the second candidate incision key point information, that is, Figure 2d The sagittal-coronal point P1;

[0173] Optionally, the intersection of the coronal plane intersection parallel line information and the sagittal plane intersection parallel line information is used as the second alternative incision key point information, including: taking the coordinate position information of the matrix elements of the set of matrix elements contained in the coronal plane intersection parallel line information and the set of matrix elements contained in the sagittal plane intersection parallel line information that overlap as the second alternative incision key point information.

[0174] Optionally, the intersection of the coronal plane intersection parallel line information and the sagittal plane intersection parallel line information is used as the second alternative incision key point information, including: using the set of matrix elements contained in the coronal plane intersection parallel line information and the set of matrix elements contained in the sagittal plane intersection parallel line information that overlap as the second alternative incision key point information.

[0175] S242, taking the intersection of the sagittal plane intersection parallel line information and the auricular canthus plane intersection parallel line information as the third candidate incision key point information, that is, Figure 2d the auricular canthus-sagittal point P4;

[0176] Optionally, the intersection of the sagittal plane intersection parallel line information and the auricular canthus plane intersection parallel line information is used as the third alternative incision key point information, including: taking the coordinate position information of the matrix elements of the set of matrix elements contained in the sagittal plane intersection parallel line information and the set of matrix elements contained in the auricular canthus plane intersection parallel line information that overlap as the third alternative incision key point information.

[0177] Optionally, the intersection of the sagittal plane intersection parallel line information and the ear canthus intersection parallel line information is used as the third alternative incision key point information, including: using the set of matrix elements contained in the sagittal plane intersection parallel line information and the set of matrix elements contained in the ear canthus intersection parallel line information that overlap as the third alternative incision key point information.

[0178] S243, taking the intersection of the coronal plane intersection parallel line information and the auricular canthus plane intersection parallel line information as the fourth candidate incision key point information, that is, Figure 2d The auricular canthus-coronal point P3;

[0179] Optionally, the intersection of the coronal plane intersection parallel line information and the auricular canthal plane intersection parallel line information is used as the fourth alternative incision key point information, including: taking the coordinate position information of the matrix elements of the set of matrix elements contained in the coronal plane intersection parallel line information and the set of matrix elements contained in the auricular canthal plane intersection parallel line information that overlap as the fourth alternative incision key point information.

[0180] Optionally, the intersection of the coronal plane intersection parallel line information and the ear canthus intersection parallel line information is used as the fourth alternative incision key point information, including: using the set of matrix elements contained in the coronal plane intersection parallel line information and the set of matrix elements contained in the ear canthus intersection parallel line information that overlap as the fourth alternative incision key point information.

[0181] Among them, the second candidate incision key point information, the third candidate incision key point information, and the fourth candidate incision key point information are the first candidate incision key point information of the top, forehead, and temporal parts respectively.

[0182] S244: Use the second candidate incision key point information, the third candidate incision key point information, and the fourth candidate incision key point information as the plurality of first candidate incision key point information.

[0183] In some optional embodiments of the application, the matrix element closest to the centroid information in the skin surface area information, or the coordinate position information of the matrix element, can also be used as one of the first candidate incision key point information among the multiple first candidate incision key point information, such as Figure 2d The closest point P2 on the body surface.

[0184] S324. Determine surgical incision position information using the centroid information, the skin internal area information, the skin surface area information, and the plurality of first candidate incision key point information.

[0185] Optionally, in the aforementioned S324, determining the surgical incision position information using the centroid information, the skin internal area information, the skin surface area information, and the plurality of first candidate incision key point information includes the following S01-S02:

[0186] S01. For each piece of first candidate incision key point information among the plurality of first candidate incision key point information, determine first connection information between the centroid information and the first candidate incision key point information, and obtain a plurality of first connection information corresponding to the plurality of first candidate incision key point information;

[0187] The first connection information includes a plurality of coordinate position information or a plurality of matrix elements.

[0188] S02. Determine surgical incision location information based on the plurality of first connection line information, the skin internal area information, and the skin surface area information.

[0189] Optionally, in the aforementioned S02, determining the surgical incision location information based on the plurality of first connection line information, the skin internal area information, and the skin surface area information includes the following S021-S022:

[0190] S021. For each first piece of line information among the plurality of first piece of line information, when the first piece of line information includes coordinate position information of target area information in the skin internal area information, updating first candidate incision key point information corresponding to the first piece of line information; when the first piece of line information does not include coordinate position information of the target area information, not updating the first candidate incision key point information corresponding to the first piece of line information, thereby obtaining a plurality of fifth candidate incision key point information corresponding to the plurality of first piece of line information, wherein the target area information includes the blood vessel area information;

[0191] Updating the first candidate incision key point information corresponding to the first connection information includes:

[0192] Taking the first candidate incision key point information corresponding to the first connection line information as the center, obtaining first candidate point information belonging to the skin surface area information at a preset distance from the center;

[0193] Determine the second connection line information between the centroid information and the first alternative point information. When the second connection line information does not contain the coordinate position information belonging to the target area information, use the first alternative point information as the first alternative incision key point information corresponding to the updated first connection line information. When the second connection line information contains the coordinate position information belonging to the target area information, update the preset distance and obtain the second alternative point information belonging to the skin surface area information with a distance from the center that is the updated preset distance; update the first alternative incision key point information corresponding to the first connection line information according to the second alternative point information.

[0194] Optionally, the aforementioned skin internal area information also includes: ventricle area information, brain area information or one or more area information.

[0195] Optionally, the aforementioned target region information may be one or more region information selected from the aforementioned vascular region information, ventricular region information, and brain region information.

[0196] Optionally, the aforementioned target area information may also include a preset marked area, which may be a preset area marked by relevant personnel through the terminal interface, such as an important functional area, a neural area, etc. Optionally, relevant personnel may use software such as ITK-Snap or 3DSlicer to manually outline the area used to indicate the preset marked area.

[0197] S022. Determine surgical incision position information based on the plurality of fifth candidate incision key point information, the lesion area information, and the skin surface area information.

[0198] Optionally, in the aforementioned S022, determining the surgical incision position information based on the plurality of fifth candidate incision key point information, the lesion area information, and the skin surface area information includes S41-S42:

[0199] S41, selecting any one of the plurality of fifth candidate incision key point information as reference point information, or obtaining a selection instruction from the subject for selecting reference point information from the plurality of fifth candidate incision key point information, and determining the reference point information according to the selection instruction;

[0200] Optionally, the subject may be a medical staff member or other relevant personnel, and the subject may select the reference point information through an operation instruction, i.e., a selection instruction, from a plurality of fifth alternative incision key point information displayed in the operation interface of the terminal.

[0201] Optionally, the reference point information may not be included in the aforementioned multiple fifth alternative incision key point information and may be manually selected by relevant personnel. This application does not impose any limitation on this.

[0202] S42. Determine the surgical incision position information based on the reference point information, the lesion area information, and the skin surface area information.

[0203] Optionally, the surgical incision location information includes bone flap incision location information. In the aforementioned S42, determining the surgical incision location information based on the reference point information, the lesion area information, and the skin surface area information includes the following S421-S423:

[0204] S421, determining projection area information of the lesion area information in the skin surface area information, using the direction from the centroid information of the lesion area information to the reference point information as a projection direction;

[0205] Optionally, the direction from the centroid information of the lesion area information to the reference point information is used as the projection direction to determine the projection area information of the lesion area information in the skin surface area information, including: for each coordinate position information in the lesion area information, obtaining the coordinate position information in the skin surface area information, the line connecting the coordinate position information is parallel to the line connecting the centroid information and the reference point information, and obtaining the projection area information of the lesion area information in the skin surface area information.

[0206] It should be noted that each matrix element in the feature information matrix in this solution corresponds to coordinate position information, and the processing of the coordinate position information may refer to the processing of the matrix element at the coordinate position information, that is, the voxel.

[0207] S422, determining corresponding bone flap incision edge line information according to the projection area information;

[0208] The bone flap incision edge line information may be multiple, for example, 4 edge line information. The bone flap incision edge line information is rectangular edge line information, and the area enclosed by the bone flap incision edge line information includes the projection area information.

[0209] Specifically, any one of the edge line information in a group of mutually parallel edge line information in the bone flap incision edge line information is parallel to the aforementioned coronal plane intersection line information, and any one of the edge line information in another group of mutually parallel edge line information is parallel to the aforementioned auricular canthus plane intersection line information.

[0210] Optionally, each coordinate position in the projection area information corresponds to multiple distances, each of which is a distance from the coordinate position to the plurality of bone flap incision edge lines. The bone flap incision edge lines are such that the minimum distance among all distances corresponding to all coordinate position information in the projection area information is greater than a preset distance, wherein the preset distance is 1 mm or 2 mm.

[0211] S423. Use the bone flap incision edge line information as the bone flap incision position information.

[0212] Optionally, the surgical incision position information also includes skin flap incision position information, and the method further includes: determining the skin flap incision position information based on the bone flap incision position information.

[0213] In some optional embodiments of the present application, determining the skin flap incision position information according to the bone flap incision position information includes:

[0214] The skin flap incision position information is generated according to the preset shape information and the bone flap incision position information.

[0215] The preset shape information can be any one or more of the following: "L-shaped", "hoof-shaped", or "straight line", see Figure 2e shown.

[0216] Optionally, the method further includes:

[0217] Acquire the lengths of multiple line segments of information of multiple line segments that pass through the centroid information of the lesion area and whose endpoints are located on the surface of the lesion area;

[0218] Get the longest line segment length among the multiple line segment lengths.

[0219] If the longest line segment length is greater than the preset line segment length, the preset shape information defaults to "horse-shaped", and if it is less than the preset line segment length, the default is "L-shaped", wherein the preset line segment length can be 1 cm.

[0220] Optionally, the aforementioned preset shape information can also be selected by relevant personnel.

[0221] When the preset shape information is L-shaped or hoof-shaped, generating the skin flap incision position information according to the preset shape information and the bone flap incision position information includes:

[0222] Obtaining central position information of the bone flap incision position information;

[0223] Taking the central position information as the center, the area enclosed by the bone flap incision position information is enlarged according to a preset enlargement ratio to obtain the incision position information to be processed;

[0224] The skin flap incision position information is generated according to the preset shape information and the incision position information to be processed.

[0225] In some optional embodiments of the present application, generating the flap incision position information according to the preset shape information and the incision position information to be processed includes:

[0226] Part of the edge line information in the incision position information to be processed is deleted according to the preset shape information to obtain the skin flap incision position information.

[0227] The line information of the endpoints of the skin flap incision position information does not include any coordinate position information in the bone flap incision position information.

[0228] Specifically, the incision position information to be processed includes four edge line information, and the incision position information to be processed encloses a rectangular area. When the preset shape information is L-shaped, part of the edge line information in the incision position information to be processed is deleted according to the preset shape information, including:

[0229] The two edge line information having a common endpoint in the incision position information to be processed are deleted.

[0230] When the preset shape information is a horseshoe shape, deleting part of the edge line information in the incision position information to be processed according to the preset shape information includes:

[0231] Delete any edge line information in the incision position information to be processed.

[0232] When the preset shape information is a straight line, generating the skin flap incision position information according to the preset shape information and the bone flap incision position information includes:

[0233] Obtaining central position information of the bone flap incision position information;

[0234] Generate a line segment information that passes through the center position information, is parallel to any edge line in the bone flap incision position information, and has a preset length;

[0235] The line segment information is used as the flap incision position information.

[0236] It should be noted that the aforementioned preset length corresponds to the length of any one or two edge line information in the bone flap incision position information, and the aforementioned preset length is greater than the edge line information in the bone flap incision position information that is parallel to the line segment information.

[0237] In some optional embodiments of the present application, the aforementioned flap incision position information can also be determined according to the instructions of relevant personnel. For example, in order to meet the patient's aesthetic needs, the doctor will also set the flap incision position information according to the hairline. The doctor can manually add the hairline as a flap.

[0238] In some optional embodiments of the present application, the shape of the bone flap may be other shapes, such as a circle, in addition to the aforementioned rectangle or square.

[0239] It should be noted that no matter what shape the bone flap is, the area enclosed by the bone flap incision position information must include all the coordinate position information in the projection area information, and all the coordinate position information in the area enclosed by the bone flap incision position information must be on the same side of the line connecting the end points of the skin flap.

[0240] S204: Generate corresponding surgical incision position prompt information according to the surgical incision position information, wherein the surgical incision position prompt information is used to prompt the surgical incision position for the target object;

[0241] Optionally, the aforementioned surgical incision position prompt information may be a colored line for indicating surgical incision position information.

[0242] In some optional embodiments of the present application, before displaying the surgical incision position prompt information, the above method also includes smoothing the bone flap incision position information and / or the skin flap incision position information. Optionally, the bone flap incision position information in a rectangular area can be smoothed into bone flap incision position information in a rounded rectangular or circular area, and the aforementioned horseshoe-shaped skin flap incision position information can be smoothed into Figure 2f Information on the location of the semicircular flap incision. Figure 2f This is a schematic diagram of the relationship between the skin flap and the bone flap when the shape of the skin flap is semicircular, wherein the line connecting the endpoints A1 and A2 of the skin flap is the folding line, and the bone flap is on the same side of the folding line.

[0243] S205, obtaining on-site image information of the target object;

[0244] S206 , displaying the surgical incision position prompt information and the on-site image information based on the multiple target feature information to be processed, the on-site image information, and the surgical incision position information.

[0245] In the aforementioned S206, displaying the surgical incision position prompt information and the on-site image information according to the multiple target feature information to be processed, the on-site image information, and the surgical incision position information includes the following S91-S96:

[0246] S91, preprocessing the plurality of target feature information to be processed to obtain a plurality of processing results, the plurality of processing results including: surgical reference surface parameter information, tissue region information, and key point information, the tissue region information including: skin surface region information and skin internal region information, the skin internal region information including: blood vessel region information and lesion region information;

[0247] S92. Acquire depth image information corresponding to the scene image information, wherein the depth image information may be acquired by a depth camera;

[0248] S93: Register the skin surface area information and the depth image information based on a preset registration algorithm to obtain a coordinate transformation relationship between a first coordinate system corresponding to the skin surface area information and a second coordinate system corresponding to the depth image information;

[0249] S94, determining a first display position of the surgical incision position prompt information in a display screen corresponding to the on-site image information according to the surgical incision position information and the coordinate conversion relationship;

[0250] Specifically, the coordinate position information of the surgical incision position information in the first coordinate system can be determined, and the coordinate position information of the surgical incision position information in the second coordinate system, that is, the first display position, can be determined based on the coordinate conversion relationship, so that the surgical incision position information is displayed on the head of the target object.

[0251] S95, displaying the on-site image information;

[0252] S96: Display the surgical incision position prompt information in the display screen of the on-site image information based on the first display position.

[0253] Specifically, see Figure 2g As shown, Figure 2g A schematic diagram of a scene of an image processing method provided in an embodiment of the present application, wherein the depth image information may be specifically point cloud information, Figure 2g The video stream in is the on-site image information captured by the camera. After the surgical incision position prompt information is displayed in the display screen of the on-site image information based on the first display position, the current screen is an AR image.

[0254] The principle of the registration algorithm can be found in related technologies.

[0255] Optionally, the camera used to determine the depth image information and the camera used to determine the scene image information can be the same camera or different cameras, but the coordinate systems of the different cameras must be the same coordinate system so that the coordinate systems based on the depth image information and the scene image information coincide with each other.

[0256] Optionally, the skin surface area information can be converted into a Surface model based on the MarchingCube algorithm, and then aligned with the 3D face point cloud information (i.e., depth image information) scanned by the depth camera. The alignment method can use the Iterative Closest Point (ICP) algorithm, which is a point cloud matching algorithm for automatic alignment or manual rotation and translation alignment by relevant personnel.

[0257] Optionally, the aforementioned on-site image information refers to on-site image information of the target subject. Specifically, the aforementioned surgical incision location prompt information and the on-site image information are displayed as an AR display. The on-site image information is a real-life image of the target subject's head. This allows relevant personnel, such as doctors, to perform craniotomy surgery through a screen or head-mounted device. Relevant personnel can also optionally adjust the surgical incision location information based on the displayed AR display.

[0258] Specifically, the surgical incision position prompt information is displayed on the head in the real head image.

[0259] In the aforementioned S96, displaying the surgical incision position prompt information in the display screen of the on-site image information based on the first display position includes:

[0260] In the display screen corresponding to the on-site image information, the surgical incision position prompt information is displayed at the first display position.

[0261] In some optional embodiments of the present application, the method further includes:

[0262] using an intersection line between the sagittal plane information and the skin surface area information as sagittal plane intersection line information;

[0263] using the intersection line of the coronal plane information and the skin surface area information as coronal plane intersection line information;

[0264] using the intersection line of the ear canthus surface information and the skin surface area information as the ear canthus surface intersection line information;

[0265] Generating sagittal plane intersection line prompt information, coronal plane intersection line prompt information, and auricular canthal plane intersection line prompt information respectively according to the sagittal plane intersection line information, the coronal plane intersection line information, and the auricular canthal plane intersection line information;

[0266] The sagittal plane intersection line prompt information, the coronal plane intersection line prompt information, and the auricular canthus plane intersection line prompt information are displayed.

[0267] Optionally, the endpoints of the sagittal plane intersection line information are the external occipital protuberance and the nasion.

[0268] The location of the sagittal intersection information can be found in Figure 2d As shown, Figure 2d A schematic diagram of a scenario for a method for determining surgical incision location information.

[0269] Optionally, the endpoints of the ear canthus-face intersection line information are the midpoint of the external auditory canal and the external canthus point.

[0270] The location of the ear canthus-face intersection line can be found in Figure 2d shown.

[0271] Optionally, the endpoints of the coronal plane intersection line information may be the midpoints of the external auditory canals on both sides.

[0272] The location of the coronal intersection information can be found in Figure 2d shown.

[0273] Optionally, the skin surface area information and each plane can be calculated based on the VTKvtkCutter function, such as

[0274] The intersection line information of the coronal plane, the sagittal plane, and the canthal plane, namely, the aforementioned coronal plane intersection line information, the sagittal plane intersection line information, and the canthal plane intersection line information.

[0275] The positions of the aforementioned sagittal plane intersection parallel line information, the aforementioned coronal plane intersection parallel line information, and the aforementioned auricular canthal plane intersection parallel line information can be found in Figure 2d shown.

[0276] Specifically, the sagittal plane intersection parallel line information, the coronal plane intersection parallel line information, and the auricular canthal plane intersection parallel line information are also displayed on the head in the real head image.

[0277] Optionally, the above method also includes: determining the second display position of any one or more first information in the display screen corresponding to the on-site image information based on the sagittal plane intersection line information, the auricular canthus plane intersection line information, and the coronal plane intersection line information and the coordinate transformation relationship; and displaying any one or more first information in the display screen corresponding to the on-site image information based on the second display position.

[0278] Optionally, the above method also includes: according to the sagittal plane intersection parallel line information, the coronal plane intersection parallel line information, the auricular canthal plane intersection parallel line information, multiple first alternative incision key point information, the coordinate position information of 25 key points, sagittal plane information, auricular canthal plane information, and any one or more second information in the coronal plane information, and the coordinate conversion relationship, determining the third display position of any one or more second information in the display screen corresponding to the on-site image information; based on the third display position, displaying any one or more second information in the display screen corresponding to the on-site image information.

[0279] The aforementioned first display position, second display position, and third display position are all positions in the second coordinate system.

[0280] The solution of this application can automatically separate multiple tissue regions, such as the brain region, ventricle region, lesion region, blood vessel region, and skin surface region, and can also automatically generate bone flap and skin flap shapes. The lesion region can be a hematoma region or a tumor region.

[0281] It should be noted that in the specific implementation of this application, information related to user information, such as medical images, etc., when the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0282] The embodiment of the present application obtains an image to be processed of a target object; inputs the image to be processed into a target image processing model to obtain multiple target feature information to be processed corresponding to the image to be processed, the multiple target feature information to be processed including target surgical reference surface feature information, target tissue region feature information, and target key point feature information; determines surgical incision position information based on the multiple target feature information to be processed; generates corresponding surgical incision position prompt information based on the surgical incision position information, the surgical incision position prompt information being used to prompt the surgical incision position for the target object; obtains on-site image information of the target object; and displays the surgical incision position prompt information and the on-site image information based on the multiple target feature information to be processed, the on-site image information, and the surgical incision position information. The solution can analyze the features in the image to be processed based on the obtained image to be processed of the target object to determine the surgical incision position information, and display the surgical incision position prompt information and the on-site image information based on the on-site image information of the target object. This can save time in determining the surgical incision position for the target object, improve the efficiency of determining the surgical incision position for the target object, and facilitate relevant personnel to adjust the surgical incision position information.

[0283] The present application also provides a data processing method. Figure 3 Schematic diagram of the data processing method, which includes the following S301-S305:

[0284] S301, obtaining a sample image;

[0285] The sample images are multiple CT images or multiple MRI images of multiple sample objects.

[0286] S302, performing feature extraction on the sample image by an initial semantic feature extraction unit in an initial image processing model to obtain a plurality of semantic feature information corresponding to the sample image, wherein the plurality of semantic feature information includes: plane semantic feature information, regional semantic feature information, and key point semantic feature information;

[0287] Among them, the network structure of the initial semantic feature extraction unit is a 3DU-net network.

[0288] Optionally, the 3DU-net network unit primarily consists of an encoder, a decoder, and skip connections. The encoder extracts image features layer by layer. Its structure consists of four stages, each of which includes two 3×3 convolutions and a downsampling layer using 2×2 max pooling. With each stage, the output feature map is scaled down by half, and the channel dimension is doubled. The decoder restores image information layer by layer. Its structure is symmetrical to the encoder and also consists of four stages, each of which includes two 3×3 convolutions and an upsampling layer using 2×2 deconvolution. With each stage, the output feature map is scaled up by one, and the channel dimension is halved. The encoder and decoder are connected via two 3×3 convolutions. Each 3×3 convolution in the network is followed by a rectified linear unit (ReLU) activation function to enhance the model's expressiveness. The output feature map of the second convolution in each stage of the encoder is transmitted to the decoder through a skip connection, and after trimming, it is channel-wise spliced ​​with the output feature map of the upsampling layer in the corresponding stage of the decoder to achieve the fusion of shallow information and deep information, providing a variety of semantic feature information for the decoding process.

[0289] Different types of semantic feature information in the aforementioned multiple semantic feature information may belong to different channels. For example, the multiple semantic feature information contains a total of 72 channels of semantic feature information, of which 24 channels of semantic feature information are plane semantic feature information, another 24 channels of semantic feature information are regional semantic feature information, and another 24 channels of semantic feature information are key point semantic feature information.

[0290] Among them, the initial image processing model in this application is a network related to the multi-task segmentation network.

[0291] The initial image processing model in this application can also be a single-task network, which uses commonly used segmentation networks, key detection networks, and plane detection networks, such as 3DUnet, 3DHourglass, etc.

[0292] S303, processing the multiple semantic feature information by an initial feature processing unit in the initial image processing model to obtain multiple predicted feature information to be processed corresponding to the multiple semantic feature information, the multiple predicted feature information to be processed including: predicted surgical reference surface feature information, predicted tissue region feature information, and predicted key point feature information;

[0293] Optionally, the initial feature processing unit for processing plane semantic feature information processes the plane semantic feature information to obtain corresponding predicted surgical reference surface feature information.

[0294] Optionally, the initial feature processing unit for processing regional semantic feature information processes the regional semantic feature information to obtain corresponding predicted tissue regional feature information.

[0295] Optionally, the initial feature processing unit for processing key point semantic feature information processes the key point semantic feature information to obtain corresponding predicted key point feature information.

[0296] In some optional embodiments of the present application, each of the aforementioned initial feature processing units may include one or more convolution blocks, and each convolution block may include: a convolution unit (Conv3D), a batch normalization unit (BatchNormalization), and a rectified linear unit (RectifiedLinearUnit, ReLU).

[0297] S304, obtaining multiple target feature label information corresponding to the sample image;

[0298] S305, training the initial image processing model according to the multiple predicted feature information to be processed and the multiple target feature label information to obtain a target image processing model, wherein the target image processing model includes a target semantic feature extraction unit and a target feature processing unit, and is configured to obtain, according to the acquired image to be processed, multiple target feature information to be processed corresponding to the image to be processed, wherein the multiple target feature information to be processed includes: target surgical reference surface feature information, target tissue region feature information, and target key point feature information;

[0299] The target image processing model is the target image processing model in the aforementioned embodiment.

[0300] Optionally, the multiple target feature label information includes: target surgical reference surface feature label information, target tissue region feature label information, and target key point feature label information. The initial image processing model is trained based on the multiple predicted feature information to be processed and the multiple target feature label information to obtain a target image processing model, including:

[0301] Determining first loss information based on a first preset loss function, the predicted surgical reference surface feature information, and the target surgical reference surface feature label information;

[0302] Determining second loss information based on a second preset loss function, the predicted tissue region feature information, and the target tissue region feature label information;

[0303] Determining third loss information based on a third preset loss function, the predicted key point feature information, and the target key point feature label information;

[0304] summing the first loss information, the second loss information, and the third loss information to obtain target loss information;

[0305] The initial image processing model is trained according to the target loss information to obtain a target image processing model.

[0306] Specifically, the first preset loss function is a mean square error function, the second preset loss function is an adaptive loss function of DiceLoss, and the third preset loss function is a mean square error function.

[0307] In some optional embodiments of the present application, the first loss information, the second loss information, and the third loss information are summed to obtain target loss information, which can be achieved by the following formula:

[0308]

[0309] Among them, L total is the target loss information, is the first loss information, M is the number of feature information matrices included in the predicted surgical reference surface feature information, y p To predict the surgical reference surface feature information, It is the preset target surgical reference surface feature label information; is the second loss information,

[0310] N is the number of feature information matrices contained in the predicted tissue region feature information, y r To predict tissue regional feature information, Organize regional feature label information for preset targets; is the third loss information, H is the number of feature information matrices contained in the predicted key point feature information, y k To predict key point feature information, It is the target key point feature label information.

[0311] The initial image processing model is trained according to the target loss information to obtain a target image processing model, including:

[0312] When it is determined that the target loss information is not greater than a first preset threshold, the initial image processing model is used as the target image processing model. When it is determined that the target loss information is greater than the first preset threshold, the model parameters of the initial image processing model are updated according to the target loss information, and the execution returns to the initial semantic feature extraction unit in the initial image processing model to perform feature extraction on the sample image until the target loss information is no greater than the first preset threshold, and the most recently updated initial image processing model is used as the target image processing model.

[0313] After the aforementioned target image processing model training is completed, the initial semantic feature extraction unit training becomes Figure 2b The target semantic feature extraction unit in the initial feature processing unit is trained to become Figure 2b The target feature processing unit in .

[0314] The target image processing model trained by this method can quickly determine a variety of target feature information to be processed corresponding to the image to be processed, thereby improving the efficiency of determining the surgical incision position for the target object.

[0315] All of the above technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0316] In order to better implement the various methods of the present application, the present application also provides an image processing device, see Figure 4 , Figure 4 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. The image processing device 40 may include:

[0317] An acquisition unit 41 is configured to acquire an image to be processed of a target object;

[0318] An input unit 42 is configured to input the image to be processed into a target image processing model to obtain a plurality of target feature information to be processed corresponding to the image to be processed, wherein the plurality of target feature information to be processed includes target surgical reference surface feature information, target tissue region feature information, and target key point feature information;

[0319] a determination unit 43, configured to determine surgical incision position information based on the plurality of target feature information to be processed;

[0320] A generating unit 44 is configured to generate corresponding surgical incision position prompt information according to the surgical incision position information, wherein the surgical incision position prompt information is used to prompt the surgical incision position for the target object;

[0321] The acquisition unit 41 is further configured to acquire on-site image information of the target object;

[0322] The display unit 45 is configured to display the surgical incision position prompt information and the on-site image information based on the multiple target feature information to be processed, the on-site image information, and the surgical incision position information.

[0323] Optionally, when the aforementioned device is used to determine the surgical incision location information based on the multiple target feature information to be processed, it is specifically used to:

[0324] Preprocessing the plurality of target feature information to be processed to obtain a plurality of processing results, the plurality of processing results including: surgical reference surface parameter information, tissue region information, and key point information, the tissue region information including: skin surface region information and skin internal region information, the skin internal region information including: blood vessel region information and lesion region information;

[0325] The surgical incision position information is determined according to the multiple processing results.

[0326] Optionally, when the aforementioned device is used to determine the surgical incision location information based on the multiple processing results, it is specifically used to:

[0327] Determining centroid information of the lesion area information according to the lesion area information;

[0328] Determining a plurality of surgical reference plane information according to the surgical reference plane parameter information and the key point information, wherein the plurality of surgical reference plane information includes: coronal plane information, sagittal plane information, and auricular canthal plane information;

[0329] Determine a plurality of first candidate incision key point information based on the centroid information and the plurality of surgical reference surface information;

[0330] The surgical incision position information is determined using the centroid information, the skin internal area information, the skin surface area information, and the plurality of first candidate incision key point information.

[0331] Optionally, the surgical reference plane parameter information includes sagittal plane parameter information, and the aforementioned apparatus, when used to determine a plurality of surgical reference plane information according to the surgical reference plane parameter information and the key point information, is specifically used to:

[0332] determining sagittal plane information according to the sagittal plane parameter information;

[0333] Determining the parameter information and the ear canthal surface information of the ear canthus surface according to the position information of the middle point of the external auditory canal and the position information of the external canthus point in the key point information;

[0334] Determining the parameter information of the coronal plane according to the position information of the middle point of the external auditory canal in the key point information and the parameter information of the canthal plane;

[0335] determining coronal plane information according to the coronal plane parameter information;

[0336] The sagittal plane information, the canthal plane information, and the coronal plane information are used as multiple surgical reference plane information.

[0337] Optionally, the device is further used for:

[0338] using an intersection line between the sagittal plane information and the skin surface area information as sagittal plane intersection line information;

[0339] using the intersection line of the coronal plane information and the skin surface area information as coronal plane intersection line information;

[0340] using the intersection line of the ear canthus surface information and the skin surface area information as the ear canthus surface intersection line information;

[0341] Generating sagittal plane intersection line prompt information, coronal plane intersection line prompt information, and auricular canthal plane intersection line prompt information respectively according to the sagittal plane intersection line information, the coronal plane intersection line information, and the auricular canthal plane intersection line information;

[0342] The sagittal plane intersection line prompt information, the coronal plane intersection line prompt information, and the auricular canthus plane intersection line prompt information are displayed.

[0343] Optionally, when the apparatus is used to determine a plurality of first candidate incision key point information based on the centroid information and the plurality of surgical reference surface information, it is specifically used to:

[0344] translating the sagittal plane information along the sagittal plane normal direction, and when the sagittal plane information is translated to include the centroid information, using the intersection line of the sagittal plane information and the skin surface area information as the sagittal plane intersection parallel line information;

[0345] translating the coronal plane information along the coronal plane normal direction, and when the coronal plane information is translated to include the centroid information, using the intersection line of the coronal plane information and the skin surface area information as the coronal plane intersection parallel line information;

[0346] The ear canthal surface information is translated along the normal direction of the ear canthal surface, and when the ear canthal surface information is translated to include the centroid information, the intersection line of the ear canthal surface information and the skin surface area information is used as the ear canthal surface intersection line parallel line information;

[0347] The multiple first candidate incision key point information is determined based on the sagittal plane intersection parallel line information, the coronal plane intersection parallel line information, and the auricular canthus plane intersection parallel line information.

[0348] Optionally, when the device is used to determine the plurality of first candidate incision key point information based on the sagittal plane intersection parallel line information, the coronal plane intersection parallel line information, and the auricular canthal plane intersection parallel line information, it is specifically used to:

[0349] The intersection of the coronal plane intersection parallel line information and the sagittal plane intersection parallel line information is used as the second candidate incision key point information;

[0350] The intersection of the parallel line information of the sagittal plane intersection line and the parallel line information of the auricular canthus plane intersection line is used as the third candidate incision key point information;

[0351] The intersection of the coronal plane intersection parallel line information and the auricular canthus plane intersection parallel line information is used as the fourth candidate incision key point information;

[0352] The second candidate incision key point information, the third candidate incision key point information, and the fourth candidate incision key point information are used as the plurality of first candidate incision key point information.

[0353] Optionally, when the device is used to determine the surgical incision position information using the centroid information, the skin internal area information, the skin surface area information, and the plurality of first candidate incision key point information, it is specifically used to:

[0354] For each piece of first candidate incision key point information among the plurality of first candidate incision key point information, determining first connection information between the centroid information and the first candidate incision key point information, to obtain a plurality of first connection information corresponding to the plurality of first candidate incision key point information;

[0355] The surgical incision position information is determined based on the plurality of first connection line information, the skin internal area information, and the skin surface area information.

[0356] Optionally, when the aforementioned device is used to determine the surgical incision location information based on the plurality of first connection line information, the skin internal area information, and the skin surface area information, it is specifically used to:

[0357] For each first connection information among the multiple first connection information, when the first connection information includes coordinate position information in the target area information in the skin internal area information, the first candidate incision key point information corresponding to the first connection information is updated; when the first connection information does not include coordinate position information in the target area information, the first candidate incision key point information corresponding to the first connection information is not updated, thereby obtaining multiple fifth candidate incision key point information corresponding to the multiple first connection information; the target area information includes the blood vessel area information;

[0358] The surgical incision position information is determined based on the plurality of fifth candidate incision key point information, the lesion area information, and the skin surface area information.

[0359] Optionally, when the apparatus is used to update the first candidate incision key point information corresponding to the first connection information, it is specifically used to:

[0360] Taking the first candidate incision key point information corresponding to the first connection line information as the center, obtaining first candidate point information belonging to the skin surface area information at a preset distance from the center;

[0361] Determine the second connection line information between the centroid information and the first alternative point information. When the second connection line information does not contain the coordinate position information belonging to the target area information, use the first alternative point information as the first alternative incision key point information corresponding to the updated first connection line information. When the second connection line information contains the coordinate position information belonging to the target area information, update the preset distance and obtain the second alternative point information belonging to the skin surface area information with a distance from the center that is the updated preset distance; update the first alternative incision key point information corresponding to the first connection line information according to the second alternative point information.

[0362] Optionally, when the aforementioned apparatus is used to determine the surgical incision position information based on the plurality of fifth candidate incision key point information, the lesion area information, and the skin surface area information, it is specifically used to:

[0363] Selecting any one of the plurality of fifth candidate incision key point information as reference point information, or obtaining a selection instruction from the object for selecting reference point information from the plurality of fifth candidate incision key point information, and determining the reference point information according to the selection instruction;

[0364] The surgical incision position information is determined based on the reference point information, the lesion area information, and the skin surface area information.

[0365] Optionally, the surgical incision location information includes bone flap incision location information, and the aforementioned device, when used to determine the surgical incision location information based on the reference point information, the lesion area information, and the skin surface area information, is specifically used to:

[0366] Determine projection area information of the lesion area information in the skin surface area information, using the direction from the centroid information of the lesion area information to the reference point information as a projection direction;

[0367] Determine the corresponding bone flap incision edge line information according to the projection area information;

[0368] The bone flap incision edge line information is used as the bone flap incision position information.

[0369] Optionally, the surgical incision location information further includes flap incision location information, and the device is further configured to:

[0370] The skin flap incision position information is determined based on the bone flap incision position information.

[0371] Optionally, when the aforementioned device is used to display the surgical incision position prompt information and the on-site image information based on the multiple target feature information to be processed, the on-site image information, and the surgical incision position information, it is specifically used to:

[0372] Preprocessing the plurality of target feature information to be processed to obtain a plurality of processing results, the plurality of processing results including: surgical reference surface parameter information, tissue region information, and key point information, the tissue region information including: skin surface region information and skin internal region information, the skin internal region information including: blood vessel region information and lesion region information;

[0373] Acquire depth image information corresponding to the scene image information;

[0374] Registering the skin surface area information with the depth image information based on a preset registration algorithm to obtain a coordinate transformation relationship between a first coordinate system corresponding to the skin surface area information and a second coordinate system corresponding to the depth image information;

[0375] Determining a first display position of the surgical incision position prompt information in a display screen corresponding to the on-site image information according to the surgical incision position information and the coordinate conversion relationship;

[0376] displaying the on-site image information;

[0377] The surgical incision position prompt information is displayed in the display screen of the on-site image information based on the first display position.

[0378] Each unit of the image processing device 40 may be implemented in whole or in part through software, hardware, or a combination thereof. Each unit may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each unit.

[0379] The image processing device 40 may be integrated into a terminal or server that has a storage device and a processor and has computing capabilities, or the image processing device 40 may be the terminal or server.

[0380] Optionally, the present application also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0381] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present application, which may be Figure 1The terminal or server shown. Figure 5 As shown, the computer device 500 may include: a communication interface 501, a memory 502, a processor 503, and a communication bus 504. The communication interface 501, the memory 502, and the processor 503 communicate with each other via the communication bus 504. The communication interface 501 is used for data communication between the computer device 500 and external devices. The memory 502 can be used to store software programs and modules. The processor 503 executes the software programs and modules stored in the memory 502, such as the software programs for the corresponding operations in the aforementioned method embodiments.

[0382] Optionally, the processor 503 may call software programs and modules stored in the memory 502 to execute the aforementioned image processing method and / or the aforementioned data processing method.

[0383] This application also provides a computer-readable storage medium for storing a computer program. The computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in each method in the embodiments of this application. For the sake of brevity, it is not further described here.

[0384] This application also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding processes of each method in the embodiments of this application. For the sake of brevity, these processes are not further described here.

[0385] This application also provides a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding process of each method in the embodiments of this application. For the sake of brevity, these instructions are not further described here.

[0386] It should be understood that the processor of the embodiments of the present application may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by hardware integrated logic circuits in the processor or software instructions. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented as a hardware decoding processor, or can be implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0387] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0388] It should be understood that the above-mentioned memories are exemplary but not restrictive. For example, the memories in the embodiments of the present application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). In other words, the memories in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0389] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0390] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0391] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0392] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0393] In addition, each functional unit in the embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0394] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0395] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An image processing method, characterized in that: include: Obtaining an image of the target object to be processed; Inputting the image to be processed into a target image processing model to obtain a plurality of target feature information to be processed corresponding to the image to be processed, wherein the plurality of target feature information to be processed includes: target surgical reference surface feature information, target tissue region feature information, and target key point feature information; Determining surgical incision position information based on the multiple target feature information to be processed; generating corresponding surgical incision position prompt information according to the surgical incision position information, wherein the surgical incision position prompt information is used to prompt the surgical incision position for the target object; Acquiring on-site image information of the target object; Displaying the surgical incision position prompt information and the on-site image information according to the multiple target feature information to be processed, the on-site image information, and the surgical incision position information; Determining surgical incision location information based on the multiple target feature information to be processed includes: Preprocessing the plurality of target feature information to be processed to obtain a plurality of processing results, the plurality of processing results including: surgical reference surface parameter information, tissue region information, and key point information, the tissue region information including: skin surface region information and skin internal region information, the skin internal region information including: blood vessel region information and lesion region information; determining surgical incision position information according to the multiple processing results; Determining surgical incision location information according to the multiple processing results includes: Determining centroid information of the lesion area information according to the lesion area information; Determining a plurality of surgical reference plane information according to the surgical reference plane parameter information and the key point information, wherein the plurality of surgical reference plane information includes: coronal plane information, sagittal plane information, and auricular canthal plane information; Determine a plurality of first candidate incision key point information based on the centroid information and the plurality of surgical reference surface information; The surgical incision position information is determined using the centroid information, the skin internal area information, the skin surface area information, and the plurality of first candidate incision key point information.

2. The method according to claim 1, characterized in that The surgical reference plane parameter information includes sagittal plane parameter information, and a plurality of surgical reference plane information is determined according to the surgical reference plane parameter information and the key point information, including: determining sagittal plane information according to the sagittal plane parameter information; Determining the parameter information and the ear canthal surface information of the ear canthus surface according to the position information of the middle point of the external auditory canal and the position information of the external canthus point in the key point information; Determining the parameter information of the coronal plane according to the position information of the middle point of the external auditory canal in the key point information and the parameter information of the canthal plane; determining coronal plane information according to the coronal plane parameter information; The sagittal plane information, the canthal plane information, and the coronal plane information are used as multiple surgical reference plane information.

3. The method according to claim 2, characterized in that The method further comprises: using an intersection line between the sagittal plane information and the skin surface area information as sagittal plane intersection line information; using the intersection line of the coronal plane information and the skin surface area information as coronal plane intersection line information; using the intersection line of the ear canthus surface information and the skin surface area information as the ear canthus surface intersection line information; Generating sagittal plane intersection line prompt information, coronal plane intersection line prompt information, and canthal plane intersection line prompt information respectively according to the sagittal plane intersection line information, the coronal plane intersection line information, and the canthal plane intersection line information; The sagittal plane intersection line prompt information, the coronal plane intersection line prompt information, and the auricular canthus plane intersection line prompt information are displayed.

4. The method according to claim 2, characterized in that Determining a plurality of first candidate incision key point information based on the centroid information and the plurality of surgical reference surface information includes: translating the sagittal plane information along the sagittal plane normal direction, and when the sagittal plane information is translated to include the centroid information, using the intersection line of the sagittal plane information and the skin surface area information as the sagittal plane intersection parallel line information; translating the coronal plane information along the coronal plane normal direction, and when the coronal plane information is translated to include the centroid information, using the intersection line of the coronal plane information and the skin surface area information as the coronal plane intersection parallel line information; The ear canthal surface information is translated along the normal direction of the ear canthal surface, and when the ear canthal surface information is translated to include the centroid information, the intersection line of the ear canthal surface information and the skin surface area information is used as the ear canthal surface intersection line parallel line information; The multiple first candidate incision key point information is determined based on the sagittal plane intersection parallel line information, the coronal plane intersection parallel line information, and the auricular canthus plane intersection parallel line information.

5. The method according to claim 4, characterized in that Determining the plurality of first candidate incision key point information according to the sagittal plane intersection parallel line information, the coronal plane intersection parallel line information, and the auricular canthus plane intersection parallel line information includes: The intersection of the coronal plane intersection parallel line information and the sagittal plane intersection parallel line information is used as the second candidate incision key point information; The intersection of the parallel line information of the sagittal plane intersection line and the parallel line information of the auricular canthus plane intersection line is used as the third candidate incision key point information; The intersection of the coronal plane intersection parallel line information and the auricular canthus plane intersection parallel line information is used as the fourth candidate incision key point information; The second candidate incision key point information, the third candidate incision key point information, and the fourth candidate incision key point information are used as the plurality of first candidate incision key point information.

6. The method according to claim 1, characterized in that Determining surgical incision position information using the centroid information, the skin internal area information, the skin surface area information, and the plurality of first candidate incision key point information includes: For each piece of first candidate incision key point information among the plurality of first candidate incision key point information, determining first connection information between the centroid information and the first candidate incision key point information, to obtain a plurality of first connection information corresponding to the plurality of first candidate incision key point information; The surgical incision position information is determined based on the plurality of first connection line information, the skin internal area information, and the skin surface area information.

7. The method according to claim 6, characterized in that Determining surgical incision location information according to the plurality of first connection line information, the skin internal area information, and the skin surface area information includes: For each first connection information among the plurality of first connection information, when the first connection information includes coordinate position information belonging to the target area information included in the skin internal area information, the first candidate incision key point information corresponding to the first connection information is updated; when the first connection information does not include coordinate position information belonging to the target area information, the first candidate incision key point information corresponding to the first connection information is not updated, thereby obtaining a plurality of fifth candidate incision key point information corresponding to the plurality of first connection information, wherein the target area information includes the blood vessel area information; The surgical incision position information is determined based on the plurality of fifth candidate incision key point information, the lesion area information, and the skin surface area information.

8. The method according to claim 7, characterized in that Updating the first candidate incision key point information corresponding to the first connection information includes: Taking the first candidate incision key point information corresponding to the first connection line information as the center, obtaining first candidate point information belonging to the skin surface area information at a preset distance from the center; Determine the second connection line information between the centroid information and the first alternative point information. When the second connection line information does not contain the coordinate position information belonging to the target area information, use the first alternative point information as the first alternative incision key point information corresponding to the updated first connection line information. When the second connection line information contains the coordinate position information belonging to the target area information, update the preset distance and obtain the second alternative point information belonging to the skin surface area information with a distance from the center that is the updated preset distance; update the first alternative incision key point information corresponding to the first connection line information according to the second alternative point information.

9. The method according to claim 7, characterized in that Determining surgical incision position information based on the plurality of fifth candidate incision key point information, the lesion area information, and the skin surface area information includes: Selecting any one of the plurality of fifth candidate incision key point information as reference point information, or obtaining a selection instruction of the object for selecting reference point information from the plurality of fifth candidate incision key point information, and determining the reference point information according to the selection instruction; The surgical incision position information is determined based on the reference point information, the lesion area information, and the skin surface area information.

10. The method according to claim 9, characterized in that The surgical incision position information includes bone flap incision position information, and determining the surgical incision position information based on the reference point information, the lesion area information, and the skin surface area information includes: Determine projection area information of the lesion area information in the skin surface area information, using the direction from the centroid information of the lesion area information to the reference point information as a projection direction; Determine the corresponding bone flap incision edge line information according to the projection area information; The bone flap incision edge line information is used as the bone flap incision position information.

11. The method according to claim 10, characterized in that The surgical incision location information also includes flap incision location information, and the method further includes: The skin flap incision position information is determined based on the bone flap incision position information.

12. The method according to claim 1, characterized in that Displaying the surgical incision position prompt information and the on-site image information according to the multiple target feature information to be processed, the on-site image information, and the surgical incision position information, including: Preprocessing the plurality of target feature information to be processed to obtain a plurality of processing results, the plurality of processing results including: surgical reference surface parameter information, tissue region information, and key point information, the tissue region information including: skin surface region information and skin internal region information, the skin internal region information including: blood vessel region information and lesion region information; Acquire depth image information corresponding to the scene image information; Registering the skin surface area information with the depth image information based on a preset registration algorithm to obtain a coordinate transformation relationship between a first coordinate system corresponding to the skin surface area information and a second coordinate system corresponding to the depth image information; Determining a first display position of the surgical incision position prompt information in a display screen corresponding to the on-site image information according to the surgical incision position information and the coordinate conversion relationship; displaying the on-site image information; The surgical incision position prompt information is displayed in the display screen of the on-site image information based on the first display position.

13. A data processing method, characterized in that: include: Get a sample image; Performing feature extraction on the sample image through an initial semantic feature extraction unit in an initial image processing model to obtain a plurality of semantic feature information corresponding to the sample image, wherein the plurality of semantic feature information includes: plane semantic feature information, region semantic feature information, and key point semantic feature information; Processing the multiple semantic feature information by an initial feature processing unit in the initial image processing model to obtain multiple predicted feature information to be processed corresponding to the multiple semantic feature information, the multiple predicted feature information to be processed including: predicted surgical reference surface feature information, predicted tissue region feature information, and predicted key point feature information; Obtaining multiple target feature label information corresponding to the sample image; The initial image processing model is trained according to the multiple predicted feature information to be processed and the multiple target feature label information to obtain a target image processing model, wherein the target image processing model includes a target semantic feature extraction unit and a target feature processing unit, which is used to obtain multiple target feature information to be processed corresponding to the image to be processed based on the acquired image to be processed, wherein the multiple target feature information to be processed includes: target surgical reference surface feature information, target tissue region feature information, and target key point feature information; Wherein, the target image processing model is the target image processing model in claim 1.

14. The method according to claim 13, characterized in that The multiple target feature label information includes: target surgical reference surface feature label information, target tissue region feature label information, and target key point feature label information. The initial image processing model is trained based on the multiple predicted feature information to be processed and the multiple target feature label information to obtain a target image processing model, including: Determining first loss information based on a first preset loss function, the predicted surgical reference surface feature information, and the target surgical reference surface feature label information; Determining second loss information based on a second preset loss function, the predicted tissue region feature information, and the target tissue region feature label information; Determining third loss information based on a third preset loss function, the predicted key point feature information, and the target key point feature label information; summing the first loss information, the second loss information, and the third loss information to obtain target loss information; The initial image processing model is trained according to the target loss information to obtain a target image processing model.

15. The method according to claim 14, characterized in that The initial image processing model is trained according to the target loss information to obtain a target image processing model, including: When it is determined that the target loss information is not greater than a first preset threshold, the initial image processing model is used as the target image processing model. When it is determined that the target loss information is greater than the first preset threshold, the model parameters of the initial image processing model are updated according to the target loss information, and the execution returns to the initial semantic feature extraction unit in the initial image processing model to perform feature extraction on the sample image until the target loss information is no greater than the first preset threshold, and the most recently updated initial image processing model is used as the target image processing model.

16. An image processing device, characterized in that: The device comprises: An acquisition unit, configured to acquire an image to be processed of a target object; An input unit, configured to input the image to be processed into a target image processing model to obtain a plurality of target feature information to be processed corresponding to the image to be processed, wherein the plurality of target feature information to be processed includes: target surgical reference surface feature information, target tissue region feature information, and target key point feature information; a determining unit, configured to determine surgical incision position information based on the plurality of target feature information to be processed; a generating unit, configured to generate corresponding surgical incision position prompt information according to the surgical incision position information, wherein the surgical incision position prompt information is used to prompt the surgical incision position for the target object; The acquisition unit is further configured to acquire on-site image information of the target object; a display unit, configured to display the surgical incision position prompt information and the on-site image information according to the plurality of target feature information to be processed, the on-site image information, and the surgical incision position information; Determining surgical incision location information based on the multiple target feature information to be processed includes: Preprocessing the plurality of target feature information to be processed to obtain a plurality of processing results, the plurality of processing results including: surgical reference surface parameter information, tissue region information, and key point information, the tissue region information including: skin surface region information and skin internal region information, the skin internal region information including: blood vessel region information and lesion region information; determining surgical incision position information according to the multiple processing results; Determining surgical incision location information according to the multiple processing results includes: Determining centroid information of the lesion area information according to the lesion area information; Determining a plurality of surgical reference plane information according to the surgical reference plane parameter information and the key point information, wherein the plurality of surgical reference plane information includes: coronal plane information, sagittal plane information, and auricular canthal plane information; Determine a plurality of first candidate incision key point information based on the centroid information and the plurality of surgical reference surface information; The surgical incision position information is determined using the centroid information, the skin internal area information, the skin surface area information, and the plurality of first candidate incision key point information.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the method according to any one of claims 1 to 15.

18. A computer device, characterized in that: The computer device includes a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 15 by calling the computer program stored in the memory.

Images