Surgical alerting method, apparatus, device, and medium based on ultrasound image visualization

Abstract

Embodiments of the present application relate to a surgical early warning method and device based on ultrasound image visualization, equipment and medium. The method comprises: extracting the corresponding organ surface from the preset voxel model according to the label corresponding to a plurality of preset organs; wherein the organ surface is composed of triangular facets; according to the set frequency, the ultrasound section is refreshed and drawn according to the pose of the ultrasound section; according to the positional relationship between the triangular facets constituting the organ surface and the pose of the ultrasound section, an organ model is drawn; in response to the user's simulated puncture operation on the organ model, the puncture needle is displayed according to the spatial position of the puncture needle. The technical scheme of the embodiments of the present application can realize the effect of improving the safety and accuracy of the operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a surgery early warning method and device based on ultrasonic image visualization, equipment and medium. BACKGROUND

[0002] Ultrasound imaging is a hospital imaging technology for imaging organs and soft tissues in the human body. Ultrasound imaging uses real-time non-invasive high-frequency sound waves to obtain a series of 2D ultrasonic images.

[0003] Ultrasound three-dimensional visualization technology can intuitively display three-dimensional information of organs and mark the relative positions of ultrasonic sections and organs in real time, which can effectively assist doctors in ultrasound diagnosis. When the doctor needs to perform a puncture operation, the doctor can perform preoperative planning through the ultrasonic three-dimensional image, but the planning of the operation often relies on the experience of the doctor, and lacks a basis for judging the risk. Therefore, it is urgent to prewarn the risk of preoperative planning to improve the safety and accuracy of the operation. SUMMARY

[0004] The present application provides a surgery early warning method and device based on ultrasonic image visualization, equipment and medium, which aims to improve the safety and accuracy of the operation.

[0005] In a first aspect, the present application provides a surgery early warning method based on ultrasonic image visualization, comprising:

[0006] extracting corresponding organ surfaces from a preset voxel model according to labels corresponding to a plurality of preset organs; wherein the organ surfaces are composed of triangular facets;

[0007] refreshing and drawing the ultrasonic section according to the pose of the ultrasonic section at a set frequency;

[0008] drawing an organ model according to the positional relationship between the triangular facets constituting the organ surfaces and the pose of the ultrasonic section;

[0009] in response to a simulated puncture operation of a user on the organ model, displaying the situation of a puncture needle passing through the preset organs according to the spatial position of the puncture needle.

[0010] In a second aspect, the present application provides a surgery early warning device based on ultrasonic image visualization, comprising:

[0011] an organ surface extraction module configured to extract corresponding organ surfaces from a preset voxel model according to labels corresponding to a plurality of preset organs; wherein the organ surfaces are composed of triangular facets;

[0012] The ultrasound section rendering module is configured to refresh the ultrasound section according to a position of the ultrasound section at a set frequency.

[0013] The organ model rendering module is configured to render an organ model according to a position relationship between the triangular patches and the ultrasound section.

[0014] The puncture simulation module is configured to simulate a puncture needle passing through the preset organ according to a spatial position of the puncture needle in response to a simulation puncture operation of a user on the organ model.

[0015] In a third aspect, an electronic device is provided, and the electronic device comprises:

[0016] One or more processors;

[0017] A memory configured to store one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the surgical warning method based on ultrasound image visualization as provided in any of the embodiments of the present application.

[0019] In a fourth aspect, a storage medium containing computer executable instructions is provided, and the computer executable instructions are used to execute the surgical warning method based on ultrasound image visualization as provided in any of the embodiments of the present application when executed by a computer processor.

[0020] The surgical warning method, device, equipment and medium based on ultrasound image visualization provided by the embodiments of the present application achieve the rendering of the organ model by combining the voxel model obtained by the three-dimensional reconstruction of the ultrasound and the position of the ultrasound section, determine the risk existing in the surgical planning by the spatial position of the puncture needle in the simulation puncture, solve the problem that the surgical planning lacks the basis for judging the risk and has certain safety hazards, and achieve the effect of improving the safety and accuracy of the surgery. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flowchart of a surgical warning method based on ultrasound image visualization provided by the first embodiment of the present application;

[0022] Figure 2 A structural schematic diagram of a surgical warning device based on ultrasound image visualization provided by the second embodiment of the present application;

[0023] Figure 3 A structural schematic diagram of an electronic device provided by the third embodiment of the present application. DETAILED DESCRIPTION

[0024] The application will be described in further detail below with reference to the drawings and embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not to be used to limit the scope of the application. In addition, it should be noted that, for the sake of brevity, only the portions of the drawings that are necessary for an understanding of the application will be described.

[0025] Embodiment one

[0026] Figure 1 A flowchart of a surgical pre-warning method based on ultrasound image visualization is provided for the first embodiment of the application. The embodiment can be applicable to ultrasound guidance and preoperative planning of a surgical robot. The method can be performed by a surgical pre-warning device based on ultrasound image visualization. The device can be implemented by hardware and / or software and can generally be integrated in an electronic device, such as a computer device. The method specifically includes the following steps:

[0027] Step 110: Extract the surface of the corresponding organ from the preset voxel model according to the labels corresponding to the plurality of preset organs.

[0028] In the preset voxel model, each voxel contains a label. The number of preset organs in the preset voxel model is equal to the number of label categories. The voxels belonging to the same organ have the same label. The extracted organ surface is composed of triangular facets.

[0029] Step 120: Refresh and draw the ultrasound section according to the pose of the ultrasound section at a set frequency.

[0030] In the preset voxel model, each voxel contains a label. The number of preset organs in the preset voxel model is equal to the number of label categories. The voxels belonging to the same organ have the same label. The extracted organ surface is composed of triangular facets.

[0031] Step 130: Draw the organ model according to the positional relationship between the triangular facets constituting the organ surface and the pose of the ultrasound section.

[0032] In the preset voxel model, each voxel contains a label. The number of preset organs in the preset voxel model is equal to the number of label categories. The voxels belonging to the same organ have the same label. The extracted organ surface is composed of triangular facets.

[0033] Step 140: In response to the user's simulated puncture operation on the organ model, display the puncture needle passing through the preset organ according to the spatial position of the puncture needle.

[0034] The user performs a simulation puncture operation, and the puncture needle can be drawn according to the puncture needle direction and the puncture needle tip point. The points on the puncture needle and the surface points of the organ model are judged to determine whether the puncture needle enters the inside of the preset organ. If the puncture needle enters the inside of the important organ, there is a risk of surgery, and the user needs to be warned.

[0035] The technical scheme of the embodiment realizes the rendering of the organ model by combining the voxel model obtained by ultrasonic three-dimensional reconstruction with the pose of the ultrasonic section, determines the risk existing in the surgical planning according to the spatial position of the simulation puncture needle, solves the problem that the surgical planning lacks a basis for judging the risk and has a certain safety hazard, and achieves the effect of improving the safety and accuracy of the surgery.

[0036] Optionally, the organ surface corresponding to each of the plurality of preset organs is extracted from the preset voxel model according to a label corresponding to the preset organ, including:

[0037] The organ surface of each preset organ is extracted in turn according to the label in a set order, and a triangular facet is extracted based on a preset surface rendering algorithm; wherein when the organ surface of the current preset organ is extracted according to the label, the labels of the remaining preset organs are set to a preset value, and after the extraction of the triangular facet of the organ surface of the current preset organ is completed, the labels of the preset organs are restored.

[0038] The same vertices of the extracted triangular facets are merged, and the normal vector of each vertex is calculated.

[0039] Wherein, the number of preset organs is N, and the preset organs are traversed from 1 to N. When the organ surface of the preset organ with label i is extracted, the labels of all voxels with labels other than i are set to 0, and the labels of the other voxels are restored after the extraction of the organ surface of the preset organ is completed. The threshold of the surface is 0.5*i, and the surface of the organ i is extracted by exemplarily using the MarchingCubes algorithm to obtain the triangular facet of the surface, and the same vertices are merged.

[0040] Optionally, the normal vector of each vertex is calculated, including:

[0041] The normal vectors of the triangular facets sharing the same vertex are weighted and averaged to obtain the normal vector of the vertex; wherein the weight is determined according to the following formula:

[0042]

[0043] In the formula, N is the normal vector of the vertex, K is the number of triangular facets sharing the vertex, S k is the area of the triangular facet, and N k is the normal vector of the triangular facet.

[0044] Optionally, the ultrasonic section is refreshed and drawn according to the pose of the ultrasonic section at a set frequency, including:

[0045] When refreshing at a set frequency each time, the joint parameters of the robot arm are acquired from the ultrasonic scanning robot, and the pose of the ultrasonic section is determined according to the calibration parameters of the ultrasonic probe and forward kinematics;

[0046] The model matrix is set according to the pose of the ultrasonic section, the ultrasonic probe model and the scanning fan plane are drawn, and the ultrasonic section is drawn according to a transparent object with a first transparency. Wherein, the ultrasonic section can be drawn according to a transparent object with an opacity of 0.5.

[0047] Optionally, the organ model is drawn according to the positional relationship between the triangular patches constituting the organ surface and the pose of the ultrasonic section, including:

[0048] The positional relationship between the triangular patches and the ultrasonic section at the current pose is determined;

[0049] The triangular patches located in front of the ultrasonic section at the current pose are drawn according to a transparent object with a second transparency;

[0050] The triangular patches located behind the ultrasonic section at the current pose are drawn according to a transparent object with a third transparency.

[0051] Wherein, the organ model refers to the organ surface model rendered, which is divided into two parts, the triangular patches located in front of the ultrasonic section can be drawn according to a transparent object with an opacity of 0.2, and the triangular patches located behind the ultrasonic section can be drawn according to an opaque object.

[0052] Optionally, the positional relationship between the triangular patches and the ultrasonic section is determined, including:

[0053] According to the pose R and t of the ultrasonic section, the plane equation of the ultrasonic section is calculated as follows:

[0054]

[0055] In the formula, r3 is the third column of the matrix R; wherein, R is a pose matrix, and t is a position vector; The transpose matrix of r3 is represented. The pose matrix can be a 3*3 matrix, and the position vector can be a 3*1 vector.

[0056] The relative positional relationship between the vertices of the triangular patches and the ultrasonic section at the current pose is determined, wherein the coordinates of the vertices of the triangular patches are P, and the coordinates of the viewpoint are O, and the positional relationship between the triangular patches and the ultrasonic section is determined according to the following formula

[0057]

[0058] If f>0, the vertex of the triangle patch is in front of the ultrasound section, if f≤0, the vertex of the triangle patch is behind the ultrasound section.

[0059] The method for determining whether the triangle patch is in front of or behind the ultrasound section is that, first, the plane equation of the ultrasound section is calculated according to the pose R and t of the ultrasound section, then the relative position relationship between each vertex of the triangle patch and the ultrasound section is determined, if the number of vertices in front of the ultrasound section is greater than or equal to 2, it is determined that the triangle patch is in front of the ultrasound section, otherwise, it is determined that the triangle patch is behind the ultrasound section.

[0060] Optionally, in response to the user's simulated puncture operation on the organ model, the situation of the puncture needle penetrating the preset organ is displayed according to the spatial position of the puncture needle, including:

[0061] The control instruction of the user's simulated operation of the puncture needle is received through the interactive device, the control instruction includes the angle θ between the axis of the puncture needle and the axis in the ultrasound image, which represents the needle insertion direction, and the coordinates x and y of the needle tip of the puncture needle in the ultrasound image, which represent the needle insertion point;

[0062] According to the needle insertion direction and the needle insertion point, the spatial position of the puncture needle is calculated:

[0063]

[0064]

[0065] Wherein, P is the spatial coordinates of the needle tip of the puncture needle, N is the direction vector of the axis of the puncture needle, is the transformation matrix from the robot end to the base coordinate system, which is obtained by forward kinematics, is the transformation matrix from the ultrasound image coordinate system to the robot end, which is obtained by calibration, s x and s y is the pixel size of the ultrasound image, which is a system parameter;

[0066] On the organ model, the puncture needle is drawn according to the spatial position P and N of the puncture needle.

[0067] The puncture needle is checked whether it penetrates the important organs, if the puncture needle penetrates at least one important organ, the user is shown warning information; wherein, the important organs are the preset organs whose labels are predefined as the important organ label set L, taking P as the starting point, traversing the points on the puncture needle in the direction of-N with a set step size δ, every time a puncture needle point is traversed, it is determined whether the current puncture needle point is located in the important organ, if the puncture needle point is located in the important organ, it means that the puncture needle penetrates the important organ, and the traversal is ended, if the puncture needle point is located outside the important organ, the traversal is continued until the puncture needle point exceeds the region of the voxel model, which means that the puncture needle does not penetrate the important organ; the recursive formula of the coordinates of the puncture needle point is as follows:

[0068] P0=P

[0069] P i+1 =P i -δN

[0070] wherein, P i is a puncture needle point, the voxel index containing the puncture needle point is calculated according to the coordinates of the puncture needle point P i , the label l of the corresponding voxel is inquired through the voxel index, if l∈L, then the puncture needle point P i is located in the important organ, if , then the puncture needle point P i is located outside the important organ, the formula of the index is as follows:

[0071]

[0072]

[0073]

[0074] wherein represents rounding down, o x , o y , o z is the center coordinate of the voxel with the index (0, 0, 0), and w is the size of the voxel, Id x represents the index value in the x direction, Id y represents the index value in the y direction, and Id z represents the index value in the z direction.

[0075] Embodiment two

[0076] Figure 2 A structure schematic diagram of a surgical early warning device based on ultrasonic image visualization provided by the embodiment two of the present application is shown in the figure, the surgical early warning device based on ultrasonic image visualization comprises an organ surface extraction module 210, an ultrasonic section drawing module 220, an organ model drawing module 230 and a puncture simulation module 240, wherein, Figure 2

[0077] The organ surface extraction module 210 is used for extracting the corresponding organ surface from the preset voxel model according to the label corresponding to a plurality of preset organs; wherein the organ surface is composed of triangular facets;

[0078] The ultrasonic section drawing module 220 is used for refreshing drawing the ultrasonic section according to the pose of the ultrasonic section according to the set frequency;

[0079] ​The organ model drawing module 230 is configured to draw an organ model according to a positional relationship of the triangular facets and the pose of the ultrasound section that constitute the organ surface.

[0080] The puncture simulation module 240 is configured to, in response to a user simulating a puncture operation on the organ model, display a situation in which a puncture needle penetrates the preset organ according to a spatial position of the puncture needle.

[0081] The technical scheme of the embodiment combines the voxel model obtained through three-dimensional ultrasound reconstruction with the pose of the ultrasound section to realize rendering of the organ model, determines the risk existing in the surgical planning according to the spatial position of the puncture needle in the simulation puncture, solves the problem that the surgical planning lacks a basis for judging the risk and thus has certain safety hazards, and achieves the effect of improving the safety and accuracy of the surgery.

[0082] Optionally, the organ surface extraction module 210 comprises:

[0083] The triangular facet extraction unit is configured to extract the organ surface of the preset organ according to the labels in a set order and extract triangular facets based on a preset surface drawing algorithm; wherein when the organ surface of the current preset organ is extracted according to the labels, the labels of the remaining preset organs are set to a preset value, and after the extraction of the triangular facets of the organ surface of the current preset organ is completed, the labels of the preset organs are restored.

[0084] The vertex normal vector calculation unit is configured to merge the same vertices of the extracted triangular facets and calculate the normal vector of each vertex.

[0085] Optionally, the vertex normal vector calculation unit is specifically configured to:

[0086] The normal vectors of the triangular facets sharing the same vertex are weighted and averaged to obtain the normal vector of the vertex; wherein the weight is determined according to the following formula:

[0087]

[0088] In the formula, N is the normal vector of the vertex, K is the number of triangular facets sharing the vertex, S k is the area of the triangular facet, and N k is the normal vector of the triangular facet.

[0089] Optionally, the ultrasound section drawing module 220 comprises:

[0090] The pose determination unit of the ultrasound section is configured to, when refreshing at a set frequency each time, acquire joint parameters of a mechanical arm from an ultrasound scanning robot, and determine the pose of the ultrasound section according to calibration parameters of an ultrasound probe and forward kinematics;

[0091] The ultrasound section drawing unit is configured to set a model matrix according to a pose of the ultrasound section, draw an ultrasound probe model and a scanning sector plane, and draw the ultrasound section as a transparent object with a first transparency.

[0092] Optionally, the organ model drawing module 230 comprises:

[0093] The position judging unit is configured to judge a position relationship between the triangular facet and the ultrasound section at the current pose.

[0094] The first drawing unit is configured to draw the triangular facet in front of the ultrasound section at the current pose as a transparent object with a second transparency.

[0095] The second drawing unit is configured to draw the triangular facet behind the ultrasound section at the current pose as a transparent object with a third transparency.

[0096] Optionally, the position judging unit is specifically configured to:

[0097] According to the pose R and t of the ultrasound section, the plane equation of the ultrasound section is calculated as follows:

[0098]

[0099] In the formula, r3 is the third column of the matrix R; wherein R is a pose matrix and t is a position vector.

[0100] The relative position relationship between the vertex of the triangular facet and the ultrasound section at the current pose is judged, wherein the vertex coordinates of the triangular facet are P and the viewpoint coordinates are O, and the position relationship between the triangular facet and the ultrasound section is determined according to the following formula:

[0101]

[0102] If f>0, the vertex of the triangular facet is in front of the ultrasound section, and if f≤0, the vertex of the triangular facet is behind the ultrasound section.

[0103] Optionally, the puncture simulation module 240 is specifically configured to:

[0104] The control instruction of the user simulating the operation of the puncture needle is received through the interactive device, the control instruction comprises an included angle 0 between the puncture needle axis and the axis in the ultrasound image, indicating the needle insertion direction, and coordinates x and y of the puncture needle tip in the ultrasound image, indicating the needle insertion point.

[0105] According to the needle insertion direction and the needle insertion point, the spatial position of the puncture needle is calculated as follows:

[0106]

[0107]

[0108] Where P is the spatial coordinate of the puncture needle tip, and N is the direction vector of the puncture needle axis. It is the transformation matrix from the robot's end effector to the base coordinate system, obtained through forward kinematics. It is the transformation matrix from the ultrasound image coordinate system to the robot end effector, obtained through calibration, s x and s y This refers to the pixel size of the ultrasound image, which is a system parameter.

[0109] On the organ model, draw the puncture needle according to its spatial positions P and N;

[0110] The system checks whether the puncture needle has passed through a vital organ. If the puncture needle has passed through at least one vital organ, a warning message is displayed to the user. Vital organs are predefined organs tagged with a set of vital organ tags L. Starting from point P, the system traverses the points on the puncture needle along the -N direction with a set step size δ. At each puncture needle point, it checks whether the current point is within a vital organ. If it is, the puncture needle has passed through the vital organ, and the traversal ends. If it is outside the vital organ, the traversal continues until the point exceeds the area of ​​the voxel model, indicating that the puncture needle has not passed through a vital organ. The recursive formula for the coordinates of the puncture needle points is as follows:

[0111] P0 = P

[0112] P i+1 =P i -δN

[0113] Among them, P i It is the puncture needle point, based on the puncture needle point P. i The coordinate calculation includes the voxel index of the puncture needle point. The label l of the corresponding voxel is retrieved using the voxel index. If l ∈ L, then the puncture needle point P is determined. i Located within vital organs, if Then the puncture needle point P i Located outside of vital organs, the formula for calculating the index is as follows:

[0114]

[0115]

[0116]

[0117] in Indicates rounding down, o x o y o z Here are the coordinates of the voxel center at index (0, 0, 0), w is the size of the voxel, and Id is the value of the voxel center. xAn index value in the x direction, Id y An index value in the y direction, Id z An index value in the z direction.

[0118] The surgical warning device based on ultrasound image visualization provided by the embodiments of the present application can execute the surgical warning method based on ultrasound image visualization provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0119] Embodiment three

[0120] Figure 3 A structural schematic diagram of an electronic device provided for the third embodiment of the present application is shown in Figure 3 The electronic device includes a processor 310, a memory 320, an input device 330, and an output device 340; the number of processors 310 in the electronic device can be one or more, Figure 3 and an example of one processor 310 is taken in the embodiment; the processor 310, the memory 320, the input device 330, and the output device 340 in the electronic device can be connected through a bus or other means, Figure 3 and an example of connection through a bus is taken in the embodiment.

[0121] The memory 320, as a kind of computer readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the surgical warning method based on ultrasound image visualization in the embodiment of the present application (for example, organ surface extraction module 210, ultrasound section drawing module 220, organ model drawing module 230, and puncture simulation module 240 in the surgical warning device based on ultrasound image visualization). The processor 310 executes various function applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 320, that is, realizes the surgical warning method based on ultrasound image visualization described above.

[0122] The memory 320 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal and the like. In addition, the memory 320 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some examples, the memory 320 can further include a memory remotely arranged with respect to the processor 310, which can be connected to the electronic device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0123] The input device 330 can be used to receive inputted digital or character information, and to generate key signal input related to user settings and function control of the electronic device. The output device 340 can include a display device such as a display screen.

[0124] Embodiment four

[0125] The embodiment four of the present application also provides a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to execute a surgical pre-warning method based on ultrasound image visualization, comprising:

[0126] According to the labels corresponding to a plurality of preset organs, corresponding organ surfaces are extracted from a preset voxel model; wherein the organ surfaces are composed of triangular facets;

[0127] According to a set frequency, the ultrasound section is refreshed and drawn according to the pose of the ultrasound section;

[0128] According to the positional relationship between the triangular facets constituting the organ surfaces and the pose of the ultrasound section, an organ model is drawn;

[0129] In response to a simulated puncture operation of a user on the organ model, a puncture needle is displayed to pass through the preset organ according to the spatial position of the puncture needle.

[0130] Of course, the computer executable instructions of the storage medium provided by the embodiment of the present application are not limited to the method operations as described above, and can also perform related operations in the surgical pre-warning method based on ultrasound image visualization provided by any embodiment of the present application.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general hardware, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH memory, a hard disk or an optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0132] It is worth noting that the above-mentioned embodiments of the surgical warning device based on ultrasonic image visualization include various units and modules only according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy mutual differentiation, and are not used to limit the protection scope of the present application.

[0133] Although the present application has been described in detail with general description, specific embodiments and experiments, some modifications or improvements can be made on the basis of the present application, which is obvious to those skilled in the art. Therefore, these modifications or improvements made on the basis of not deviating from the spirit of the present application, all belong to the scope of protection required by the present application.

Claims

1. A surgical pre-alarm method based on ultrasound image visualization, characterized in that, The method comprises the following steps: extracting organ surfaces corresponding to a plurality of preset organs from a preset voxel model according to labels corresponding to the plurality of preset organs; wherein the organ surfaces are composed of triangular facets; refreshing and drawing the ultrasound section according to the pose of the ultrasound section at a set frequency; drawing an organ model according to the positional relationship between the triangular facets constituting the organ surfaces and the pose of the ultrasound section; in response to a user's simulated puncture operation on the organ model, displaying a situation in which a puncture needle penetrates the preset organs according to the spatial position of the puncture needle; the response to the user's simulated puncture operation on the organ model, the situation in which the puncture needle penetrates the preset organs according to the spatial position of the puncture needle, comprises: receiving a control instruction for the user's simulated operation of the puncture needle through an interactive device, the control instruction including an angle θ between the axis of the puncture needle and the axis in the ultrasound image, representing the direction of needle insertion, the coordinates x and y of the needle tip of the puncture needle in the ultrasound image, representing the needle insertion point; calculating the spatial position of the puncture needle according to the direction of needle insertion and the needle insertion point: where P is the spatial coordinates of the needle tip, N is the direction vector of the needle axis, is the transformation matrix from the robot end to the base coordinate system, which is obtained by forward kinematics, is the transformation matrix from the ultrasound image coordinate system to the robot end, which is obtained by calibration, s x and s y is the ultrasound image pixel size, which is a system parameter; drawing the puncture needle on the organ model according to the spatial position P and N of the puncture needle; checking whether the puncture needle penetrates important organs, and if the puncture needle penetrates at least one of the important organs, displaying warning information to the user; wherein the important organs are the preset organs whose labels are predefined as an important organ label set L, taking P as the starting point, traversing the points on the puncture needle in the direction of -N at a set step size δ, and judging whether the current puncture needle point is located in an important organ every time a puncture needle point is traversed, if the puncture needle point is located in an important organ, it indicates that the puncture needle penetrates the important organ, and the traversal ends, if the puncture needle point is located outside the important organ, the traversal continues until the puncture needle point exceeds the area of the voxel model, indicating that the puncture needle does not penetrate the important organ; the recursive formula of the coordinates of the puncture needle point is as follows: P0=P P i+1 = P i - δN Wherein, P i is the puncture needle point, the voxel index containing the puncture needle point is calculated according to the coordinates of the puncture needle point P i , the label l of the corresponding voxel is inquired through the voxel index, if 1∈L, the puncture needle point P i is located in the important organ, if , the puncture needle point P i is located outside the important organ, and the formula for calculating the index is as follows: wherein denotes rounding down, o x ,o y ,o z is the voxel center coordinate indexed by (0, 0, 0), w is the size of the voxel, Id x denotes the index value in the x direction, Id y denotes the index value in the y direction, Id z denotes the index value in the z direction.

2. The method of claim 1, wherein, the extraction of the organ surfaces corresponding to a plurality of preset organs from a preset voxel model according to labels corresponding to the plurality of preset organs, comprises: extracting the organ surfaces of the preset organs according to the labels in a set order, and extracting the triangular facets based on a preset surface drawing algorithm; wherein when the organ surface of the current preset organ is extracted according to the label, the labels of the remaining preset organs are set to a preset value, and after the extraction of the triangular facets of the organ surface of the current preset organ is completed, the labels of the preset organs are restored; merging the same vertices of the extracted triangular facets, and calculating the normal vector of each vertex.

3. The method of claim 2, wherein, the calculation of the normal vector of each vertex, comprises: weighting and averaging the normal vectors of the triangular facets sharing the same vertex to obtain the normal vector of the vertex; wherein the weight is determined according to the following formula: where N is the normal vector of the vertex, K is the number of the triangular patches sharing the vertex, S k is the area of the triangular patch, N k is the normal vector of the triangular patch.

4. The method of claim 1, wherein, the refreshing and drawing of the ultrasound section according to the pose of the ultrasound section at a set frequency, comprises: when refreshing at the set frequency each time, obtaining the joint parameters of the mechanical arm from the ultrasound scanning robot, and determining the pose of the ultrasound section according to the calibration parameters of the ultrasound probe and forward kinematics; According to the pose of the ultrasound section, a model matrix is set, an ultrasound probe model and a scanning sector plane are drawn, and the ultrasound section is drawn according to a transparent object with a first transparency.

5. The method of claim 4, wherein, The organ model is drawn according to the positional relationship between the triangular patches constituting the organ surface and the pose of the ultrasound section. The positional relationship between the triangular patch and the ultrasound section at the current pose is determined. The triangular patch in front of the ultrasound section at the current pose is drawn according to a transparent object with a second transparency. The triangular patch behind the ultrasound section at the current pose is drawn according to a transparent object with a third transparency.

6. The method of claim 5, wherein, The positional relationship between the triangular patch and the ultrasound section is determined, including: According to the pose R and t of the ultrasound section, the plane equation of the ultrasound section is calculated as follows: Wherein, r3 is the third column of the matrix R; wherein, R is a pose matrix, and t is a position vector; The relative positional relationship between the vertex of the triangular patch and the ultrasound section at the current pose is determined, wherein the vertex coordinates of the triangular patch are P, and the viewpoint coordinates are O, and the positional relationship between the triangular patch and the ultrasound section is determined according to the following formula If f>0, the vertex of the triangular patch is in front of the ultrasound section, and if f≤0, the vertex of the triangular patch is behind the ultrasound section.

7. A surgical alerting device based on ultrasound image visualization, characterized by, It includes: An organ surface extraction module is configured to extract organ surfaces corresponding to a plurality of preset organs from a preset voxel model according to labels corresponding to the organs; wherein the organ surfaces are composed of triangular patches; An ultrasound section drawing module is configured to refresh draw an ultrasound section according to a pose of the ultrasound section at a set frequency; An organ model drawing module is configured to draw an organ model according to a positional relationship between the triangular patches constituting the organ surface and the pose of the ultrasound section; A puncture simulation module is configured to display a puncture needle penetrating the preset organ according to a spatial position of the puncture needle in response to a user simulating a puncture operation on the organ model; The puncture simulation module is specifically configured to: Receive a control instruction of a user simulating a puncture needle through an interactive device, wherein the control instruction includes an angle θ between an axis of the puncture needle and an axis in an ultrasound image, indicating a needle insertion direction, coordinates x and y of a needle tip of the puncture needle in the ultrasound image, indicating a needle insertion point; According to the needle insertion direction and the needle insertion point, the spatial position of the puncture needle is calculated: where P is the spatial coordinates of the needle tip, N is the direction vector of the needle axis, is the transformation matrix from the robot end to the base coordinate system, which is obtained by forward kinematics, is the transformation matrix from the ultrasound image coordinate system to the robot end, which is obtained by calibration, s x and s y is the ultrasound image pixel size, which is a system parameter; Draw the puncture needle on the organ model according to the spatial position P and N of the puncture needle. checking whether the puncture needle passes through an important organ, and if the puncture needle passes through at least one of the important organs, showing a warning message to a user; wherein the important organs are the preset organs predefined as an important organ label set L by the label, and starting from P, a point on the puncture needle is traversed in a -N direction at a set step size δ, and every time a puncture needle point is traversed, it is determined whether the current puncture needle point is located in an important organ, if the puncture needle point is located in an important organ, it indicates that the puncture needle passes through the important organ, and the traversal is ended, if the puncture needle point is located outside the important organ, the traversal is continued until the puncture needle point exceeds the region of the voxel model, indicating that the puncture needle does not pass through the important organ; and a recursive formula of coordinates of the puncture needle point is as follows: P0=P P i+1 = P i - δN Wherein, P i is the puncture needle point, the voxel index containing the puncture needle point is calculated according to the coordinates of the puncture needle point P i , the label l of the corresponding voxel is inquired through the voxel index, if l∈L, then the puncture needle point P i is located in the important organ, if , then the puncture needle point P i is located outside the important organ, and the formula for calculating the index is as follows: wherein denotes rounding down, o x ,o y ,o z is the voxel center coordinate indexed by (0, 0, 0), w is the size of the voxel, Id x denotes the index value in the x direction, Id y denotes the index value in the y direction, Id z denotes the index value in the z direction.

8. An electronic device, comprising: comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the surgical warning method based on ultrasound image visualization as claimed in any one of claims 1-6.

9. A storage medium containing computer-executable instructions, wherein: the computer executable instructions when executed by a computer processor are used to perform the surgical warning method based on ultrasound image visualization as claimed in any one of claims 1-6.

Images