Dqn dynamic surgical path selection system, method, terminal and medium

Abstract

The application provides a DQN dynamic operation path selection system, method, terminal and medium, comprising: a data set acquisition module, which is used for acquiring a data set required for model training; a model training module, which is used for inputting the data set acquired by the data set acquisition module into a DQN model for training to obtain a dynamic operation path prediction model; and a model deployment module, which is used for deploying the dynamic operation path prediction model to an external terminal device, so as to acquire current input patient head image information, take current operation knife step position data, current operation key area coordinate data and operation target terminal point position data as input data, and regenerate a next best operation path at a preset interval time. The application can dynamically calculate the best operation path in each period in the transmaxillary transpterygoid implantation operation process, can assist a doctor to safely and efficiently implement the transmaxillary implantation operation technical scheme, can assist the doctor to safely and efficiently implement the transmaxillary implantation operation, and fills the gap in the current market.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optimal surgical path selection, and particularly to a DQN dynamic surgical path selection system, method, terminal and medium. BACKGROUND

[0002] Traditional transzygomatic transpterygoid implant surgery generally uses a static fixed angle method to design a surgical path, which cannot overcome the complex soft and hard tissue environment and motion environment of the oral cavity.

[0003] Due to the long duration of clinical surgery, the oral tissue of the patient is pulled, moved or the posture of the patient changes during the surgery, which causes the relative displacement of important clinical anatomy. The fixed surgical path planned in advance is likely to pass through important clinical anatomy, including the orbital region, the pterygoid venous plexus, the skull base and the lateral facial muscles and skin of the malar bone, in the displaced oral environment.

[0004] Therefore, how to dynamically calculate the optimal surgical path in each period of the transzygomatic transpterygoid implant surgery is a technical problem that needs to be solved in the field. SUMMARY

[0005] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a DQN dynamic surgical path selection system, method, terminal and medium, which solves the technical problem of how to dynamically calculate the optimal surgical path in each period of the transzygomatic transpterygoid implant surgery.

[0006] To achieve the above-mentioned purposes and other related purposes, the first aspect of the present application provides a DQN dynamic surgical path selection system, comprising: a data set acquisition module for acquiring a data set required for model training; a model training module for inputting the data set acquired by the data set acquisition module into a DQN model for training to obtain a dynamic surgical path prediction model; a model deployment module for deploying the dynamic surgical path prediction model obtained by training to an external terminal device, so as to acquire current input patient head image information, take current surgical knife step position data, current surgical key area coordinate data and surgical target endpoint position data as input data, and regenerate the next optimal surgical path at a preset interval time.

[0007] In some embodiments of the first aspect of the present application, the process of acquiring the data set by the data set acquisition module includes: first, acquiring the whole head image information of the patient before the operation; the whole head image information includes hard tissue position information and soft tissue information containing facial nerves and blood vessels; second, training the model with the whole head image information and a plurality of offset amount information of the enumerated soft tissue as input data.

[0008] In some embodiments of the first aspect of the present application, the state vector of the DQN model comprises a step position of a scalpel in a surgical procedure, a current state, and reward information that has been obtained; and the action selection output by the DQN model comprises a moving direction, a moving distance, and a dwell time.

[0009] In some embodiments of the first aspect of the present application, the model training module further performs the following during the process of training the DQN model: based on an experience replay algorithm, storing several pieces of experience data obtained by one or more greedy strategies into a replay buffer for subsequent random selection of experience data from the replay buffer during the training process.

[0010] In some embodiments of the first aspect of the present application, the model training module applies the DQN model to a continuous action space by performing any one of the following during the process of training the DQN model: based on a normalized advantage algorithm, selecting a quadratic function to fit the Q function so as to apply the DQN model to the continuous action space; or discretizing the position coordinates of important clinical anatomical points and solving the maximum Q value of the several points obtained by discretization.

[0011] In some embodiments of the first aspect of the present application, the model training module sets a larger reward for a surgical focus area with less risk and a smaller reward for a surgical focus area with more risk during the model training process; and sets a larger reward for a surgical path farther away from a risk area and a smaller reward for a surgical path closer to the risk area.

[0012] In some embodiments of the first aspect of the present application, the model training module sets a higher reward for an action that reaches a target area, and a higher reward for a surgical path angle that is the same as a current scalpel angle to reduce the number of angles in the surgical procedure.

[0013] In some embodiments of the first aspect of the present application, the model deployment module renders and displays the best surgical path and the recommended surgical path range using different colors for annotation.

[0014] To achieve the above object and other related objects, the second aspect of the present application provides a DQN dynamic surgical path selection method, comprising: obtaining a data set required for model training; inputting the data set obtained by the data set obtaining module to a DQN model for training to obtain a dynamic surgical path prediction model; deploying the dynamic surgical path prediction model obtained by training to an external terminal device to obtain current input patient head image information, using current scalpel step position data, current surgical focus area coordinate data, and surgical target endpoint position data as input data, and regenerating a next best surgical path at a preset interval.

[0015] To achieve the above object and other related objects, the third aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the DQN dynamic surgical path selection method.

[0016] To achieve the above object and other related objects, the fourth aspect of the present application provides an electronic terminal, which comprises a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to make the terminal execute the DQN dynamic surgical path selection method.

[0017] As described above, the DQN dynamic surgical path selection system, method, terminal and medium of the present application have the following beneficial effects: the present application can dynamically calculate the optimal surgical path in each period of the transzygomatic transpterygoid implantation surgery process, and provides a technical solution which is simple to operate and can assist doctors to safely and efficiently implement the transzygomatic implantation surgery, and is a navigation device for assisting doctors to safely and efficiently implement the transzygomatic implantation surgery, which fills the gap in the current market. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A structure schematic diagram of a DQN dynamic surgical path selection system in an embodiment of the present application is shown.

[0019] Figure 2 A training process schematic diagram of a DQN model in an embodiment of the present application is shown.

[0020] Figure 3 A process schematic diagram of a DQN model in an embodiment of the present application is shown.

[0021] Figure 4 A schematic diagram of several experiences stored in a replay buffer in an embodiment of the present application is shown.

[0022] Figure 5 A flowchart of a DQN dynamic surgical path selection method in an embodiment of the present application is shown.

[0023] Figure 6 A structure schematic diagram of an electronic terminal in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0025] It should be noted that in the following description, reference is made to the accompanying drawings, which illustrate several embodiments of this application. It should be understood that other embodiments may also be used, and changes in mechanical composition, structure, electrical system, and operation may be made without departing from the spirit and scope of this application. The following detailed description should not be considered limiting, and the scope of the embodiments of this application is defined only by the claims of the published patent. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. Spatially related terms, such as “upper,” “lower,” “left,” “right,” “below,” “below,” “lower part,” “above,” “upper part,” etc., may be used herein to illustrate the relationship between one element or feature shown in the figures and another element or feature.

[0026] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," "fixing," and "holding" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0027] Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, operation, element, component, item, kind, and / or group, but do not preclude the presence, occurrence, or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms “or” and “and / or” as used herein are interpreted as inclusive, or mean any one or any combination thereof. Thus, “A, B, or C” or “A, B, and / or C” means “any one of: A; B; C; A and B; A and C; B and C; A, B, and C.” Exceptions to this definition arise only when combinations of elements, functions, or operations are inherently mutually exclusive in some manner.

[0028] To solve the problems in the background art, the present application provides a DQN dynamic surgical path selection method, device, terminal and medium, which provides a deep learning algorithm for dynamically selecting the best surgical path according to the important clinical anatomical points that have been acquired in real time. In the training of reinforcement learning, the current surgical step position is taken as the starting point; the important clinical anatomical regions are labeled, and different penalties are set for different clinical anatomies; the NAF algorithm is used to make the control output continuous; the position coordinates of the important clinical anatomical points are discretized to reduce the amount of data required for training; rewards are set for reaching the target region, and the like.

[0029] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the embodiments of the present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the application.

[0030] Before the present application is further described, the terms and phrases involved in the embodiments of the present application are explained, which are applicable to the following explanations:

[0031] <1> Transciliary transorbital implant surgery: a high-difficult complex implant technology that uses the zygoma and orbital plate to fix the implant, fixes the implant in the zygoma area by fixing the implant in the zygoma area, and then fixes the orbital plate in the area adjacent to the maxilla.

[0032] <2> DQN (Deep Q-network): Deep Q-network, which refers to a Q-Learning algorithm based on deep learning.

[0033] <3> Greedy algorithm: refers to making the best choice at the moment when solving a problem, i.e. not considering the overall optimization, and the algorithm gets a local optimal solution in a certain sense.

[0034] The embodiments of the present application provide a DQN dynamic surgical path selection method, a system for DQN dynamic surgical path selection method, and a storage medium for storing an executable program for implementing the DQN dynamic surgical path selection method. As for the implementation of the DQN dynamic surgical path selection method, the exemplary implementation scenarios of the DQN dynamic surgical path selection will be described.

[0035] As shown in Figure 1 , a structure diagram of a DQN dynamic surgical path selection system in the embodiments of the present application is shown. The structure of the DQN dynamic surgical path selection system in the present embodiment mainly includes the following: a data set acquisition module 101, a model training module 102 and a model deployment module 103.

[0036] The data set acquisition module 101 is configured to acquire a data set required for model training.

[0037] In the embodiment of the present application, the process of acquiring the data set by the data set acquisition module 101 includes: first, acquiring preoperative static whole head image information of a patient; the whole head image information includes hard tissue position information and soft tissue information containing facial nerves and blood vessels; second, using the whole head image information and enumerated soft tissue offset information as input data to train the model.

[0038] The model training module 102 is configured to input the data set acquired by the data set acquisition module 101 into a DQN model for training to obtain a dynamic surgical path prediction model.

[0039] DQN (Deep Q-Network) refers to a reinforcement learning algorithm based on deep learning, which can directly output the action required for a specific task from the input image without manual feature extraction, thereby realizing autonomous learning in a complex environment. The goal is to learn how to make the best decision for a specific environment through a series of interactions. In reinforcement learning, the agent constantly observes the environment, makes decisions, and finds the best one in the behavior sequence through continuous attempts. Unlike supervised learning, there is no label or pre-determined correct answer in reinforcement learning, so the best decision needs to be learned through reward signals and interaction with the environment.

[0040] Specifically, a deep neural network is used in the training of the DQN model, which is a Q network (Q-Network). The input of the Q network is a state vector, and the output is the expected reward of obtaining each action in the state. In the process of model training, the DQN algorithm continuously adjusts the Q network parameters so that it can approximate the true action-value function. In the output of the network, if the action with the maximum expected reward is consistent with the true reward, it means that the output of the Q network is good enough and the training can be stopped.

[0041] The training process of the DQN model is as follows: Figure 2As shown, the input parameter of the DQN model is a state vector (state S), and the output is the expected reward of each action under the state vector. It should be understood that the Q-Learning algorithm maintains a Q-table, which stores the reward obtained by taking action a under each state s, i.e., the state-value function Q(s, a), and this algorithm has great limitations. In reality, in many cases, the state space faced by the reinforcement learning task is continuous, and there are infinitely many states, and in this case, the table cannot be used to store the value function. Therefore, a function Q(s, a, w) is used to approximate the state-value function Q(s, a), which is called a Q-network, and w is the parameter of the neural network training.

[0042] For example, the state s corresponds to three actions, and the corresponding approximate state-value function is constructed as follows: Q(s, "left"; w), Q(s, "right"; w), and Q(s, "up"; w), which respectively represent the training parameters of the left action under the state s, the right action under the state s, and the up action under the state s.

[0043] In the embodiments of the present application, the state vector of the DQN model includes the step position of the surgical knife in the surgical process, the current state, and the reward information obtained; and the action selection output by the DQN model includes the moving direction, the moving distance, the staying time, etc.

[0044] In the embodiments of the present application, a corresponding reward value is set for each action in the training process of the DQN model. In the DQN algorithm, the reward value is usually a real number, which will affect the final surgical path selected by the algorithm. If an action improves its reward value, the algorithm is more likely to select the action; and if an action cannot improve its reward value, the algorithm is more likely to reduce the frequency of selecting the action.

[0045] In the embodiments of the present application, the DQN model is combined with the Figure 3 The process of training the DQN model is further explained and described, and the steps include the following:

[0046] Step S3A: initializing the Q-network, inputting the state S t , and outputting the reward value Q of all actions under S t .

[0047] Step S3B: using the greedy strategy to select an action a t , inputting the action a t into the environment to obtain a new state S t+1 and r.

[0048] Step S3C: calculating the target value: y t = rt + γ · max a Q(S t+1 , a; w); where r t represents the value obtained from actual observation; max a Q(S t+1 , a; w) represents the estimated value made by the Q network at state S t+1 ; and γ represents a constant.

[0049] Step S3D: Calculate the loss function as follows: L = 1 / 2 [y t - Q(s, a; w)] 2 .

[0050] Step S3E: Update the Q parameters to make Q(S t , a t ) as close as possible to y t .

[0051] Step S3F: From the above steps, a four-tuple experience transition: (s t , a t , r t , s t+1 ) can be obtained, which is discarded after use.

[0052] Step S3G: Input the new state, and repeat the above steps to update the model.

[0053] It should be noted that the greedy strategy means that when solving a problem, always make the best choice at the moment, that is, do not consider the overall optimization, and the algorithm gets a local optimal solution in a certain sense. The execution process of the greedy algorithm is as follows: a mathematical model is established to describe the problem; the problem to be solved is divided into several sub-problems; each sub-problem is solved to obtain a local optimal solution of the sub-problem; and the local optimal solution of the sub-problem is combined to obtain a solution of the original problem. The characteristic of the greedy algorithm is to proceed step by step, and the optimal choice is usually based on the current situation according to a certain optimization measure, without considering the overall situation, which saves a lot of time that must be spent to exhaust all possibilities to find the optimal solution. The greedy algorithm uses a top-down approach to make successive greedy choices in an iterative manner. Each time a greedy choice is made, the problem to be solved is simplified into a smaller sub-problem. Through each step of greedy selection, an optimal solution to the problem can be obtained. Although a local optimal solution is obtained at each step, the global solution obtained is not necessarily optimal, so the greedy algorithm does not backtrack.

[0054] Preferably, in the process of training the DQN model, each piece of experience data transition is discarded after being used, which not only wastes computing resources but also slows down the model training. Therefore, in the embodiment of the present application, the experience data obtained by one or more greedy strategies is stored in a replay buffer based on an experience replay algorithm, and the experience data is randomly selected from the replay buffer in the subsequent training process. For example, the experience data stored in the replay buffer includes experience data 1 (s Figure 4 t , a t , r t , s t+1 ); experience data 2 (s t+1 , a t+1 , r t+1 , s t+2 ); experience data 3 (s t+2 , a t+2 , r t+2 , s t+3 ), and so on.

[0055] In some examples, the experience replay process of the experience replay algorithm includes the following: storing a preset number of experience data obtained in the model training process in a replay buffer; storing the current newly observed experience data in the replay buffer and deleting the earliest stored experience data in the replay buffer, so that the number of stored experience data in the replay buffer does not exceed the preset number; when the DQN model needs to be updated, a small batch of experience data is randomly sampled from the replay buffer; and the sampled experience data is used for training to update the policy of the DQN model.

[0056] In the embodiment of the present application, although the DQN model can select actions by calculating the maximum value of the Q value, it is not efficient and applicable in dealing with continuous action problems, especially for surgical procedures that require continuous and smooth actions. This is because, if there are only a few discrete actions, these actions can be substituted into the DQN model to calculate the Q value and select the corresponding action. However, if the action is continuous, the action is a vector, and there are corresponding values in the vector, so it is impossible to enumerate all actions to calculate the corresponding Q value. Therefore, the embodiment of the present application provides the following two ways to apply deep neural network Q-Learning to continuous action space.

[0057] Method one) applying deep neural network Q-Learning to continuous action space based on NAF (Normalized Advantage Functions) normalized advantage algorithm. Specifically, a function cluster that is easy to optimize is selected to fit the Q function, and the easy-to-optimize function includes a quadratic function. Taking the quadratic function as an example: ​

[0058]

[0059] Thus a neural network or other structure is trained, with input state s and output (μ, P, V), where μ and V are vectors and P is a matrix. This way does not need to make changes in the algorithm compared with the original non-continuous QDN algorithm, is very easy, and is also efficient.

[0060] Method two) discretize the position coordinates of important clinical anatomical points, and solve the maximum Q value of the discretized points. Taking N points as an example:

[0061] More preferably, the model training module 102 sets a larger reward for a surgical key area with smaller risk and a smaller reward for a surgical key area with larger risk during the model training process; the farther away from the risk area, the larger the reward of the surgical path, and the closer to the risk area, the smaller the reward of the surgical path.

[0062] More preferably, the model training module 102 sets a higher reward for the action of reaching the target area, and sets a higher reward for the same angle of the surgical path and the current surgical knife to reduce the corner angle in the surgical process.

[0063] The model deployment module 103 is configured to deploy the trained dynamic surgical path prediction model to an external terminal device, to obtain the current input patient head image information, to take the current surgical knife step position data, the current surgical key area coordinate data, and the surgical target terminal point position data as input data, and to regenerate the next best surgical path at a preset interval time.

[0064] More preferably, the model deployment module 103 renders and displays the best surgical path and the recommended surgical path range with different colors after generating the next best surgical path, so as to clearly show and better assist doctors in decision-making. For example, the recommended surgical path range can be rendered in blue, and the best surgical path can be set in green, etc. The present example is only for reference and is not used for limitation.

[0065] More preferably, the current surgical knife step position data includes orientation data and coordinate data of the surgical knife.

[0066] More preferably, the regeneration of the next best surgical path at a preset interval time includes generating the angle range, the recommended local best angle, and the distance of the surgical path.

[0067] It should be noted that the DQN dynamic surgical path selection system provided in the above embodiment is only used for example to illustrate the division of the above program modules during the DQN dynamic surgical path selection, and in actual application, the above processing can be completed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above processing.

[0068] In the above, the structure and implementation process of the DQN dynamic surgical path selection system in the embodiment of the application are explained in detail. In the following, another DQN dynamic surgical path selection method, terminal and medium in the embodiment of the application will be further explained in combination with the embodiment.

[0069] As shown in the flowchart of the DQN dynamic surgical path selection method in the embodiment of the application. Figure 5 The flow of the DQN dynamic surgical path selection method in the embodiment of the application includes the following steps:

[0070] Step S51: Obtain a data set required for model training.

[0071] In some examples, the process of obtaining the data set includes: first, obtaining preoperative static overall head image information of a patient; the overall head image information includes hard tissue position information and soft tissue information containing facial nerves and blood vessels; second, training the model by taking the overall head image information and enumerated soft tissue offset information as input data.

[0072] In some examples, the state vector of the DQN model includes the step position of the surgical knife in the surgical process, the current state and the reward information obtained; the action selection output by the DQN model includes the moving direction, the moving distance and the staying time.

[0073] Step S52: input the data set obtained by the data set obtaining module into the DQN model for training to obtain a dynamic surgical path prediction model.

[0074] In some examples, the model training module further performs the following during the DQN model training process: based on the experience replay algorithm, store several pieces of experience data obtained by one or more greedy strategies in a replay buffer for subsequent training process to randomly select experience data from the replay buffer.

[0075] In some examples, the model training module applies the DQN model to the continuous action space by performing any one of the following during the process of DQN model training: based on a normalized advantage algorithm, selecting a quadratic function to fit the Q function so as to apply the DQN model to the continuous action space; or discretizing the position coordinates of the important clinical anatomical points and solving the maximum Q value of the discretized points.

[0076] In some examples, the model training module sets a larger reward for a surgical focus area with a smaller risk and a smaller reward for a surgical focus area with a larger risk during the model training process; and sets a larger reward for a surgical path farther away from the risk area and a smaller reward for a surgical path closer to the risk area.

[0077] In some examples, the model training module sets a higher reward for an action reaching the target area and a higher reward for a surgical path angle being the same as a current surgical knife angle to reduce the corner angle in the surgical process.

[0078] Step S53: deploying the trained dynamic surgical path prediction model to an external terminal device to obtain current input patient head image information, taking the current surgical knife step position data, the current surgical focus area coordinate data and the surgical target end position data as input data, and regenerating the next optimal surgical path at a preset interval time.

[0079] In some examples, the optimal surgical path and the recommended surgical path range are rendered and displayed using different colors after the model is deployed.

[0080] It should be noted that the DQN dynamic surgical path selection method provided by the above embodiments and the DQN dynamic surgical path selection system embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0081] The DQN dynamic surgical path selection method provided by the embodiments of the present application can be implemented on the terminal side or the server side. As for the hardware structure of the DQN dynamic surgical path selection terminal, please refer to Figure 6An optional hardware structure schematic diagram of the DQN dynamic operation path selection terminal 600 provided by the embodiment of the present application is shown in FIG. 6. The terminal 600 can be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, etc. The DQN dynamic operation path selection terminal 600 includes at least one processor 601, a memory 602, at least one network interface 604, and a user interface 606. The various components in the apparatus are coupled together by a bus system 605. It can be understood that the bus system 605 is used to realize the connection communication between the components. The bus system 605 includes a data bus, a power bus, a control bus, and a status signal bus, in addition to the data bus. However, for the purpose of clear illustration, all the buses are marked as the bus system in FIG. 6. Figure 6

[0082] The user interface 606 can include a display, a keyboard, a mouse, a trackball, a click gun, a key, a button, a touchpad, or a touch screen, etc.

[0083] It can be understood that the memory 602 can be a volatile memory or a non-volatile memory, or both. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiment of the present application is intended to include but not limited to these and any other suitable categories of memory.

[0084] The memory 602 in the embodiment of the present application is used to store various categories of data to support the operation of the DQN dynamic operation path selection terminal 600. Examples of these data include any executable programs for operating on the DQN dynamic operation path selection terminal 600, such as an operating system 6021 and an application program 6022. The operating system 6021 contains various system programs, such as a framework layer, a core library layer, a driver layer, etc., for realizing various basic services and processing hardware-based tasks. The application program 6022 can contain various application programs, such as a media player (MediaPlayer), a browser (Browser), etc., for realizing various application services. The DQN dynamic operation path selection method provided by the embodiment of the present application can be contained in the application program 6022.

[0085] ​The method disclosed by the embodiments of the present application can be applied to the processor 601 or implemented by the processor 601. The processor 601 can be an integrated circuit chip having a signal processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit or an instruction in the form of software in the processor 601. The processor 601 can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The processor 601 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. The general-purpose processor 601 can be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiments of the present application, the hardware decoding processor can be directly implemented or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the foregoing method.

[0086] In the exemplary embodiments, the DQN dynamic surgical path selection terminal 600 can be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), etc. for executing the foregoing method.

[0087] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by computer program-related hardware. The foregoing computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the foregoing storage medium includes ROM, RAM, magnetic disc or optical disc, and various media that can store program codes.

[0088] In the embodiments provided in the present application, the computer readable and writable storage medium can include a read-only memory, a random access memory, an EEPROM, a CD-ROM or other optical disk storage device, a magnetic disk storage device or other magnetic storage device, a flash memory, a U disk, a mobile hard disk, or any other medium capable of storing desired program code in the form of instructions or data structures and capable of being accessed by a computer. In addition, any connection can be appropriately referred to as a computer readable medium. For example, if instructions are sent from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave is included in the definition of the medium. However, it should be understood that the computer readable and writable storage medium and the data storage medium do not include connections, carriers, signals or other transitory media, but are intended for non-transitory, tangible storage media. As used in the application, magnetic disks and optical disks include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks and Blu-ray disks, in which magnetic disks typically magnetically copy data, and optical disks optically copy data with a laser.

[0089] In summary, the present application provides a DQN dynamic surgical path selection system, method, terminal and medium, which can dynamically calculate the optimal surgical path in each period of the transzygomatic transpterygoid implantation surgery process, and provides a technical solution that is simple to operate and can assist doctors to safely and efficiently perform transzygomatic implantation surgery. It is a navigation device that assists doctors to safely and efficiently perform transzygomatic implantation surgery, filling the gap in the current market. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has high industrial utilization value.

[0090] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not intended to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical idea disclosed in the present application should be covered by the claims of the present application.

Claims

1. A DQN dynamic surgical path selection system, characterized in that, include: The dataset acquisition module is used to acquire the dataset required for model training. The process of acquiring the dataset by the dataset acquisition module includes: first, acquiring the patient's preoperative static overall head image information; the overall head image information includes hard tissue location information and soft tissue information containing facial nerves and blood vessels; second, using the overall head image information and several possible offset information of the enumerated soft tissue as input data to train the model. The model training module is used to input the dataset acquired by the dataset acquisition module into the DQN model for training to obtain a dynamic surgical path prediction model. The state vector of the DQN model includes the stepping position of the scalpel during surgery, the current state, and the obtained reward information. The action selection output by the DQN model includes the movement direction, movement distance, and dwell time. During model training, the model training module assigns larger rewards to low-risk surgical key areas and smaller rewards to high-risk surgical key areas; it also assigns larger rewards to surgical paths further away from the risk area and smaller rewards to surgical paths closer to the risk area; the model training module assigns higher rewards to actions that reach the target area and assigns higher rewards to actions where the surgical path angle is the same as the current scalpel angle to reduce angles during surgery; a corresponding reward value is assigned to each action during the DQN model training process; the reward value affects the final selected surgical path. The model deployment module is used to deploy the trained dynamic surgical path prediction model to an external terminal device to obtain the current input patient head image information, using the current scalpel stepping position data, the current surgical focus area coordinate data, and the surgical target endpoint position data as input data, and regenerate the next optimal surgical path at preset intervals.

2. The DQN dynamic surgical path selection system according to claim 1, characterized in that, The model training module also performs the following during the DQN model training process: based on the experience replay algorithm, several pieces of experience data obtained by one or more greedy strategies are stored in the replay buffer, so that experience data can be randomly selected from the replay buffer during subsequent training.

3. The DQN dynamic surgical path selection system according to claim 1, characterized in that, During the training of the DQN model, the model training module applies the DQN model to the continuous action space by performing one of the following methods: selecting a quadratic function to fit the Q function based on the normalization dominance algorithm so that the DQN model can be applied to the continuous action space; or, discretizing the position coordinates of important clinical anatomical points and solving for the maximum Q value of several points obtained by discretization.

4. The DQN dynamic surgical path selection system according to claim 1, characterized in that, The model deployment module uses different colors to render and display the optimal surgical path and the recommended range of surgical paths.

5. A DQN dynamic surgical path selection method, characterized in that, include: Obtain the dataset required for model training; the process of obtaining the dataset includes: first, obtaining the patient's preoperative static overall head image information; the overall head image information includes hard tissue location information and soft tissue information containing facial nerves and blood vessels; second, using the overall head image information and several possible offset information of the enumerated soft tissue as input data to train the model. The dataset acquired by the dataset acquisition module is input into the DQN model for training to obtain a dynamic surgical path prediction model. The state vector of the DQN model includes the scalpel's stepping position during surgery, the current state, and the obtained reward information. The action selection output by the DQN model includes the movement direction, movement distance, and dwell time. During model training, higher rewards are assigned to low-risk surgical key areas, and lower rewards are assigned to high-risk surgical key areas. Higher rewards are assigned to surgical paths further away from the risk area, and lower rewards are assigned to surgical paths closer to the risk area. Actions reaching the target area are assigned higher rewards, and actions with the same surgical path angle as the current scalpel angle are assigned higher rewards to reduce angles during surgery. A corresponding reward value is assigned to each action during the DQN model training process; the reward value affects the final selected surgical path. The trained dynamic surgical path prediction model is deployed to an external terminal device to obtain the current input patient head image information. The current scalpel stepping position data, the current surgical focus area coordinate data, and the surgical target endpoint position data are used as input data, and the next optimal surgical path is regenerated at a preset interval.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the DQN dynamic surgical path selection method of claim 5.

7. An electronic terminal, characterized in that, include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory to cause the terminal to perform the DQN dynamic surgical path selection method as described in claim 5.

Images