DQN dynamic operation path selection system and method, terminal and medium

Abstract

The invention provides a DQN dynamic operation path selection system and method, a terminal and a medium, and the system comprises a data set obtaining module which is used for obtaining a data set required by model training; the model training module is used for inputting the data set obtained by the data set obtaining module into a DQN model for training so as to obtain a dynamic operation path prediction model; the model deployment module is used for deploying the dynamic operation path prediction model to external terminal equipment so as to obtain currently input patient head image information, and current scalpel stepping position data, current operation key area coordinate data and operation target end point position data are used as input data; and regenerating the next section of optimal operation path at the preset interval time. According to the technical scheme, the optimal operation path in each time period in the zygomatic and wing penetrating implanting operation process can be dynamically calculated, a doctor can be assisted in safely and efficiently implementing the zygomatic bone penetrating implanting operation, the doctor can be assisted in safely and efficiently implementing the zygomatic bone penetrating implanting operation, and the blank of the current market is filled.

Description

Technical Field

[0001] The present application relates to the technical field of optimal surgical path selection, and in particular to a DQN dynamic surgical path selection system, method, terminal and medium. Background Art

[0002] Traditional transzygomatic and transalar implant surgeries generally use a static fixed angle method to design the surgical path, which cannot overcome the complex soft and hard tissue environment and movement environment of the oral cavity.

[0003] Since the clinical operation process is long, the patient's oral tissue is pulled and moved or the patient's posture changes during the operation, resulting in relative displacement of important clinical anatomy. The fixed surgical path designed and planned in advance is very likely to pass through important clinical anatomy in the oral environment after displacement, including the orbital area, pterygoid venous plexus, skull base, and facial muscles and skin outside the zygomatic bone.

[0004] In view of this, how to dynamically calculate the optimal surgical path at each time period during transzygomatic and transalauricular implant surgery is a technical problem that urgently needs to be solved in this field. Summary of the invention

[0005] In view of the shortcomings of the prior art mentioned above, the purpose of the present application is to provide a DQN dynamic surgical path selection system, method, terminal and medium to solve the technical problem of how to dynamically calculate the optimal surgical path in each time period during transzygomatic and transalar implant surgery.

[0006] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a DQN dynamic surgical path selection system, including: a data set acquisition module, used to acquire the data set required for model training; a model training module, used to input the data set acquired by the data set acquisition module into the DQN model for training to obtain a dynamic surgical path prediction model; a model deployment module, used to deploy the trained dynamic surgical path prediction model to an external terminal device to obtain the currently input patient head image information, using the current scalpel stepping position data, the current surgical key area coordinate data, and the surgical target end position data as input data, and regenerating the next optimal surgical path at a preset interval.

[0007] In some embodiments of the first aspect of the present application, the process of acquiring a data set by the data set acquisition module includes: first, acquiring the patient's preoperative static overall head image information; the overall head image information includes hard tissue position information and soft tissue information including facial nerves and blood vessels; secondly, using the overall head image information and several types of offset information that may be generated by the enumerated soft tissue as input data to train the model.

[0008] In some embodiments of the first aspect of the present application, the state vector of the DQN model includes the stepping position of the scalpel during the operation, the current state and the reward information already obtained; the action choices output by the DQN model include the moving direction, the moving distance and the staying time.

[0009] In some embodiments of the first aspect of the present application, the model training module also performs the following during the DQN model training process: based on the experience replay algorithm, a number of experience data obtained by one or more greedy strategies are stored in a replay buffer for randomly selecting experience data from the replay buffer during subsequent training.

[0010] In some embodiments of the first aspect of the present application, the model training module applies the DQN model to the continuous action space by executing any of the following methods during the DQN model training process: based on the normalized advantage algorithm, a quadratic function is selected to fit the Q function so that the DQN model is applied to the continuous action space; or, the position coordinates of important clinical anatomical points are discretized to solve the maximum Q value of several points obtained by discretization.

[0011] In some embodiments of the first aspect of the present application, during the model training process, the model training module sets a larger reward for key surgical areas with lower risks, and sets a smaller reward for key surgical areas with higher risks; sets a larger reward for a surgical path that is farther away from the risk area, and sets a smaller reward for a surgical path that is closer to the risk area.

[0012] In some embodiments of the first aspect of the present application, the model training module sets a higher reward for the action of reaching the target area, and sets a higher reward for the surgical path angle being the same as the current scalpel angle to reduce the angle during the operation.

[0013] In some embodiments of the first aspect of the present application, the model deployment module renders and displays the optimal surgical path and the recommended surgical path range using different colors.

[0014] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a DQN dynamic surgical path selection method, including: obtaining the data set required for model training; inputting the data set acquired by the data set acquisition module into the DQN model for training to obtain a dynamic surgical path prediction model; deploying the trained dynamic surgical path prediction model to an external terminal device to obtain the currently input patient head image information, using the current scalpel stepping position data, the current surgical key area coordinate data, and the surgical target end position data as input data, and regenerating the next optimal surgical path at a preset interval.

[0015] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the DQN dynamic surgical path selection method is implemented.

[0016] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides an electronic terminal, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the DQN dynamic surgical path selection method.

[0017] As described above, the DQN dynamic surgical path selection system, method, terminal and medium of the present application have the following beneficial effects: the present application can dynamically calculate the optimal surgical path in each time period during the transzygomatic and transalar implant surgery, providing a technical solution that is easy to operate and can assist doctors in safely and efficiently performing transzygomatic implant surgery. It is also a navigation device that assists doctors in safely and efficiently performing transzygomatic implant surgery, filling the gap in the current market. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Shown is a structural schematic diagram of a DQN dynamic surgical path selection system in one embodiment of the present application.

[0019] Figure 2 Shown is a schematic diagram of the training process of the DQN model in one embodiment of the present application.

[0020] Figure 3 Shown is a schematic diagram of the training process of the DQN model in one embodiment of the present application.

[0021] Figure 4 Shown is a schematic diagram of several experiences stored in the playback buffer in one embodiment of the present application.

[0022] Figure 5 Shown is a flowchart of a DQN dynamic surgical path selection method in one embodiment of the present application.

[0023] Figure 6 Shown is a schematic diagram of the structure of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION

[0024] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0025] It should be noted that in the following description, with reference to the accompanying drawings, several embodiments of the present application are described in the accompanying drawings. It should be understood that other embodiments may also be used, and mechanical composition, structure, electrical and operational changes may be made without departing from the spirit and scope of the present application. The following detailed description should not be considered restrictive, and the scope of the embodiments of the present application is limited only by the claims of the published patents. The terms used here are only for describing specific embodiments and are not intended to limit the present application. Spatially related terms, such as "upper", "lower", "left", "right", "below", "below", "lower", "above", "upper", etc., may be used in the text to facilitate the description of the relationship between an element or feature shown in the figure and another element or feature.

[0026] In this application, unless otherwise clearly specified and limited, the terms "install", "connect", "connect", "fix", "hold" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0027] Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless there is an indication to the contrary in the context. It should be further understood that the terms "comprise", "include" indicate the presence of the described features, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Exceptions to this definition will only occur when the combination of elements, functions or operations is inherently mutually exclusive in some way.

[0028] In order to solve the problems in the above-mentioned background technology, the present invention provides a DQN dynamic surgical path selection method, device, terminal and medium, and provides a deep learning algorithm for dynamically selecting the best surgical path based on the important clinical anatomical points that have been acquired in real time. In the training of enhanced learning, the current scalpel stepping position is used as the starting point; the important clinical anatomical areas are marked, and different penalties are set for different clinical anatomies; the NAF algorithm is used to make the control output continuous; the position coordinates of important clinical anatomical points are discretized to reduce the amount of data required for training; rewards are set for reaching the target area, etc.

[0029] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the technical solution in the embodiments of the present invention is further described in detail through the following embodiments and in combination with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the invention.

[0030] Before further describing the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are applicable to the following interpretations:

[0031] <1> Transzygomatic and transalauricular implant surgery: a highly complex dental implant technique that uses the zygomatic bone and alar plate to fix the implant. The dental implant is fixed through the zygomatic bone, the implant is fixed in the zygomatic area, and then the alar plate is fixed in the area adjacent to the maxillary bone.

[0032] <2> DQN (Deep Q-network): Deep Q network refers to the Q-Learning algorithm based on deep learning.

[0033] <3> Greedy algorithm: When solving a problem, it always makes the best choice at the moment, that is, it does not consider the overall optimal solution. The algorithm obtains a local optimal solution in a certain sense.

[0034] The embodiments of the present invention provide a DQN dynamic surgical path selection method, a system for the DQN dynamic surgical path selection method, and a storage medium storing an executable program for implementing the DQN dynamic surgical path selection method. As for the implementation of the DQN dynamic surgical path selection method, the embodiments of the present invention will illustrate an exemplary implementation scenario of the DQN dynamic surgical path selection.

[0035] like Figure 1 , which shows a structural diagram of a DQN dynamic surgical path selection system in an embodiment of the present invention. The structure of the DQN dynamic surgical path selection system in this embodiment mainly includes the following: a data set acquisition module 101, a model training module 102 and a model deployment module 103.

[0036] The data set acquisition module 101 is used to acquire the data set required for model training.

[0037] In an embodiment of the present invention, the process of the data set acquisition module 101 acquiring the data set includes: first, acquiring the patient's preoperative static overall head image information; the overall head image information includes hard tissue position information and soft tissue information including facial nerves and blood vessels; secondly, using the overall head image information and several types of offset information that may be generated by the enumerated soft tissue as input data to train the model.

[0038] The model training module 102 is used to input the data set acquired by the data set acquisition module 101 into the DQN model for training to obtain a dynamic surgical path prediction model.

[0039] DQN (Deep Q-Network) refers to a reinforcement learning algorithm based on deep learning. It can directly output the actions required for a specific task from the input image without manually extracting features, thereby achieving autonomous learning in a complex environment. The goal is to learn how to make the best decision for a specific environment through a series of interactions. In reinforcement learning, the agent constantly observes the environment, makes decisions, and constantly tries to find the best behavior sequence. Unlike supervised learning, there are no labels or predetermined correct answers in reinforcement learning, so the best decision needs to be learned through reward signals and interactions with the environment.

[0040] Specifically, a deep neural network is used in the training of the DQN model. This deep neural network is the Q-Network. The input of the Q-Network is the state vector, and the output is the expected reward for each action in this state. During the model training process, the DQN algorithm continuously adjusts the parameters of the Q-Network so that it can approximate the true action-value function. In the output of the network, if the action with the largest expected reward matches the true reward, it means that the output of the Q-Network is good enough and the training can be stopped.

[0041] The training process of the DQN model is as follows Figure 2As shown, the input parameter of the DQN model is the state vector (state S), and the output is the expected reward for each action under the state vector. It should be understood that the Q-Learning algorithm maintains a Q-table table, and uses the Q-table table to store the reward obtained by taking action a under each state s, that is, the state-value function Q(s,a). This algorithm has great limitations. In many cases in reality, the state space faced by reinforcement learning tasks is continuous and there are infinite states. In this case, the value function can no longer be stored in a table. In view of this, the function Q(s,a,w) is used to approximate the state-value function Q(s,a), which is called a Q network, and w is the parameter of neural network training.

[0042] For example, state s corresponds to three actions, and the corresponding approximate state-value functions are constructed as follows: Q(s, "left"; w), Q(s, "right"; w), Q(s, "up"; w), which respectively represent the left action and its training parameters under state s, the right action and its training parameters under state s, and the up action and its training parameters under state s.

[0043] In an embodiment of the present invention, the state vector of the DQN model includes the stepping position of the scalpel during the operation, the current state and the reward information obtained; the action options output by the DQN model include the moving direction, moving distance, stay time, etc.

[0044] In the embodiment of the present invention, a corresponding reward value is set for each action in the training process of the DQN model. In the DQN algorithm, the reward value is usually a real number, which will affect the surgical path finally selected by the algorithm. If an action increases its reward value, the algorithm is more likely to select the action; if an action cannot increase its reward value, the algorithm is more likely to reduce the frequency of selecting the action.

[0045] In the embodiment of the present invention, combined with Figure 3 The training process of the DQN model is further explained. The steps include the following:

[0046] Step S3A: Initialize the Q network and enter state S t , output S t The reward value Q value of all actions.

[0047] Step S3B: Using the greedy strategy, select an action a t , and the action a t Input into the environment to obtain a new state S t+1 and r;

[0048] Step S3C: Calculate target value: y t =rt +γ·max a Q(S t+1 , a; w); where r t Indicates the value obtained based on actual observation; max a Q(S t+1 , a; w) represents the state of the Q network in S t+1 γ is an estimated value made when ; γ represents a constant.

[0049] Step S3D: Calculate the loss function as follows: L = 1 / 2[y t -Q(s,a;w)] 2 .

[0050] Step S3E: Update the Q parameter so that Q(S t , a t ) as close to y as possible t .

[0051] Step S3F: From the above steps, we can get a four-tuple empirical transition: (s t , a t , r t ,s t+1 ) and discard after use.

[0052] Step S3G: Input the new state and repeat the above steps to update the model.

[0053] It should be noted that the greedy strategy means that when solving a problem, the best choice is always made at the moment, that is, the algorithm does not consider the overall optimal solution, and the solution obtained by the algorithm is a local optimal solution in a certain sense. The execution process of the greedy algorithm is as follows: establish a mathematical model to describe the problem; divide the problem to be solved into several sub-problems; solve each sub-problem to obtain the local optimal solution of the sub-problem; synthesize the local optimal solution of the sub-problem into a solution to the original problem. The characteristic of the greedy algorithm is that it proceeds step by step, often based on the current situation and making the best choice according to a certain optimization measure, without considering various possible overall situations, saving a lot of time that must be spent to exhaust all possibilities to find the optimal solution. The greedy algorithm uses a top-down, iterative method to make successive greedy choices. Each time a greedy choice is made, the problem is simplified into a smaller sub-problem. Through each step of greedy choice, an optimal solution to the problem can be obtained. Although it is necessary to ensure that the local optimal solution can be obtained at each step, the resulting global solution is sometimes not necessarily the best, so the greedy algorithm does not backtrack.

[0054] Preferably, during the DQN model training process, each experience data transition is discarded after being used. This not only wastes computing resources but also slows down the model training speed. Therefore, in the embodiment of the present invention, based on the experience replay algorithm, several experience data obtained by one or more greedy strategies are stored in the replay buffer, so that experience data can be randomly selected from the replay buffer during subsequent training. For example, the replay buffer stores several experience data such as Figure 4 As shown, including empirical data 1(s t , a t , r t ,s t+1 ); empirical data 2(s t+1 , a t+1 , r t+1 ,s t+2 ); empirical data 3(s t+2 , a t+2 , r t+2 ,s t+3 )etc.

[0055] In some examples, the experience replay process of the experience replay algorithm includes the following: storing a preset number of experience data obtained during model training in a replay buffer; storing the currently newly observed experience data in the replay buffer, and deleting the earliest experience data stored in the replay buffer so that the number of experiences stored in the replay buffer does not exceed a preset number; when the DQN model needs to be updated, randomly sampling a small batch of experience data from the replay buffer; and using the sampled experience data for training to update the strategy of the DQN model.

[0056] In the embodiment of the present invention, although the DQN model can select actions by calculating the maximum value of the Q value, it is not efficient and applicable in dealing with the problem of continuous actions, especially for surgical procedures that require continuous and smooth actions. This is because if there are only a few discrete actions, these actions can be substituted into the DQN model to calculate the Q value and then select the corresponding action, but if the action is continuous, the action at this time is a vector, and the vector has corresponding values, and it is impossible to enumerate all actions to calculate the corresponding Q value. In view of this, the embodiment of the present invention provides the following two methods to apply the deep neural network Q-Learning to the continuous action space.

[0057] Method 1) Based on the NAF (Normalized Advantage Functions) normalized advantage algorithm, the deep neural network Q-Learning is applied to the continuous action space. Specifically, a cluster of functions that are easy to optimize is selected to fit the Q function, and the functions that are easy to optimize include quadratic functions. Take the quadratic function as an example:

[0058]

[0059] Thus, a neural network or other structure is trained, with the input being the state s and the output being (μ, P, V), where μ and V are both vectors and P is a matrix; This method does not require any algorithmic changes compared to the original discontinuous QDN algorithm, is very easy, and is equally efficient.

[0060] Method 2) Discretize the position coordinates of important clinical anatomical points and solve the maximum Q value of several discretized points. Take N points obtained by discretization as an example:

[0061] More preferably, during the model training process, the model training module 102 sets a larger reward for key surgical areas with lower risks, and sets a smaller reward for key surgical areas with higher risks; sets a larger reward for a surgical path that is farther away from the risk area, and sets a smaller reward for a surgical path that is closer to the risk area.

[0062] Preferably, the model training module 102 sets a higher reward for the action of reaching the target area, and sets a higher reward for the surgical path angle being the same as the current scalpel angle to reduce the angle during the operation.

[0063] The model deployment module 103 is used to deploy the trained dynamic surgical path prediction model to an external terminal device to obtain the currently input patient head image information, use the current scalpel stepping position data, the current surgical key area coordinate data, and the surgical target end position data as input data, and regenerate the next optimal surgical path at a preset interval.

[0064] Preferably, after the model deployment module 103 generates the next best surgical path, it renders and annotates the best surgical path and the recommended surgical path range with different colors and displays them for clear display, so as to better assist doctors in making decisions. For example, the recommended surgical path range can be rendered in blue, while the best surgical path can be set in green, etc. This example is for reference only and is not intended to be limiting.

[0065] More preferably, the current scalpel stepping position data includes the scalpel's orientation data and coordinate data.

[0066] More preferably, the regenerating the next optimal surgical path at a preset interval time includes generating an angle range of the surgical path and recommending a local optimal angle and distance.

[0067] It should be noted that: the DQN dynamic surgical path selection system provided in the above embodiment only uses the division of the above program modules as an example when performing DQN dynamic surgical path selection. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above.

[0068] In the above, the structure and implementation process of a DQN dynamic surgical path selection system in an embodiment of the present invention are explained in detail. In the following, another DQN dynamic surgical path selection method, terminal and medium in an embodiment of the present invention will be further explained in combination with the embodiment.

[0069] like Figure 5 As shown, a schematic diagram of a process flow of a DQN dynamic surgical path selection method in an embodiment of the present invention is shown. The process flow of the DQN dynamic surgical path selection method in an embodiment of the present invention includes the following:

[0070] Step S51: Obtain the data set required for model training.

[0071] In some examples, the process of acquiring a data set includes: first, acquiring static overall head image information of the patient before surgery; the overall head image information includes hard tissue position information and soft tissue information including facial nerves and blood vessels; second, using the overall head image information and several types of offset information that may be generated by the enumerated soft tissue as input data to train the model.

[0072] In some examples, the state vector of the DQN model includes the stepping position of the scalpel during the operation, the current state, and the reward information that has been obtained; the action choices output by the DQN model include the moving direction, moving distance, and stay time.

[0073] Step S52: inputting the data set acquired by the data set acquisition module into the DQN model for training to obtain a dynamic surgical path prediction model.

[0074] In some examples, the model training module also performs the following during the DQN model training process: based on the experience replay algorithm, a number of experience data obtained by one or more greedy strategies are stored in a replay buffer for randomly selecting experience data from the replay buffer during subsequent training.

[0075] In some examples, the model training module applies the DQN model to the continuous action space by executing any of the following methods during the DQN model training process: based on the normalized advantage algorithm, a quadratic function is selected to fit the Q function so that the DQN model is applied to the continuous action space; or, the position coordinates of important clinical anatomical points are discretized to solve the maximum Q value of several points obtained by discretization.

[0076] In some examples, during the model training process, the model training module sets a larger reward for key surgical areas with lower risks and a smaller reward for key surgical areas with higher risks; sets a larger reward for a surgical path that is farther away from the risk area and a smaller reward for a surgical path that is closer to the risk area.

[0077] In some examples, the model training module sets a higher reward for the action of reaching the target area, and sets a higher reward for the surgical path angle being the same as the current scalpel angle to reduce the angle during the operation.

[0078] Step S53: Deploy the trained dynamic surgical path prediction model to an external terminal device to obtain the currently input patient head image information, use the current scalpel step position data, the current surgical key area coordinate data, and the surgical target end position data as input data, and regenerate the next optimal surgical path at a preset interval.

[0079] In some examples, after the model is deployed, the optimal surgical path and the recommended surgical path range are rendered and displayed using different colors.

[0080] It should be noted that the DQN dynamic surgical path selection method provided in the above embodiment and the DQN dynamic surgical path selection system embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0081] The DQN dynamic surgical path selection method provided in the embodiment of the present invention can be implemented on the terminal side or the server side. As for the hardware structure of the DQN dynamic surgical path selection terminal, please refer to Figure 6, is an optional hardware structure diagram of the DQN dynamic surgical path selection terminal 600 provided in an embodiment of the present invention. The terminal 600 may be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, etc. The DQN dynamic surgical path selection terminal 600 includes: at least one processor 601, a memory 602, at least one network interface 604 and a user interface 606. The various components in the device are coupled together through a bus system 605. It can be understood that the bus system 605 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 605 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 6 In the specification, various buses are labeled as bus systems.

[0082] The user interface 606 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0083] It is understood that the memory 602 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0084] The memory 602 in the embodiment of the present invention is used to store various types of data to support the operation of the DQN dynamic surgical path selection terminal 600. Examples of these data include: any executable program used to operate on the DQN dynamic surgical path selection terminal 600, such as an operating system 6021 and an application 6022; the operating system 6021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 6022 can include various applications, such as a media player (MediaPlayer), a browser (Browser), etc., for implementing various application services. The DQN dynamic surgical path selection method provided in the embodiment of the present invention can be included in the application 6022.

[0085] The method disclosed in the above embodiment of the present invention can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 601 or the instruction in the form of software. The above processor 601 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 601 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 601 may be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0086] In an exemplary embodiment, the DQN dynamic surgical path selection terminal 600 can be used by one or more application-specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0087] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0088] In the embodiments provided in the present application, the computer readable and writable storage medium may include a read-only memory, a random access memory, an EEPROM, a CD-ROM or other optical disk storage device, a disk storage device or other magnetic storage device, a flash memory, a USB flash drive, a mobile hard disk, or any other medium that can be used to store a desired program code in the form of an instruction or data structure and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if the instruction is sent from a website, a server or other remote source using a coaxial cable, an optical fiber cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, optical fiber cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. However, it should be understood that computer readable and writable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but are intended to be non-temporary, tangible storage media. Disk and disc, as used in this application, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[0089] In summary, the present application provides a DQN dynamic surgical path selection system, method, terminal and medium. The present application can dynamically calculate the optimal surgical path in each time period during the transzygomatic and transalar implant surgery, providing a simple operation and a technical solution that can assist doctors to safely and efficiently perform transzygomatic implant surgery, and is a navigation device that assists doctors to safely and efficiently perform transzygomatic implant surgery, filling the gap in the current market. Therefore, the present application effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0090] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A DQN dynamic surgical path selection system, characterized in that: include: The data set acquisition module is used to obtain the data set required for model training; A model training module, used for inputting the data set acquired by the data set acquisition module into the DQN model for training to obtain a dynamic surgical path prediction model; The model deployment module is used to deploy the trained dynamic surgical path prediction model to an external terminal device to obtain the currently input patient head image information, use the current scalpel stepping position data, the current surgical key area coordinate data, and the surgical target end position data as input data, and regenerate the next optimal surgical path at a preset interval.

2. The DQN dynamic surgical path selection system according to claim 1, characterized in that: The process of acquiring a data set by the data set acquisition module includes: first, acquiring the patient's preoperative static whole head image information; the whole head image information includes hard tissue position information and soft tissue information including facial nerves and blood vessels; second, using the whole head image information and several types of offset information that may be generated by the enumerated soft tissue as input data to train the model.

3. The DQN dynamic surgical path selection system according to claim 1, characterized in that: The state vector of the DQN model includes the stepping position of the scalpel during the operation, the current state and the reward information already obtained; the action choices output by the DQN model include the moving direction, moving distance and stay time.

4. The DQN dynamic surgical path selection system according to claim 1, characterized in that: The model training module also performs the following during the DQN model training process: based on the experience replay algorithm, a number of experience data obtained by one or more greedy strategies are stored in the replay buffer, so that experience data can be randomly selected from the replay buffer during subsequent training.

5. The DQN dynamic surgical path selection system according to claim 1, characterized in that: The model training module applies the DQN model to the continuous action space by executing any of the following methods during the DQN model training process: based on the normalized advantage algorithm, a quadratic function is selected to fit the Q function so that the DQN model is applied to the continuous action space; or, the position coordinates of important clinical anatomical points are discretized to solve the maximum Q value of several points obtained by discretization.

6. The DQN dynamic surgical path selection system according to claim 1, characterized in that: During the model training process, the model training module sets a larger reward for key surgical areas with lower risks, and sets a smaller reward for key surgical areas with higher risks; sets a larger reward for surgical paths that are farther away from the risk area, and sets a smaller reward for surgical paths that are closer to the risk area.

7. The DQN dynamic surgical path selection system according to claim 1, characterized in that: The model training module sets a higher reward for the action of reaching the target area, and sets a higher reward for the surgical path angle to be the same as the current scalpel angle to reduce the angle during the operation.

8. The DQN dynamic surgical path selection system according to claim 1, characterized in that: The model deployment module renders and displays the optimal surgical path and the recommended surgical path range using different colors.

9. A DQN dynamic surgical path selection method, characterized in that: include: Get the data set required for model training; Inputting the data set acquired by the data set acquisition module into the DQN model for training to obtain a dynamic surgical path prediction model; The trained dynamic surgical path prediction model is deployed to an external terminal device to obtain the currently input patient head image information, use the current scalpel step position data, the current surgical key area coordinate data, and the surgical target endpoint position data as input data, and regenerate the next optimal surgical path at a preset interval.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the DQN dynamic surgical path selection method according to claim 10 is implemented.

11. An electronic terminal, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory, so that the terminal performs the DQN dynamic surgical path selection method as claimed in claim 10.

Images