Pose reconstruction method, apparatus and system

Abstract

The application provides a pose reconstruction method, device and system, and relates to the technical field of medicine. The method comprises the following steps: obtaining target information collected by an OFDR system when performing shape sensing on a target; the target information is used to reflect optical signal information of the target at different positions; positioning the target based on the target information to obtain an end position; performing reverse pose reconstruction on the target by taking the end position as a starting point to obtain initial pose information; the initial pose information comprises a plurality of shape points of the target; performing shape optimization on the target based on each shape point in the initial pose information to obtain optimized target pose information; and the target pose information is used to describe the spatial pose of the target. The application solves the problem that the accuracy of pose reconstruction in related technologies is not high.

Description

Technical Field

[0001] This application relates to the field of medical technology, and more specifically, to a posture reconstruction method, apparatus, and system. Background Technology

[0002] In modern minimally invasive surgery (MIS), continuum and soft robots, due to their flexibility and miniature size, allow surgical devices to reach areas that are difficult for the human body to access, especially in surgeries involving complex anatomical structures. Precise surgical navigation is crucial for reducing surgical risks and improving success rates.

[0003] However, traditional surgical navigation systems have limitations in real-time accuracy and response speed in complex environments. Furthermore, electromagnetic interference in the surgical environment can affect the performance of traditional electromagnetic navigation devices, further reducing the reliability of the navigation system.

[0004] Currently, with the development of fiber optic sensing technology, the use of fiber optics for shape sensing is becoming increasingly popular. Fiber optics have advantages such as high sensitivity, small size, and resistance to electromagnetic interference, making them particularly suitable for shape sensing in complex surgical environments. However, because fiber optics are subjected to multiple bends or pressures during surgery, signal distortion can occur, thus limiting the continuity and accuracy of the shape and failing to meet the high-precision spatial attitude reconstruction requirements of complex instrument shapes.

[0005] As can be seen from the above, the problem of low accuracy in attitude reconstruction still needs to be solved. Summary of the Invention

[0006] This application provides a posture reconstruction method, apparatus, system, electronic device, storage medium, and computer program product, which can solve the problem of low posture reconstruction accuracy in related technologies. The technical solutions are as follows:

[0007] According to one aspect of this application, an attitude reconstruction method is applied to a server, the server interacting with an optical frequency domain reflectance (OFDR) system, the method comprising: acquiring target information collected by the OFDR system through shape sensing of a target; the target information reflecting optical signal information of the target at different positions; locating the target based on the target information to obtain an end position; using the end position as a starting point to perform reverse attitude reconstruction of the target to obtain initial attitude information; the initial attitude information including multiple shape points of the target; optimizing the shape of the target based on each of the shape points in the initial attitude information to obtain optimized target attitude information; the target attitude information describing the spatial attitude of the target.

[0008] According to one aspect of this application, an attitude reconstruction device is applied to a server, the server interacting with an optical frequency domain reflectance (OFDR) system. The device includes: an acquisition module for acquiring target information collected by the OFDR system through shape sensing of a target; the target information reflecting optical signal information of the target at different positions; a positioning module for locating the target based on the target information to obtain an end position; a reconstruction module for performing reverse attitude reconstruction on the target using the end position as a starting point to obtain initial attitude information; the initial attitude information including multiple shape points of the target; and an optimization module for optimizing the shape of the target based on each of the shape points in the initial attitude information to obtain optimized target attitude information; the target attitude information describing the spatial attitude of the target.

[0009] In an exemplary embodiment, the reconstruction module is configured to select an applicable path for the target according to a path planning method to obtain a reference path; take the end position as the starting point and the known initial position of the target as the ending point, and extend the shape of the target from the starting point to the ending point according to the reference path to obtain the initial posture information.

[0010] In an exemplary embodiment, the reconstruction module is configured to generate the position of the previous shape point for the end position according to the reference path; if the position of the previous shape point does not conform to the initial position, then the position of the previous shape point is taken as the current position, and the position of the previous shape point is generated for the current position according to the reference path, until the position of the previous shape point conforms to the initial position; and the initial posture information is obtained based on the shape points generated between the end position and the initial position.

[0011] In an exemplary embodiment, the optimization module is configured to perform global optimization on each of the shape points in the initial pose information to obtain intermediate pose information; the intermediate pose information includes the coarse position corresponding to each shape point; perform local optimization on the coarse position corresponding to each shape point to obtain the fine position corresponding to each shape point; and obtain the target pose information based on the fine position corresponding to each shape point.

[0012] In an exemplary embodiment, the optimization module is configured to treat each shape point in the initial posture information as a particle and initialize each particle; under path constraints, each particle moves in a set multidimensional space; during the movement, the velocity and / or position of each particle is iteratively updated until the iteration condition is met; and based on each particle, the intermediate posture information is obtained.

[0013] In an exemplary embodiment, the optimization module is used to perform local gradient adjustment on the coarse position corresponding to each shape point; through the local gradient adjustment, the error between each shape point and the reference shape is minimized to obtain the fine position of each shape point; the reference shape is obtained based on the coarse position of each shape point according to a set rule.

[0014] In an exemplary embodiment, the target localization is achieved through a coordinate localization model; the coordinate localization model includes an input layer, an encoding layer, a flattening layer, a fully connected layer, and an output layer; the localization module is used to input the target information into the coordinate localization model using the input layer, extract the features of the target information through the encoding layer to obtain the target features; convert the target features into a one-dimensional vector through the flattening layer, and map the one-dimensional vector through the fully connected layer to obtain the end position of the target; and output the end position of the target through the output layer.

[0015] According to one aspect of this application, an attitude reconstruction device is applied to an OFDR system. The device includes: a shape sensing module for sensing the shape of a target and obtaining optical signal information of the target at different positions; and an information processing module for generating target information from the optical signal information and sending the target information to a server, so that the server reconstructs the attitude of the target according to the attitude reconstruction method described above.

[0016] In an exemplary embodiment, the shape sensing module includes a laser emitting unit and a main interferometer module, and the information processing module includes an acquisition unit: the laser emitting unit is used to output a laser signal; the main interferometer unit is used to divide the laser signal into a probe light signal and a reference light signal; control the probe light signal to enter the target to receive light signal information reflected from the target at different positions; perform optical interference with the reference light signal based on each of the light signal information to generate interference fringe signals; and the acquisition unit is used to acquire the interference fringe signals and perform signal conversion based on the interference fringe signals to generate target information.

[0017] According to one aspect of this application, a posture reconstruction system is provided, the system comprising an interactive OFDR system and a server, wherein the OFDR system is used to perform shape sensing on a target to obtain optical signal information of the target at different positions; generate target information from the optical signal information; the server is used to acquire the target information; locate the target based on the target information to obtain an end position; perform reverse posture reconstruction on the target using the end position as a starting point to obtain initial posture information; the initial posture information includes multiple shape points of the target; optimize the shape of the target based on each shape point in the initial posture information to obtain optimized target posture information; the target posture information is used to describe the spatial posture of the target.

[0018] The beneficial effects of the technical solution provided in this application are:

[0019] In the above technical solution, this solution uses the end position for reverse attitude reconstruction. By predicting and reversing the end position, the accumulation of errors can be effectively reduced. Combined with shape optimization technology, the optimized target attitude information is more in line with the actual spatial attitude of the target, which can better meet the accuracy requirements in complex scenarios and greatly improve the accuracy of attitude reconstruction. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a structural block diagram of an OFDR system according to an exemplary embodiment;

[0022] Figure 2 This is a flowchart illustrating an attitude reconstruction method according to an exemplary embodiment;

[0023] Figure 3 yes Figure 2 A schematic diagram illustrating the specific implementation of the coordinate positioning model involved in the corresponding embodiment;

[0024] Figure 4 yes Figure 2 A flowchart of step 350 in one embodiment corresponds to the following example;

[0025] Figure 5 yes Figure 2 A flowchart of step 370 in one embodiment corresponds to the following example;

[0026] Figure 6 yes Figure 5 A flowchart of step 371 in one embodiment corresponds to the following example;

[0027] Figure 7 This is a schematic diagram illustrating the specific implementation of a posture reconstruction method in an application scenario;

[0028] Figure 8 yes Figure 7 A schematic diagram illustrating the specific implementation of global and local optimization of the target involved;

[0029] Figure 9 This is a structural block diagram of an attitude reconstruction device according to an exemplary embodiment. Detailed Implementation

[0030] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0031] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this disclosure means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0032] Minimally Invasive Surgical Incision (MIS) is an advanced surgical technique that allows surgeons to access a patient's body through small incisions or natural cavities. Compared to traditional open surgery, MIS offers numerous benefits, such as reduced trauma, pain relief, faster recovery, and shorter hospital stays. MIS has important applications in endoscopic surgery, interventional vascular surgery, and ablation procedures. MIS has been continuously evolving since the 1990s. However, some limitations remain. MIS procedures typically rely on various endoscopes and other precision instruments, the operation of which requires a high level of skill and experienced surgeons. Especially in complex anatomical structures or confined surgical spaces, operators not only need to perform precise manipulations within a limited field of vision but also need to overcome the physical limitations of instrument manipulation, such as insufficient tactile feedback and limited instrument flexibility. These factors can increase surgical risks and even lead to intraoperative complications. Furthermore, MIS procedures often rely on imaging technologies such as X-rays, ultrasound, or CT scans for intraoperative navigation. However, these imaging methods sometimes fail to provide sufficient real-time performance and resolution, preventing surgeons from fully and accurately determining the location and boundaries of lesions during surgery, thus affecting the surgical outcome.

[0033] To address these challenges, various sensing methods, including fluorescence fluoroscopy, electromagnetic (EM) tracking systems, and electrical impedance tomography (EIT), have been proposed to acquire the shape of interventional devices. Unfortunately, fluorescence fluoroscopy emits harmful radiation and only provides two-dimensional images of the patient; EM systems are sensitive to the presence of metallic materials and are highly susceptible to the complex electromagnetic environment of the operating room; furthermore, using EM systems for shape sensing requires integrating different coils or electrodes within the surgical instrument structure, leading to increased size; and computed tomography also poses significant radiation hazards. These issues complicate its application in clinical practice.

[0034] In recent years, with the development of fiber optic sensing technology, the use of fiber optics for shape sensing has become increasingly popular. Fiber optics offer advantages such as high sensitivity, small size, and resistance to electromagnetic interference, making them particularly suitable for shape sensing in complex surgical environments. Fiber Bragg gratings (FBGs) and optical frequency domain reflectance (OFDR) are two common fiber optic sensing technologies. FBG technology measures shape and strain by embedding multiple gratings within the fiber. Its main advantages are high sensing speed, suitability for real-time monitoring of instrument movement, and relatively simple hardware integration. OFDR technology, on the other hand, utilizes phase information in the optical frequency domain to acquire strain and deformation information of the fiber through interferometry, providing extremely high resolution and sensitivity, making it suitable for high-precision spatial attitude reconstruction tasks.

[0035] However, OFDR and FBG present some challenges in practical applications. When using OFDR for shape sensing, the reflected signal at the fiber end may be affected by mechanical forces, especially when the fiber is subjected to multiple bends or pressures during surgery, leading to signal distortion. In addition, since the OFDR demodulation process relies on phase information, it is necessary to maintain the stability of the fiber to avoid demodulation errors caused by positional offsets or bending. When using FBG for shape sensing, the FBG demodulation process is limited by the position and number of gratings, and can usually only perform strain measurements on discrete points of the fiber. Therefore, it has limitations in terms of shape continuity and accuracy, and cannot meet the high-precision spatial attitude reconstruction requirements of complex instrument shapes.

[0036] Currently, a method using artificial neural networks (ANNs) to replace traditional shape reconstruction models has been proposed. This reduces the dependence on the precise position of the fiber relative to the centerline, thereby simplifying the fiber integration process and improving overall accuracy. Furthermore, this method achieves high-precision reconstruction of surgical instrument shapes by training a single bias-placed multi-core fiber, reducing the occupancy of the central channel, and exhibiting superior reconstruction performance in both free space and constrained environments.

[0037] However, the aforementioned method still relies on traditional model-based reconstruction methods during shape reconstruction. A drawback of this approach is its dependence on integral calculations. In practical applications, the integration process can lead to the gradual accumulation of errors, especially in long optical fibers or sensing links, where the impact of these errors is amplified. Ultimately, the accuracy of the end-position is significantly affected by the accumulated errors, making it unsuitable for some high-precision applications. Therefore, this method has certain limitations in terms of end-position accuracy in shape reconstruction.

[0038] Furthermore, the human body contains many cavities with very small radii of curvature. Due to the limitations of FBG (Fast-Filled Geometry) technology, it becomes physically constrained when encountering small radii of curvature, making it difficult to accurately capture deformation. This causes a significant decrease in the measurement accuracy and sensitivity of FBG sensors under conditions of high curvature, thus affecting the accuracy of overall shape perception and reconstruction.

[0039] As can be seen from the above, the relevant technologies still suffer from low accuracy in pose reconstruction.

[0040] Therefore, the attitude reconstruction method provided in this application can effectively improve the accuracy of attitude reconstruction. Accordingly, the attitude reconstruction method is applicable to an attitude reconstruction system, which includes a server and an OFDR system.

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0042] Figure 1 This is a schematic diagram of an implementation environment involved in a posture reconstruction method. It should be noted that this implementation environment is merely an example adapted to the present invention and should not be considered as providing any limitation on the scope of the invention.

[0043] The implementation environment includes attitude reconfiguration device 110 and server 130.

[0044] like Figure 1 As shown, in some embodiments, the attitude reconstruction device 110 is applied to an OFDR system. The attitude reconstruction device 110 includes: a shape sensing module for sensing the shape of a target and obtaining light signal information of the target at different positions; and an information processing module for generating target information from the light signal information and sending the target information to the server 130, so that the server 130 can reconstruct the attitude of the target.

[0045] In one possible implementation, the shape sensing module includes a laser emitting unit and a main interferometer module, and the information processing module includes an acquisition unit: a laser emitting unit for outputting laser signals; a main interferometer unit for dividing the laser signals into probe light signals and reference light signals; controlling the probe light signals to enter the target to receive light signal information reflected back from different positions on the target; performing optical interference with the reference light signals based on the light signal information to generate interference fringe signals; and an acquisition unit for acquiring the interference fringe signals and performing signal conversion based on the interference fringe signals to generate target information.

[0046] First, it should be noted that the reference light signal can be used as the comparison light for the interference fringe signal, and is used to perform optical interference with the light signal information reflected back from the target. The probe light signal is used to emit to the target (such as an optical fiber or other target), and the light signal reflected back carries the spatial position information of the target, thereby obtaining the light signal information (such as strain or shape change).

[0047] Regarding the main interferometer unit, a beam splitter can be used to divide the laser signal into a reference light signal and a probe light signal. The returned light signals are then combined with the reference light signal to obtain the interference fringe signal. Because of the interference fringe signal, the acquisition unit converts it into a digital signal to obtain target information, facilitating subsequent attitude reconstruction by the server.

[0048] In one possible embodiment, the attitude reconstruction device further includes an auxiliary reflector unit, a wavelength calibration unit, and an electronically controlled clock synchronization unit.

[0049] Among them, the auxiliary reflector unit can be used to compensate for signal attenuation and noise problems in the target, ensuring that the signal quality meets the requirements of interferometric measurement; the wavelength calibration unit is used to perform precise wavelength control and real-time calibration of the laser emission unit to match the high requirements of the OFDR system for spectral resolution; the electronically controlled clock synchronization unit is used to control the timing and synchronization operation of each unit.

[0050] Server 130 can be an electronic device such as a desktop computer, laptop computer, or server, or it can be a computer cluster consisting of multiple servers, or even a cloud computing center consisting of multiple servers. Server 130 is used to provide backend services, such as, but not limited to, attitude reconstruction services.

[0051] The server 130 and the attitude reconstruction device 110 establish a network communication connection in advance via wired or wireless means, and data transmission between the server 130 and the attitude reconstruction device 110 is realized through this network communication connection. The transmitted data includes, but is not limited to, target information, etc.

[0052] In one application scenario, through the interaction between the attitude reconstruction device 110 and the server 130, the attitude reconstruction device 110 records the reflection spectrum data of the target, that is, the light signal of the target at different positions, and obtains the target information. The attitude reconstruction device then uploads the target information to the server 130 to request the server 130 to provide attitude reconstruction services.

[0053] For server 130, after receiving the target information uploaded by attitude reconstruction device 110, it calls attitude reconstruction service to reconstruct the attitude of the target based on the target information and obtain the target attitude information of the target, thereby solving the problem of poor attitude reconstruction effect in related technologies.

[0054] In one possible implementation, a gesture reconstruction system includes: an interactive OFDR system and a server.

[0055] Among them, the OFDR system is used to sense the shape of the target and obtain the light signal information of the target at different positions; the target information is generated from the light signal information.

[0056] The server is used to acquire target information; locate the target based on the target information to obtain the end position; use the end position as the starting point to reconstruct the target's attitude in reverse to obtain initial attitude information; the initial attitude information includes multiple shape points of the target; optimize the target's shape based on each shape point in the initial attitude information to obtain optimized target attitude information; the target attitude information is used to describe the target's spatial attitude.

[0057] Therefore, the OFDR system rapidly senses the shape of the target to obtain its spatial information, which is then sent to the server. The server's attitude reconstruction method quickly processes this information and outputs the target's precise spatial attitude, supporting real-time feedback and ensuring the accuracy of surgery or equipment operation. For example, in minimally invasive surgery, the OFDR system can acquire the optical signal information of the fiber optic cable embedded in the instrument in real time, generate precise target information, and send it to the server. Combined with attitude reconstruction methods, the spatial attitude of the fiber optic cable can be further calculated, providing real-time and accurate reference information for the surgery.

[0058] Please see Figure 2 This application provides an attitude reconstruction method, which is applicable to electronic devices. For example, the electronic device may be... Figure 1 The server 130 in the implementation environment is shown.

[0059] In the following method embodiments, for ease of description, the execution subject of each step of the method is an electronic device, but this does not constitute a specific limitation.

[0060] like Figure 2 As shown, the method may include the following steps:

[0061] Step 310: Obtain target information collected by the OFDR system through shape sensing of the target.

[0062] Among them, the target information is used to reflect the optical signal information of the target at different locations. The target refers to the optical fiber, which can be a multi-fiber cluster or a multi-core optical fiber, without limitation.

[0063] Shape sensing refers to measuring the geometry and position of a target to determine its attitude and location in space. In one possible implementation, shape sensing can be achieved using an OFDR system.

[0064] Specifically, the OFDR system can read the reflection spectrum data of the multi-fiber clusters / multi-core fibers embedded in minimally invasive surgical instruments (i.e., the light signals of the target at different locations) to obtain target information.

[0065] It should be noted that changes in the optical signal in the target information can reflect the curvature and deformation of each small segment of the target, helping to analyze the physical state of the target at various minute locations, thus providing basic support for subsequent positioning and attitude reconstruction of the fiber optic end.

[0066] Regarding the acquisition of target information, the target information can originate from real-time capture and acquisition by the optical frequency domain reflection module, or it can be target information pre-stored in the electronic device for a historical time period acquired by the optical frequency domain reflection module. Therefore, for the electronic device, after the optical frequency domain reflection module acquires the target information, it can process the target information in real time, or it can pre-store it for later processing. For example, it can process the target information when the CPU of the electronic device is low, or it can process images according to the instructions of the staff. Thus, the attitude reconstruction in this embodiment can be based on target information acquired in real time or target information acquired over a historical time period; no specific limitation is made here.

[0067] Step 330: Locate the target based on the target information to obtain the end position.

[0068] First, it should be noted that the end position refers to the outermost point of the target in three-dimensional space. If the target is an optical fiber, then the end position is the outermost point of the optical fiber in three-dimensional space. It can be understood that when performing surgery using surgical instruments embedded with optical fibers, the end position can be considered as the position of the critical working end or contact end of the surgical instrument.

[0069] It is understandable that the tip of a surgical instrument is usually the part that directly contacts the patient's tissue, which may be in an invisible area inside the human body. With the help of the position of the end of the optical fiber, high-precision guidance can be achieved in the invisible area. In other words, accurately determining the position of the end can greatly improve the positioning accuracy of the surgery.

[0070] Therefore, the end position is an important point for reconstructing the shape of the optical fiber and inferring the posture of the surgical instrument. In addition, by combining the positions of other shape points, the spatial state of the surgical instrument can be fully described.

[0071] In one possible implementation, the target is located using a coordinate localization model to obtain the end position. The coordinate localization model is a trained machine learning model capable of locating the end position of the target. The coordinate localization model includes an input layer, an encoding layer, a flattening layer, a fully connected layer, and an output layer.

[0072] Specifically, the localization process includes: inputting target information into the coordinate localization model using the input layer, extracting features of the target information through the encoding layer to obtain target features; converting the target features into a one-dimensional vector through the flattening layer, and mapping the one-dimensional vector through the fully connected layer to obtain the end position of the target; and outputting the end position of the target through the output layer.

[0073] Figure 3 The diagram illustrates a specific implementation of a coordinate positioning model, such as... Figure 3As shown, the coordinate localization model includes an input layer, an encoding layer, a flattening layer, a fully connected layer, and an output layer. The output layer directly outputs the three-dimensional coordinates (x, y, z) of the target's end position. The encoding layer includes multiple one-dimensional convolutional layers, each followed by a batch normalization layer, a ReLU activation function layer, and a max-pooling layer to extract target features. The output of the convolutional layers undergoes dimensionality reduction processing through a max-pooling layer, then the flattening layer transforms the multi-dimensional target features into a one-dimensional vector. The fully connected layer further optimizes and maps the one-dimensional vector to obtain high-precision three-dimensional coordinates of the target's end position. Finally, the output layer outputs the high-precision three-dimensional coordinates of the target's end position. This coordinate localization model achieves high-precision conversion from complex optical signals to end positions by combining signal features of target information with coordinate prediction, making it suitable for fiber optic end-position localization requirements in minimally invasive surgical scenarios.

[0074] For example, if the target is an optical fiber, the target information of the optical fiber is input into the coordinate positioning model and encoded into signal features for extracting target information, such as light intensity change patterns and phase information; the flattening layer and the fully connected layer integrate the extracted signal features into a low-dimensional spatial expression and establish a mapping between signal features and coordinates; the output layer outputs the predicted end position (x, y, z).

[0075] Step 350: Using the end position as the starting point, reverse the attitude reconstruction of the target to obtain the initial attitude information.

[0076] The initial attitude information includes multiple shape points of the target, which reflect the geometric features of the target at different positions.

[0077] First, it should be noted that the end position can be regarded as the end point of the fiber shape or path. Then, based on the extension of the end position, the positions of other shape points of the fiber can be determined step by step, thereby realizing attitude reconstruction.

[0078] It is understandable that if the target is an optical fiber on a surgical instrument, its initial position can be a fixed point on the surgical instrument at one end of the fiber, that is, the starting point of the fiber inside or at the entrance of the surgical instrument. Therefore, the initial position is known in three-dimensional space. In contrast to the initial position, the end position may be unknown. Therefore, the target is located through step 330 to obtain the end position of the target.

[0079] Furthermore, having obtained the initial and end positions, reverse attitude reconstruction of the target can be performed. Specifically, in one possible implementation, reverse attitude reconstruction includes: determining the path of the target from the initial position to the end position, which describes the overall shape of the optical fiber in space, including bending, twisting, and extension, thereby obtaining multiple shape points of the target from the initial position to the end position; connecting these multiple shape points from the initial position to the end position generates the initial attitude information of the target, which is used to describe the target's attitude in three-dimensional space obtained from the reverse attitude reconstruction.

[0080] It's important to note that while the target's initial position is known, not all shape points along the path can be determined. This is because forward attitude reconstruction requires deriving the motion patterns of the target at each shape point along the path. In other words, forward attitude reconstruction is susceptible to error accumulation, affecting the final overall shape accuracy. In contrast, reverse reconstruction based on the end-effector position can control these errors at the starting point of attitude generation, i.e., the initial position. Since the initial position is known, it provides calibration information for the endpoint, ensuring that the reconstructed shape of the target conforms to reality.

[0081] Step 370: Optimize the shape of the target based on each shape point in the initial pose information to obtain the optimized target pose information.

[0082] Among them, target attitude information is used to describe the spatial attitude of the target, which can be the target's position, orientation, and orientation in three-dimensional space. Taking an optical fiber as an example, the target attitude information can describe not only the degree of bending, orientation, and rotation of the optical fiber in space, but also other information.

[0083] First, it should be noted that the initial attitude information is derived from the target's end-effector position and initial position. While it can provide the target's approximate shape and spatial location, it may not perfectly reflect the actual situation. For example, the target may be affected by external forces, such as bending or twisting, leading to errors in the initial attitude information.

[0084] Therefore, it is necessary to optimize the shape of the initial pose information to obtain a more accurate target pose information by adjusting the shape points in the initial pose information. Specifically, the initial pose information can be optimized using optimization algorithms, such as gradient descent, particle swarm optimization, and genetic algorithms, which are not limited here.

[0085] Then, under the guidance of the optimization algorithm, the positions of each shape point in the initial attitude information can be gradually adjusted so that the positions of each shape point gradually approach the actual spatial attitude of the target, thereby obtaining the optimized target attitude information.

[0086] Through the above process, reverse attitude reconstruction is performed using the end position. By predicting and reversing the end position, the accumulation of errors can be effectively reduced. Combined with shape optimization technology, the optimized target attitude information is more in line with the actual spatial attitude of the target, which can better meet the accuracy requirements in complex scenarios and greatly improve the accuracy of attitude reconstruction.

[0087] Please see Figure 4 In one exemplary embodiment, step 350 may further include the following steps:

[0088] Step 351: Select an applicable path for the target according to the path planning method to obtain a reference path.

[0089] First, it should be noted that a reference path can be viewed as a theoretical path for the target's motion in three-dimensional space. It describes the geometry and dynamics from the starting point to the ending point, including features such as bending and twisting. A reference path provides a way to constrain the spatial attitude of a target based on path rules.

[0090] Regarding path planning methods, appropriate methods can be flexibly selected based on the characteristics of the target and the application scenario. For example, the applicable path can be flexibly selected based on the characteristics of the surgical instruments. Path planning methods include Frenet framework, Bishop framework, or spiral extension method, which are not limited here.

[0091] Therefore, according to the path planning method, a reference path can be formed based on the initial and final positions of the target, and this reference path conforms to the actual length of the target.

[0092] Step 353: Using the end position as the starting point and the known initial position of the target as the ending point, extend the shape of the target from the starting point to the ending point according to the reference path to obtain the initial attitude information.

[0093] As mentioned earlier, using the end position as the starting point can reduce the accumulated error. By utilizing the initial position of the end point and the constraints of the reference path, a path that fits the target is generated, thereby obtaining the initial attitude information. The initial attitude information includes a set of shape points on the path, which can describe the initial spatial attitude of the target.

[0094] In one possible implementation, step 353 may further include the following steps: generating the position of the previous shape point for the end position according to the reference path; if the position of the previous shape point does not conform to the initial position, then taking the position of the previous shape point as the current position, generating the position of the previous shape point for the current position according to the reference path, until the position of the previous shape point conforms to the initial position; obtaining the initial pose information based on the shape points generated between the end position and the initial position.

[0095] It can be understood that, starting from the end position and using the reference path as a constraint, the position of the previous shape point can be generated progressively. This is a process of generating the previous shape point by tracing backward from the end position along the reference path. The target's endpoint is understood to be the known initial position. Therefore, if the generated position of the previous shape point is not the initial position, the previous shape point is generated by tracing backward from the current position along the reference path until the generated previous shape point matches the initial position. This indicates that the reverse attitude reconstruction process has been completed, and the initial attitude information of the target can then be constructed based on the generated shape points.

[0096] Under the above embodiments, by using this reverse attitude reconstruction method, the position of the shape point is gradually derived from the end position to the initial position, which not only ensures the continuity of the target's spatial attitude, but also precisely controls the target's spatial attitude. This ensures that the target's initial attitude information accurately reflects its actual spatial attitude and avoids the error accumulation problem that may occur due to directly generating shape points from the initial position in the forward direction.

[0097] Please see Figure 5 In one exemplary embodiment, step 370 may further include the following steps:

[0098] Step 371: Perform global optimization on each shape point in the initial attitude information to obtain intermediate attitude information.

[0099] The intermediate pose information includes the approximate position of each shape point. The intermediate pose information is an intermediate result obtained during the global optimization process. It can reflect the spatial pose obtained during the optimization process of the target, but this spatial pose may not have reached the optimal solution.

[0100] Global optimization refers to finding the globally optimal solution for the spatial pose of a target within the entire three-dimensional space, rather than being limited to optimization in a local region. Unlike local optimization, global optimization considers all possible solutions, thus avoiding getting trapped in local optima.

[0101] Specifically, the initial attitude information can be globally optimized using a particle swarm optimization algorithm. One possible implementation is as follows: Figure 6 As shown, step 371 may also include the following steps:

[0102] Step 3711: Treat each shape point in the initial posture information as a particle and initialize each particle.

[0103] In the particle swarm optimization algorithm, each shape point is regarded as a "particle" that moves in space to find the optimal solution. Initialization refers to giving each particle (shape point) an initial position and an initial velocity. These values ​​can be randomly generated or initialized based on some prior knowledge.

[0104] Step 3713: Under path constraints, each particle moves in a set multidimensional space.

[0105] Among them, path constraints refer to the conditions that must be followed during the movement of particles, such as geometric constraints on the target shape and physical limitations on the target motion.

[0106] The concept of setting up a multi-dimensional space can refer to particles moving freely within a multi-dimensional coordinate system. For example, in three-dimensional space, each particle has three dimensions: x, y, and z.

[0107] By constraining the movement of particles under path rules and physical conditions, the movement of particles is ensured to maintain the rationality of the spatial shape of the target during the movement process.

[0108] Step 3715: During the movement, the velocity and / or position of each particle are iteratively updated until the iteration condition is met.

[0109] First, it should be noted that in the particle swarm optimization algorithm, the velocity and position of particles are iteratively updated based on the state of the previous moment and the guidance of the optimal solution in the particle swarm. If the movement of the particle swarm satisfies the iteration condition, it can be considered that the iterative update has been completed and the optimal solution has been obtained.

[0110] The iteration condition refers to the criterion for stopping iterative updates during the execution of the particle swarm optimization algorithm. It indicates whether each particle has completed its fallback update. The iteration condition can be that the error of the iterative update is less than a certain threshold, or that the maximum number of iterations has been reached; however, this is not limited here.

[0111] Step 3717: Obtain intermediate attitude information based on each particle.

[0112] It is understandable that by continuously iterating and updating the position and velocity of the particles, the particles gradually converge to an optimal solution, that is, the best match of the spatial attitude of the target, so as to obtain the globally optimized intermediate attitude information based on each particle.

[0113] Through the above process, using a global optimization method, the problem of local optima can be avoided, ensuring that the target's spatial pose is as close as possible to its true spatial pose globally. Global optimization ensures a reasonable distribution of shape points and optimizes the overall effect of the target's spatial pose.

[0114] Step 373: Perform local optimization on the coarse position corresponding to each shape point to obtain the fine position corresponding to each shape point.

[0115] Local optimization refers to further fine-tuning the coarse position of each shape point. Through local optimization, the result of attitude reconstruction is made closer to the actual spatial attitude of the target in detail, optimizing the shape curvature and reducing the deviation from the path or endpoint.

[0116] Local optimization can be achieved through local gradient adjustment methods, such as the Adam algorithm, which is not limited here. Specifically, in one possible implementation, step 373 may also include the following steps: performing local gradient adjustment on the coarse positions corresponding to each shape point; minimizing the error between each shape point and the reference shape through local gradient adjustment to obtain the fine position of each shape point.

[0117] First, it should be noted that local gradient adjustment refers to the process of fine-tuning the rough position of each shape point through gradient descent and other optimization methods, thereby gradually reducing the error.

[0118] The reference shape is obtained based on the rough position of each shape point according to the set rules. The reference shape can provide an ideal spatial pose framework as the basis for local optimization of shape points, ensuring that the local optimization process proceeds in the right direction.

[0119] In one possible implementation, the rules can be defined as a local optimization model, which is a machine learning model that has been trained and has the ability to locally optimize the coarse positions of each shape point. Specifically, the training process of the local optimization model can include: acquiring sample pose information, which includes the sample space pose and sample shape points of multiple samples, where the sample space pose is the actual spatial pose of the sample; and continuously updating the model parameters by minimizing the error between the reference shape and the sample space pose. The position of each shape point is locally gradient adjusted during training, so that the fitting degree of the local optimization model gradually improves. The final local optimization model should be able to generate a reference shape that is as close as possible to the actual spatial pose of the target when the coarse position of each shape is input.

[0120] Therefore, by adjusting the local gradient, the error between the position of the shape point and the reference shape can be gradually reduced, so that the final position of the shape point is as close as possible to the reference shape. When the error is minimized, the precise position of each shape point can be obtained.

[0121] Through the above process, after global optimization, the fine position of each shape point is adjusted through local optimization, which can effectively optimize the target's attitude information, improve the accuracy of the shape points in space, and finally obtain more accurate target attitude information, thereby ensuring that the final position of the shape points is as close as possible to the target's true spatial attitude.

[0122] Step 375: Based on the fine positions corresponding to each shape point, obtain the target pose information.

[0123] By processing the precise positions of shape points, these finely adjusted positions can provide accurate target spatial pose, ultimately generating accurate target pose information.

[0124] In one possible implementation, the precise locations of all shape points are collected, and these shape points are connected according to the spatial structure and connection method of the target to generate the overall spatial form of the target. Based on the distribution and connection of the shape points, complete target attitude information is formed, thereby ensuring that the connection between the shape points conforms to the actual geometric characteristics of the target and forms a coherent path, so that the target attitude information can accurately reflect the target's position, angle and shape in space.

[0125] By combining the above embodiments with particle swarm optimization and local gradient optimization, the precise spatial attitude of the target can be gradually approximated through global and local optimization steps, thereby ensuring that the generated target attitude information is more accurate and providing high-precision target attitude information for practical applications.

[0126] Figure 7 This is a schematic diagram illustrating the specific implementation of an attitude reconstruction method in an application scenario. The application scenario is a minimally invasive surgery scenario, which enables high-precision positioning of the distal end of minimally invasive surgical instruments and accurate reconstruction of complex spatial attitudes.

[0127] Specifically, such as Figure 8 As shown, this method uses an optical frequency domain reflectance (OFDR) module to read the reflectance spectrum data of multiple fiber clusters / multi-core fibers embedded in minimally invasive surgical instruments to obtain target information. For a detailed description of the OFDR system structure, please refer to [link to relevant documentation]. Figure 1 It includes a shape sensing module and an information processing module.

[0128] The target information is then input into a high-precision fiber optic end positioning neural network model trained by a convolutional neural network (CNN). This model can accurately predict the coordinates of the fiber optic end and use them as the starting point for subsequent reverse attitude reconstruction.

[0129] Figure 8 A schematic diagram illustrating a specific implementation of global and local optimization of a target is shown, such as... Figure 8 As shown, through Figure 8 The Frenet framework / Bishop framework / spiral extension method shown reconstructs the coarse spatial attitude of the optical fiber starting from the RS point (i.e., the coordinates of the end of the optical fiber). Then, using the known initial position, and combining the particle swarm optimization (PSO) optimization algorithm and the Adam optimization algorithm, the spatial attitude of the optical fiber is further optimized, and finally the shape curve of the spatial attitude of the optical fiber (i.e., the target attitude information) is generated.

[0130] Specifically, starting from the RS point at the end of the fiber predicted by CNN, the shape is gradually extended along a preset path. Multiple path planning methods, including the Frenet framework, Bishop framework, and spiral extension method, are employed, with the appropriate path flexibly selected based on spatial attitude reconstruction and instrument characteristics. The generation of each shape point is based on the position of the previous point and is constrained by the motion laws within the current path framework, thus gradually extending to the known initial coordinates. These shape points constitute a coarse geometric framework (i.e., initial attitude information), providing the necessary structural foundation for subsequent global and local fine-tuning optimization. In other words, the coarse geometric framework not only provides the initial shape for subsequent particle swarm optimization based on global search but also enables precise reconstruction of the fiber's spatial attitude through fine-tuning within known coordinates, ensuring accurate fiber positioning and shape control.

[0131] Regarding Particle Swarm Optimization (PSO), shape points are initialized as "particles" based on the coarse geometric framework generated by inverse shape reconstruction. Each particle represents the position of a shape point, and these particles move freely in multidimensional space to find the shape that best meets the constraints. PSO's global search mechanism enables it to optimize the positions of all shape points within a larger space. By moving the particle swarm in a broader space, it avoids getting trapped in local optima. During the search process, PSO iteratively updates based on the velocity and position of the particle swarm to find the optimal shape that meets the constraints of the starting point, ending point, and coarse geometric framework—that is, intermediate pose information.

[0132] Based on the intermediate pose information provided by PSO, the Adam optimization algorithm further refines the positions of each shape point. Through local gradient adjustment, Adam ensures that the shape reconstruction result is closer to the expectation in detail, optimizes the shape curvature and reduces the deviation from the path or endpoint. It can further improve local accuracy while ensuring that the spatial pose of the target conforms to the overall structure, thereby obtaining the target pose information.

[0133] In this application scenario, the end position of the optical fiber is directly predicted using a CNN neural network model, and reverse attitude reconstruction is completed using a combination of the Frenet framework / Bishop framework / spiral extension method and PSO and Adam optimization algorithms. This method significantly improves the accuracy of end-effector positioning and ensures high accuracy of the attitude reconstruction results. Taking optical fiber embedded surgical instruments as an example, the end position often directly corresponds to the surgical operation point (such as the cutting or probing area), and its accuracy directly affects the functional performance of the surgical instrument. End-effector position-based reconstruction better meets practical needs and ensures accurate attitude in critical areas. Therefore, this method is particularly suitable for high-precision optical fiber spatial attitude reconstruction and precise reconstruction of complex instruments in minimally invasive surgery.

[0134] Furthermore, the high-precision fiber end positioning and spatial attitude reconstruction method based on neural networks and OFDR overcomes the impact of the cumulative integral error on the end positioning accuracy in existing model-based shape reconstruction methods, resulting in higher fiber end position positioning accuracy. In addition, it overcomes the limitations of grating position and number in the FBG demodulation process, and significantly improves the continuity and accuracy of spatial attitude.

[0135] It should be noted that the above application scenarios can be used not only for the positioning of surgical instruments at the end of the instrument and spatial attitude reconstruction during minimally invasive surgical palpation, but also for attitude monitoring of equipment such as surgical robot arms, aircraft, wind power equipment, submarine cables, and oil transportation pipelines.

[0136] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0137] The following are embodiments of the apparatus described in this application, which can be used to execute the attitude reconstruction method involved in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of the attitude reconstruction method involved in this application.

[0138] Please see Figure 9 This application provides an attitude reconstruction device 900, including but not limited to: an acquisition module 910, a positioning module 930, a reconstruction module 950, and an optimization module 970.

[0139] The acquisition module 910 is used to acquire target information obtained by the OFDR system through shape sensing of the target; the target information is used to reflect the light signal information of the target at different positions.

[0140] The positioning module 930 is used to locate the target based on the target information and obtain the end position.

[0141] The reconstruction module 950 is used to perform reverse attitude reconstruction of the target using the end position as the starting point to obtain initial attitude information; the initial attitude information includes multiple shape points of the target.

[0142] The optimization module 970 is used to optimize the shape of the target based on each shape point in the initial attitude information to obtain the optimized target attitude information; the target attitude information is used to describe the spatial attitude of the target.

[0143] It should be noted that the attitude reconstruction device provided in the above embodiments is only illustrated by the division of the above functional modules when performing attitude reconstruction. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the attitude reconstruction device will be divided into different functional modules to complete all or part of the functions described above.

[0144] Furthermore, the attitude reconstruction device and attitude reconstruction method embodiments provided in the above embodiments belong to the same concept, and the specific way in which each module performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0145] Compared with related technologies, this solution uses the end position for reverse attitude reconstruction. By predicting and reversing the end position, error accumulation can be effectively reduced. Combined with shape optimization technology, the optimized target attitude information is more in line with the actual spatial attitude of the target, which can better meet the accuracy requirements in complex scenarios and greatly improve the accuracy of attitude reconstruction.

[0146] Furthermore, by using this reverse attitude reconstruction method, the position of the shape point is gradually derived from the end position to the initial position, which not only ensures the continuity of the target's spatial attitude, but also precisely controls the target's spatial attitude. This ensures that the target's initial attitude information accurately reflects its actual spatial attitude, avoiding the error accumulation problem that may occur due to directly generating shape points from the initial position in the forward direction.

[0147] Furthermore, by combining particle swarm optimization and local gradient optimization, the precise spatial attitude of the target can be gradually approximated through global and local optimization steps, thereby ensuring that the generated target attitude information is more accurate and providing high-precision target attitude information for practical applications.

[0148] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A posture reconstruction method, characterized in that, Applied to a server-side application, wherein the server interacts with an optical frequency domain reflectance (OFDR) system, the method includes: The target information is acquired by the OFDR system through shape sensing of the target; the target information is used to reflect the optical signal information of the target at different positions. Based on the target information, the target is located to obtain the end position; The target's attitude is reconstructed by using the end position as the starting point to obtain initial attitude information; the initial attitude information includes multiple shape points of the target. The target is shaped based on each shape point in the initial attitude information to obtain optimized target attitude information; the target attitude information is used to describe the spatial attitude of the target.

2. The method as described in claim 1, characterized in that, The step of using the end position as the starting point to perform reverse attitude reconstruction on the target to obtain initial attitude information includes: A suitable path is selected for the target using the path planning method, resulting in a reference path. Using the end position as the starting point and the known initial position of the target as the ending point, the target is extended in shape from the starting point to the ending point according to the reference path to obtain the initial attitude information.

3. The method as described in claim 2, characterized in that, Taking the end position as the starting point and the known initial position of the target as the ending point, the target is extended in shape from the starting point to the ending point according to the reference path to obtain the initial attitude information, including: Based on the reference path, generate the position of the previous shape point for the end position; If the position of the previous shape point does not conform to the initial position, then the position of the previous shape point is taken as the current position, and the position of the previous shape point is generated for the current position according to the reference path, until the position of the previous shape point conforms to the initial position; The initial posture information is obtained based on the shape points generated between the end position and the initial position.

4. The method as described in claim 1, characterized in that, The step of optimizing the target's shape based on each shape point in the initial pose information to obtain optimized target pose information includes: Global optimization is performed on each shape point in the initial pose information to obtain intermediate pose information; the intermediate pose information includes the approximate position corresponding to each shape point. Local optimization is performed on the coarse positions corresponding to each shape point to obtain the fine positions corresponding to each shape point; The target pose information is obtained based on the fine position corresponding to each of the shape points.

5. The method as described in claim 4, characterized in that, The step of globally optimizing each shape point in the initial pose information to obtain intermediate pose information includes: Each shape point in the initial posture information is taken as a particle, and each particle is initialized; Under path constraints, each particle moves in a defined multidimensional space; During the movement, the velocity and / or position of each particle are iteratively updated until the iteration condition is met; The intermediate attitude information is obtained based on each of the particles.

6. The method as described in claim 4, characterized in that, The step of locally optimizing the coarse position corresponding to each shape point to obtain the fine position corresponding to each shape point includes: Local gradient adjustment is performed on the approximate positions corresponding to each of the aforementioned shape points; By adjusting the local gradient, the error between each shape point and the reference shape is minimized, and the fine position of each shape point is obtained; the reference shape is obtained based on the coarse position of each shape point according to a set rule.

7. The method according to any one of claims 1 to 6, characterized in that, The target is located using a coordinate positioning model, which includes an input layer, an encoding layer, a flattening layer, a fully connected layer, and an output layer. The step of locating the target based on the target information to obtain the end position includes: The target information is input into the coordinate positioning model using the input layer, and the features of the target information are extracted through the encoding layer to obtain the target features; The target features are transformed into a one-dimensional vector through the flattening layer, and the one-dimensional vector is mapped through the fully connected layer to obtain the end position of the target. The end position of the target is output through the output layer.

8. An attitude reconstruction device, characterized in that, The device, used in an OFDR system, includes: A shape sensing module is used to sense the shape of a target and obtain optical signal information of the target at different positions. An information processing module is used to generate target information from the optical signal information and send the target information to the server, so that the server performs attitude reconstruction on the target according to any one of claims 1 to 7.

9. The apparatus as claimed in claim 8, characterized in that, The shape sensing module includes a laser emitting unit and a main interferometer module, and the information processing module includes a data acquisition unit; wherein... The laser emitting unit is used to output laser signals; The main interferometer unit is used to divide the laser signal into a probe light signal and a reference light signal; control the probe light signal to enter the target to receive light signal information reflected back from different positions of the target; and perform optical interference with the reference light signal based on each light signal information to generate interference fringe signals. The acquisition unit is used to acquire the interference fringe signal and perform signal conversion based on the interference fringe signal to generate target information.

10. An attitude reconstruction system, characterized in that, The system includes an interactive OFDR system and a server, wherein, The OFDR system is used to perform shape sensing on the target, obtain optical signal information of the target at different positions, and generate target information from the optical signal information. The server is used to acquire the target information; locate the target based on the target information to obtain the end position; use the end position as the starting point to reconstruct the target's attitude in reverse to obtain initial attitude information; the initial attitude information includes multiple shape points of the target; optimize the target's shape based on each shape point in the initial attitude information to obtain optimized target attitude information; the target attitude information is used to describe the target's spatial attitude.

Images