5G remote magnetically controlled capsule endoscope detection control system

Abstract

The application discloses a 5G remote magnetic control capsule endoscope detection control system and relates to the technical field of remote detection control. A collection and decoupling module acquires network quality characteristic data, capsule pose time series data, stomach cavity visual characteristic data and remote expected control data; a space drift prediction module calculates network time delay and space window period according to network quality characteristics, extrapolates and predicts capsule space position changes, and generates space drift prediction data; a feedforward compensation module generates a reverse magnetic field gradient compensation amount as a feedforward compensation parameter according to a predicted drift vector; a servo feedback module generates a feedback error correction parameter by using pixel offset and image blurring in visual characteristics; an instruction fusion module superimposes a target driving instruction; and a safety arbitration module outputs a driving instruction or switches a self-locking protection signal according to whether network quality is in a safety range. The application enables the remote magnetic control capsule endoscope to still maintain accurate and safe gastrointestinal examination operation under non-ideal network conditions.

Description

Technical Field

[0001] This invention relates to the field of remote detection and control technology, specifically a 5G remote magnetically controlled capsule endoscope detection and control system. Background Technology

[0002] Remote magnetically controlled capsule endoscopy is an important development direction in the field of gastrointestinal disease screening. It aims to extend the diagnostic capabilities of specialists to grassroots or remote areas via 5G networks, enabling precise control and lesion observation even under physically isolated conditions. In recent years, with advancements in high-bandwidth mobile communication and high-precision magnetic drive technology, remote endoscopy control solutions have gradually evolved towards dynamic and real-time operation. Current technologies primarily use wide-area networks to transmit the doctor's joystick commands in real-time to the magnetically controlled robot at the patient's end, and utilize high-speed video streams to transmit gastric images back to the doctor's control terminal, achieving a remote closed loop of "operation-execution-feedback." Under ideal working conditions with excellent network coverage, these methods can effectively complete routine gastric movement and observation, demonstrating good remote operability.

[0003] However, in real-world cross-regional medical scenarios, the unavoidable physical characteristics of 5G wide area networks, such as base station switching, channel congestion, or sudden network jitter, cause the remote control link to be in a state of dynamic fluctuation. This network instability leads to millisecond-level delays and irregular jitter in the time it takes for control commands to reach the patient. Physically, this manifests as the disruption of the magnetodynamic causal chain of the capsule endoscope in the unstructured environment of the stomach cavity. At this time, the inherent natural peristalsis and gastric fluid flow within the stomach cavity cause the capsule, which has lost the constraint of immediate commands, to experience inertial drift in spatial orientation and directional lag, resulting in uncontrollable fluctuations in the original physical equilibrium state.

[0004] When dealing with such unstable network environments, existing technologies often rely on pure "network pass-through" or simple delay and waiting mechanisms. They fail to provide real-time local compensation for the spatiotemporal coupling characteristics between control delay fluctuations and the dynamic behavior within the capsule. This can easily cause severe shaking of the capsule endoscope during movement, loss of lesion field of view, and even the risk of scratching the digestive tract mucosa due to inertial drift during network disconnection vacuum periods. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a 5G remote magnetically controlled capsule endoscope detection and control system.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] This invention discloses a 5G remote magnetically controlled capsule endoscope detection and control system, comprising:

[0008] The acquisition and decoupling module is used to periodically acquire network quality characteristic data of the wide area network transmission link, capsule pose time sequence data and gastric visual characteristic data of the capsule endoscope, as well as remote desired control data output by the remote control terminal.

[0009] The spatial drift prediction module is used to calculate the extrapolated time-domain reference interval of the current network latency window based on the network quality characteristic data; and to extrapolate and predict the spatial position change of the capsule endoscope within the extrapolated time-domain reference interval based on the historical change trend of the capsule pose time series data, thereby generating spatial drift prediction data.

[0010] The feedforward compensation module is used to calculate the predicted spatial drift vector based on the spatial drift prediction data, and generate a magnetic field gradient compensation amount to offset the time delay interference based on the direction and magnitude of the predicted spatial drift vector, as a feedforward magnetic field compensation parameter.

[0011] The servo feedback module is used to extract the target center pixel offset and image edge blur from the gastric cavity visual feature data, and to perform reverse compensation calculation based on the motion deviation of the capsule endoscope relative to the target lesion to generate feedback error correction parameters.

[0012] The instruction fusion module is used to superimpose and correct the remote desired control data, the feedforward magnetic field compensation parameters and the feedback error correction parameters to generate target driving instructions;

[0013] The security arbitration module is used to determine whether the network quality characteristic data is within a preset communication security range: if it is within the communication security range, the target driving instruction is output; if it exceeds the communication security range, the target driving instruction is blocked, and a preset local steady-state control parameter set is called to generate a self-locking protection control signal and output it.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0015] 1. This invention can predict the potential positional drift of the capsule during periods without new instructions. Before remote instructions arrive due to network latency, the local edge control terminal proactively applies a compensating magnetic field to counteract latency interference. This feedforward prediction and compensation mechanism ensures that the capsule's actual trajectory closely matches the remote operator's desired trajectory, effectively suppressing control lag and position overshoot caused by network latency.

[0016] 2. This invention enables the system to accurately correct the capsule's posture based on pixel offset when the target deviates from the center of the field of view due to gastric wall peristalsis or relative sliding between the capsule and the mucosa. Simultaneously, image blur is introduced as an attenuation factor to avoid drastic miscorrections caused by image quality degradation. This allows the system to maintain stable visual tracking performance even in complex internal environments, improving the completeness of the examination and the reliability of the diagnosis.

[0017] 3. This invention avoids the risk of the capsule impacting the stomach wall or accidentally entering a dangerous area due to uncontrolled remote commands during severe network congestion or interruption. Simultaneously, the self-locking protection signal utilizes locally preset steady-state control parameters, independent of remote communication, thus ensuring patient safety even under extreme network conditions. Attached Figure Description

[0018] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0019] Figure 1 This is a system module connection diagram of the present invention;

[0020] Figure 2 This is a timing flowchart of the present invention. Detailed Implementation

[0021] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0022] In existing technologies, remote magnetically controlled capsule endoscopy relies heavily on stable high-speed networks for real-time command transmission. When 5G networks experience latency fluctuations or brief congestion, the desired control data sent from the remote control terminal cannot be synchronously applied to the capsule inside the body, causing capsule movement to lag behind the intended operation. Traditional methods passively adapt to the delay by reducing operating speed or adding redundant commands, which is insufficient to maintain precise control in time-varying network environments. The lag effect is further amplified, especially during gastric peristalsis or changes in patient position, easily leading to missed lesions or capsule collision with the stomach wall. Existing systems lack proactive prediction and compensation mechanisms for capsule drift during network latency windows and do not incorporate security protection logic based on dynamic network quality arbitration, thus limiting the reliability of remote magnetically controlled capsule endoscopy in wide area network environments.

[0023] To address the aforementioned issues, this invention introduces parallel acquisition and decoupling of multi-source data at the local edge control terminal. It calculates the length of the current network latency window by real-time analysis of network quality characteristic data and extrapolates and predicts spatial position changes within this window using historical capsule pose data and motion vector fields from gastric visual features. Furthermore, the predicted spatial drift vector is converted into a magnetic field gradient compensation quantity with opposite direction and proportional amplitude, which is then superimposed on the desired control command as a feedforward compensation parameter. This proactively offsets motion deviations caused by latency before the remote command arrives. Simultaneously, the visual servo feedback module extracts the target center pixel offset and image edge blurring, generating error correction parameters that are fused with the desired command to form a position closed loop. The safety arbitration module dynamically determines whether to output a drive command or switch a self-locking protection signal based on the network quality threshold. This achieves a complete remote control link from perception, prediction, compensation to safety redundancy, effectively improving the operational accuracy and safety of the 5G remote magnetically controlled capsule endoscope under complex network conditions.

[0024] After introducing the basic concept of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] Example:

[0026] like Figure 1 As shown, a 5G remote magnetically controlled capsule endoscope detection and control system includes:

[0027] The acquisition and decoupling module is used to periodically acquire network quality characteristic data of the wide area network transmission link, capsule pose time sequence data and gastric visual characteristic data of the capsule endoscope, as well as remote desired control data output by the remote control terminal.

[0028] The spatial drift prediction module is used to calculate the extrapolated time-domain reference interval of the current network latency window based on network quality characteristic data; based on the historical change trend of capsule pose time series data, it extrapolates and predicts the spatial position change of the capsule endoscope in the body within the extrapolated time-domain reference interval, and generates spatial drift prediction data.

[0029] The feedforward compensation module is used to calculate the predicted spatial drift vector based on the spatial drift prediction data, and generate a magnetic field gradient compensation amount to offset the time delay interference based on the direction and magnitude of the predicted spatial drift vector, which serves as the feedforward magnetic field compensation parameter.

[0030] The servo feedback module is used to extract the target center pixel offset and image edge blur from the visual feature data of the gastric cavity, and to perform reverse compensation calculation based on the motion deviation of the capsule endoscope relative to the target lesion to generate feedback error correction parameters.

[0031] The instruction fusion module is used to superimpose and correct the remote desired control data, feedforward magnetic field compensation parameters and feedback error correction parameters to generate target drive instructions;

[0032] The security arbitration module is used to determine whether the network quality characteristic data is within the preset communication security range: if it is within the communication security range, the target driving command is output; if it exceeds the communication security range, the target driving command is blocked, and the preset local steady-state control parameter set is called to generate a self-locking protection control signal and output it.

[0033] like Figure 2 The diagram shown is a timing flowchart of this application. The physical implementation of this application is deployed on a local edge control terminal at the patient's end, including a 5G wireless communication module, a central processing unit (CPU), a graphics processing unit (GPU), an external magnetic control actuator array, and a storage device. The external magnetic control actuator array consists of multiple independently controllable electromagnetic coils, used to synthesize a spatial magnetic field outside the patient to drive the movement of the capsule endoscope inside the body. The storage device contains computer program instructions that execute the logic of each module, achieving the following workflow through the use of hardware resources:

[0034] The acquisition and decoupling module acquires four heterogeneous data sources in parallel at a fixed control cycle (e.g., 5ms to 20ms): network quality characteristic data is read through the link status interface of the 5G communication module; capsule pose timing data is read through the capsule's built-in inertial measurement unit and magnetic positioning coil; gastric visual feature data is read through the image feedback channel; and remote desired control data is read through the command channel of the remote control terminal. The system decouples the four data sources according to their origins and sends them to their respective processing branches to avoid cross-interference of network jitter on the timing of pose and visual data processing.

[0035] The spatial drift prediction module first analyzes the network quality characteristic data, estimates the network latency window during which no new instructions arrive, and delineates the extrapolation time domain reference interval accordingly. Then, it retrieves the historical trend of the capsule pose time series data, combines it with the environmental disturbances inverted from the gastric cavity visual characteristic data, and extrapolates the spatial position and attitude of the capsule during the period of losing immediate instruction constraints within the extrapolation time domain reference interval, outputting spatial drift prediction data.

[0036] The feedforward compensation module calculates the predicted spatial drift vector based on the spatial drift prediction data and solves for the magnetic field gradient compensation amount. This allows the external magnetically controlled actuator array to apply reverse traction to the predicted drift before the remote command arrives, serving as the feedforward magnetic field compensation parameter. The servo feedback module processes the transmitted gastric cavity visual feature data, extracts the pixel offset of the target lesion center relative to the image field center and the image edge blurring, and calculates the feedback error correction parameter to correct the cumulative deviation of the feedforward prediction.

[0037] The instruction fusion module synthesizes the target driving instruction by combining the remote desired control data, feedforward magnetic field compensation parameters, and feedback error correction parameters in a vector superposition manner. The safety arbitration module performs a safety judgment on the network quality characteristic data. When the communication is within the safe range, the target driving instruction is output through the external magnetic control actuator array. When the communication exceeds the safe range, the instruction is blocked and the signal is switched to a self-locking protection control signal based on the local steady-state control parameter set, so that the entire control link forms a closed loop between perception, prediction, compensation, fusion, and safety redundancy.

[0038] Through the aforementioned local edge control process combining feedforward prediction and feedback correction, this application can still actively apply a compensating magnetic field to counteract delay interference based on local data when remote commands are delayed due to network latency. This ensures that the actual movement trajectory of the capsule closely matches the remote operator's desired trajectory, suppressing control lag and position overshoot caused by network latency. Simultaneously, the safety arbitration mechanism enables the system to promptly detach from the remote link when the network is severely degraded, reducing the risk of the capsule scratching the digestive tract mucosa due to inertial drift during network vacuum periods.

[0039] This application further proposes specific steps for calculating the extrapolated time-domain reference interval of the current network latency window based on network quality characteristic data. The spatial drift prediction module parses the network quality characteristic data and extracts the one-way transmission latency from it. With network jitter time The sum of these two factors is determined as the current network latency window period. The calculation formula is as follows:

[0040] ;

[0041] in, This is the current network latency window period. For one-way transmission delay, Network jitter time; all three are measured in milliseconds. One-way transmission delay. Network jitter time is given by the link statistics reported by the 5G communication module. The standard deviation of the one-way transmission delay within the most recent control cycles is used to characterize the uncertainty margin of the link fluctuation.

[0042] The system further acquires the current image sampling frame rate of the capsule endoscope inside the body. (Unit: frames / second), calculate its reciprocal to obtain the single-frame sampling period. ,Right now ;when When sampling from 25 frames per second to 50 frames per second, the single frame sampling period is... The corresponding range is between 0.02s and 0.04s. The system will adjust the single-frame sampling period. The minimum time-resolved step size for extrapolation ensures that spatial extrapolation is consistent with the image return time.

[0043] In one embodiment, if the current network latency window period Greater than or equal to the single frame sampling period The system uses a single-frame sampling period. Using the step size, the current network delay window is divided into K equally spaced discrete time slices, forming an extrapolated time domain reference interval. The number of discrete time slices K is determined by rounding down the following formula:

[0044] ;

[0045] in, This indicates the floor operation; for example, when =60ms When the time interval is 20ms, K=3, and the extrapolated time domain reference interval is divided into three discrete time slices with a step size of 20ms. If the current network latency window... Less than a single frame sampling period Then the system will no longer subdivide it, and will directly set the current network latency window period. Treat it as a discrete time slice, i.e., let K=1, to avoid meaningless multi-step extrapolation within a very short window period.

[0046] By quantizing the transmission delay and jitter of the network link into an extrapolable discrete time domain interval, this application ensures that the step size of the subsequent spatial drift extrapolation strictly corresponds to the actual length of the network delay window. This avoids the problems of insufficient extrapolation under long delay and excessive extrapolation under short delay caused by a fixed step size, and also synchronizes the extrapolation beat with the image sampling beat, providing a time reference that is adaptively matched with the network state for feedforward compensation.

[0047] This application further proposes specific steps for generating spatial drift prediction data within an extrapolated time-domain reference interval. The spatial drift prediction module extracts a preset number (e.g., the most recent 5 to 10 control cycles) of historical 3D coordinate points and attitude quaternions from the capsule pose timing data, and constructs local motion differential equations with position p and velocity v as state variables; when no new remote commands arrive during the network delay window, the driving acceleration of the previous effective control cycle is kept as a known input term. And treat the internal environment disturbance as an external physical constraint. By injecting this local equation of motion, the acceleration composition relationship is obtained:

[0048] ;

[0049] Where 'a' is the composite acceleration vector of the capsule within the body. Given the known driving acceleration input, The system analyzes visual feature data of the gastric cavity to extract the motion vector field of the gastric wall mucosa, calculates the average optical flow velocity of the motion vector field within the field of view, and generates the environmental disturbance acceleration variable by mapping it according to the scaling factor λ. ,Right now ,in The average optical flow velocity vector per unit time. This is a pre-calibrated mapping coefficient from visual motion to physical acceleration, used to convert dynamic mechanical disturbances such as gastric peristalsis and gastric juice flow into equivalent accelerations acting on the capsule. The calibration method involves applying a known optical flow perturbation to an isolated gastric model. The equivalent acceleration generated by the capsule was measured using a magnetic positioning system. The slope is obtained by performing linear fitting. The typical value range is 0.5~3.0s. -1 Preferred =1.2s -1 .

[0050] Within the extrapolated time-domain reference interval, the system uses a single-frame sampling period. Using time-domain integration of the local motion differential equations with a step size as the recursive step, the predicted spatial position and predicted spatial attitude are output time-slice by time slice. The recursive relationship between position and velocity is as follows:

[0051] ;

[0052] in, , Let be the predicted position and predicted velocity for the nth discrete time slice, respectively. , This is the predicted value for the next discrete time slice;

[0053] The attitude quaternion is updated according to the increment of angular velocity ω within a single frame sampling period, i.e. ,in To represent quaternion multiplication, This is a unit rotation quaternion constructed from the angular velocity increment. The system sequentially concatenates the predicted spatial position and predicted spatial attitude output from each discrete time slice to form spatial drift prediction data covering the entire network delay window.

[0054] By converting the visual motion vector field of the gastric cavity into environmental disturbance acceleration and injecting it into the local motion differential equation for time-slice recursion, this application can still perform physically consistent extrapolation of the capsule's pose evolution based on its own motion inertia and internal dynamic disturbances during the network vacuum period when there is a lack of immediate command constraints. This allows the spatial drift prediction data to truly reflect the capsule's drift trend during this period, providing a physically based prediction input for feedforward compensation.

[0055] This application further proposes that, before generating the feedforward magnetic field compensation parameters, the feedforward compensation module also performs a real-time calculation step of the effective force-bearing area and magnetic moment orientation of the capsule within the body. The system extracts the predicted spatial attitude from the spatial drift prediction data and retrieves the pre-stored rigid body geometric 3D model of the capsule, which is represented by a mesh or parametric surface of the capsule's rotational shape. The system calculates the unit vector of the current motion direction based on the displacement directions of adjacent coordinate points in the capsule's pose time series data. ,Right now ,in, This represents the position increment between adjacent control cycles.

[0056] The system rotates the rigid body geometric 3D model of the capsule according to the predicted spatial attitude, and then directs the rotated geometric 3D model towards a unit vector perpendicular to the current direction of motion. By orthogonally projecting the plane onto the surface and integrating the area enclosed by the projected contour, the predicted fluid collision area of ​​the capsule at the leading edge of motion within the body can be calculated. Predicting fluid collision area It is used to characterize the size of the frontal cross section of the capsule in its current attitude. The larger the frontal cross section, the greater the drag it experiences under the same fluid disturbance.

[0057] The system further calculates the current magnetic moment vector direction m of the permanent magnet inside the capsule based on the predicted spatial attitude, i.e. ,in Let q be the rotation matrix determined by the predicted attitude quaternion. This represents the direction of the inherent magnetic moment of the permanent magnet in the capsule's body coordinate system. The system will predict the fluid collision area. The current magnetic moment vector direction m is used as an additional feature input feedforward compensation module, which is then used to calculate the direction and magnitude of the subsequent magnetic field gradient compensation.

[0058] By acquiring the upflow cross-sectional area and magnetic moment orientation in real time before compensation calculation, this application enables the feedforward compensation to no longer treat the capsule as an isotropic point, but instead takes into account its attitude-related upflow characteristics and magnet orientation. This ensures that the subsequent generated magnetic field gradient compensation is matched with the actual stress conditions of the capsule in terms of direction and amplitude, thereby improving the accuracy of feedforward compensation in offsetting drift.

[0059] This application further proposes a specific calculation method for generating feedforward magnetic field compensation parameters. The feedforward compensation module calculates the predicted spatial position in the spatial drift prediction data. Current real-space position in capsule pose time series data The difference is used to obtain the predicted spatial drift vector d, i.e. This vector represents the predicted offset of the capsule relative to its current position during the network delay window without compensation.

[0060] The system converts remote desired control data into desired magnetic field vectors. And according to the desired magnetic field vector Calculate the desired force direction unit vector with the current magnetic moment vector direction m. ( (Characterizing the traction direction of the capsule's expected motion by the remote operator); then, the predicted spatial drift vector d is spatially projected in the opposite direction of the expected force to obtain the projected scalar value. The calculation formula is as follows:

[0061] ;

[0062] in, Here, is the projected scalar value, and d is the predicted spatial drift vector. The vector represents the desired direction of force, with "·" indicating the vector dot product. Projected scalar value. This characterizes the magnitude of the component in the predicted drift that deviates from the desired traction direction and needs to be preferentially canceled; the more severe the deviation of the predicted drift from the desired traction direction, the greater the impact. The larger.

[0063] The system is based on the projected scalar value Collision area with predicted fluid Calculate the magnetic field gradient compensation amount using the following formula The magnetic field gradient compensation amount is then used as the output feedforward magnetic field compensation parameter.

[0064] ;

[0065] in, This is the magnetic field gradient compensation amount. The preset reference frontal cross-sectional area (e.g., 50mm) 2 );

[0066] k is a preset proportional coefficient, which is determined by the magnetic field gradient-force transmission characteristics calibration of the magnetic actuator; in one embodiment, the value of k ranges from 0.5 to 2.0 (unit is (T / m) / mm), and the typical value is 1.0.

[0067] By projecting the predicted drift in the opposite direction of the desired traction and proportionally converting it into a magnetic field gradient compensation amount with the opposite direction and proportional amplitude, this application enables the feedforward compensation to apply the reverse magnetic field traction only to the harmful drift component that deviates from the desired trajectory, without interfering with the normal motion component along the desired direction. This allows the position deviation caused by the time delay to be offset in advance during the network time delay window, reducing control lag and overshoot.

[0068] This application further proposes specific steps for generating feedback error correction parameters. The servo feedback module obtains the preset image field-of-view center coordinates. Feature point matching is performed on the visual feature data of the gastric cavity to calculate the current two-dimensional pixel coordinates (u,v) of the geometric center of the target lesion in the current image coordinate system; then, the algebraic difference between the current two-dimensional pixel coordinates and the coordinates of the image field of view center is calculated to generate the target center pixel offset. ,Right now This offset represents the direction and magnitude of the target lesion's deviation from the center of the field of view within the image plane.

[0069] The system utilizes the inverse matrix of the preset visual Jacobian matrix J. Offset the target center pixel Transformed into a visual tracking position deviation vector in physical three-dimensional space ,Right now The visual Jacobian matrix J describes the linear mapping relationship between the capsule's minute displacement in physical three-dimensional space and the target's pixel displacement in the image plane, and is obtained by calibrating the current working distance within the camera.

[0070] The system extracts image edge blur from gastric cavity visual feature data by: calculating the gradient magnitude sequence of image edge pixels and then calculating the mean square error of this gradient magnitude sequence. As a statistic of image sharpness (the sharper the image, the better) The larger the value, the more normalized and inverted it into a quantized value b of ambiguity, i.e. ,in The standard deviation of the predefined clear image reference is, and its value is... Dimensionality consistent (edge ​​gradient magnitude). The calibration method is as follows: During the calibration phase, a standard clear reference image is selected, and the mean square error of its edge gradient magnitude is calculated as... In this embodiment, for an 8-bit grayscale image, take... =45; This means that the value is limited to the interval [0,1]; therefore, the blurrier the image, the closer b is to 1.

[0071] The system tracks the position deviation vector based on visual tracking. The feedback error correction parameter is calculated using the quantization value 'b' of the ambiguity, as follows:

[0072] ;

[0073] in, Feedback error correction parameters (vector, unit AND) Similarly, in this embodiment, T / m is used. This is the position deviation vector for visual tracking (unit: mm). This is the quantization value of the ambiguity (dimensionless, 0~1). This is a preset proportionality coefficient; The preset ambiguity attenuation coefficient (dimensionless, satisfying...) The system calculates the coefficients. The lower limit is truncated to 0 to ensure that the correction direction is not opposite to the expected deviation direction.

[0074] When the image is clear (b approaches 0), the correction amount approaches... When the image is blurred (b approaches 1), the correction amount decays to... This automatically reduces the correction intensity when image quality deteriorates, suppressing drastic erroneous corrections. In this embodiment, we take... ,in The value range is 0.1~1.0 (T / m) / mm. The value range is [0, 0.6].

[0075] By mapping pixel offset to physical position deviation through the visual Jacobian inverse matrix and introducing an attenuation factor related to ambiguity, this application can accurately correct the capsule pose based on pixel offset when the target deviates from the center of the field of view due to gastric wall peristalsis or relative sliding between the capsule and the mucosa. At the same time, it automatically reduces the correction intensity when the image quality deteriorates, so that the system can maintain stable visual tracking performance in complex in vivo environments, thereby improving the integrity of the examination and the reliability of diagnosis.

[0076] This application further proposes that, after generating the target drive command, the command fusion module also performs a magnetic saturation constraint and actuator physical redline safety verification step. The system converts the target drive command into target excitation current parameters for each electromagnetic coil in the external magnetically controlled actuator array. (i is the coil number), and retrieve the maximum physical current threshold pre-stored in the external actuator. Maximum physical current threshold The upper limit of current is determined by the coil's rated current and heat dissipation capacity, and is used to define the upper limit of current that will prevent the hardware from overheating and being damaged.

[0077] The system compares the excitation current parameters of each target one by one. With the maximum physical current threshold If all target excitation current parameters do not exceed the maximum physical current threshold, the target drive command is output directly; if any target excitation current parameter exceeds the maximum physical current threshold, the system calculates the global attenuation coefficient η based on the over-limit ratio, using the following formula:

[0078] ;

[0079] Where η is the global attenuation coefficient. The maximum physical current threshold, The maximum value among all target excitation current parameters of the coils is η; since this maximum value exceeds the threshold, η is between 0 and 1. The system uses a global attenuation coefficient η to proportionally reduce the magnetic field strength scalar in the target drive command, that is, multiply the magnetic field strength scalar by η, and correspondingly, the excitation current of each coil is proportionally reduced to η· This ensures that the current in all coils does not exceed the maximum physical current threshold after reduction, while maintaining the current ratio of each coil to keep the direction of the synthesized magnetic field unchanged. The system outputs the reduced safety target drive command to the external magnetically controlled actuator instead of the original target drive command.

[0080] By applying a global proportional attenuation to the target drive command, this application can compress the current of each coil to within the physical red line without changing the direction of the synthesized magnetic field when the required magnetic field exceeds the hardware capability due to the superposition of feedforward compensation or feedback correction. This avoids coil overcurrent damage and maintains the correctness of the drive direction, thus achieving safe coordination between strong compensation requirements and actuator physical constraints.

[0081] This application further proposes that, before the security arbitration module performs its judgment, it also includes a multi-route timestamp alignment and communication packet loss rate sliding window statistics step. The system uniformly timestamps network quality characteristic data, capsule pose timing data, gastric visual characteristic data, and remote desired control data onto a local clock reference, and pushes them into the local edge buffer queue in chronological order of the timestamps; the system sets a time retention threshold. (For example, from 300ms to 800ms), historical expired data in the cache queue with timestamps earlier than the current time minus the time retention threshold are removed to ensure that the data involved in the judgment has temporal consistency.

[0082] The system is set to have a preset time axis length. A sliding observation window (e.g., 500ms to 2000ms) is used to count the number of data packets that fail to arrive on time in the wide area network transmission link within the sliding observation window. The expected total number of data packets is calculated based on the link nominal packet rate and the sliding observation window length. Then, the real-time data packet loss rate L is calculated using the following formula:

[0083] ;

[0084] Where L is the real-time data packet loss rate. To view the number of data packets that did not arrive on time in the sliding observation window, This represents the expected total number of data packets within the sliding observation window. The system compares the real-time data packet loss rate L with a preset packet loss rate threshold. The latency in the network quality characteristic data is compared with the preset latency threshold. Comparison; in one embodiment, packet loss rate threshold The value ranges from 3% to 8% (typical value 5%), and the latency threshold is... The value ranges from 100ms to 200ms (typical value 150ms).

[0085] If the real-time data packet loss rate L is greater than the preset packet loss rate threshold Or the latency in the network quality characteristic data is greater than the preset latency threshold. If the network quality characteristic data exceeds the preset communication security range, the system determines that the data has exceeded the security threshold and submits the determination to the security arbitration module. By aligning with timestamps to remove expired data and using a sliding window to calculate the real-time packet loss rate, this application enables the network quality assessment used for security arbitration to simultaneously consider both latency and packet loss, and by using a sliding window to reflect the instantaneous fluctuations of the link rather than single-point sampling, it improves the real-time performance and jitter resistance of communication security determination.

[0086] This application further proposes that the system also includes a control instability prediction module, used to intervene in advance when the network is still within a safe range but the compensation is about to fail. The control instability prediction module extracts the predicted spatial drift vector d generated by the feedforward compensation module in multiple consecutive control cycles, and compares the magnitude ‖d‖ of the predicted spatial drift vector in each control cycle with a preset spatial drift threshold. The comparison is performed, and the number of cycles c that continuously exceed the spatial drift threshold is counted; in one embodiment, the spatial drift threshold is... The value ranges from 1.5mm to 3.0mm.

[0087] If the number of consecutive periods c exceeds the preset instability detection threshold... (For example, taking 3 to 8 consecutive cycles), the control instability prediction module determines that the capsule endoscope inside the body is in a control instability state and outputs an instability trigger signal to the safety arbitration module. Using continuous counting rather than single limit exceedances as the judgment basis can avoid occasional instantaneous drift that falsely triggers the protection.

[0088] Upon receiving the instability trigger signal, the safety arbitration module directly blocks the target drive command and generates a self-locking protection control signal output by calling the preset local steady-state control parameter set. This causes the capsule endoscope inside the body to hover in place under the constraint of the locking magnetic field synthesized by the external magnetically controlled actuator. By predicting instability through the over-limit count of continuous multi-cycle predicted drift, this application can trigger self-locking protection in advance when the network indicators have not yet exceeded the safety range, but the feedforward compensation can no longer suppress the drift within the allowable range. This prevents the capsule from deviating from the safe position under continuous accumulated drift, further improving the safety redundancy of remote control.

[0089] This application further proposes that, before activating the preset local steady-state control parameter set to generate the self-locking protection control signal, the security arbitration module also performs a dynamic configuration step of self-locking stiffness based on gastric peristalsis energy spectrum analysis. The system extracts a preset number of consecutive frames (e.g., the most recent 16 to 32 frames) of gastric visual feature data before the network quality feature data abruptly exceeds the communication security range, as a sample sequence reflecting the current gastric motion state.

[0090] The system performs temporal spectral analysis of the extracted visual feature data of the gastric cavity: first, it calculates the dense optical flow field of the gastric wall frame by frame, and then performs discrete Fourier transform on the time series of optical flow field amplitudes to obtain its spectrum; the frequency corresponding to the maximum energy in the spectrum is taken as the dominant peristaltic frequency of the gastric wall muscles in the current pathological state. And the kinetic energy distribution spectral density S(f) is formed by the energy of each frequency component. The dominant creep frequency... The kinetic energy distribution spectral density S(f) characterizes the speed of periodic peristalsis in the gastric cavity, while the energy intensity distribution of the peristaltic disturbance is characterized by the kinetic energy distribution spectral density S(f).

[0091] The system is based on the dominant peristaltic frequency The minimum stiffness scalar of the local locking magnetic field is calculated using the kinetic energy distribution spectral density S(f). The calculation formula is as follows:

[0092] ;

[0093] in, For the minimum stiffness scalar, The peak energy of the kinetic energy distribution spectral density S(f) near the dominant creep frequency. The dominant peristaltic frequency is α, which is a preset energy-stiffness calibration coefficient. The calibration method is as follows: on a biomimetic stomach model with a known peristaltic energy spectrum, the minimum locking stiffness required to maintain the capsule in situ suspension is measured, and α is derived by reverse calculation. The preferred value is α=0.15.

[0094] This formula shows that the higher the dominant peristaltic frequency and the greater the peak peristaltic energy, the greater the minimum stiffness of the locking magnetic field required to resist periodic disturbances in the gastric cavity. The system is based on the minimum stiffness scalar... Dynamically correct the steady-state control current in the local steady-state control parameter set. ,make Follow The monotonically increasing force provides sufficient locking force to overcome current creep disturbances during hovering.

[0095] The system is based on the minimum stiffness scalar The steady-state control current in the local steady-state control parameter set is dynamically corrected according to the following formula. : ,in The reference locking current is 0.5A. A preset current-stiffness mapping coefficient (preferably 0.2A / ((T / m) / mm)) is used to provide a locking force sufficient to overcome current creep disturbances during hovering.

[0096] By performing energy spectrum analysis on gastric peristalsis before switching to self-locking protection and dynamically configuring the locking stiffness accordingly, this application ensures that the locking magnetic field applied by the self-locking protection matches the actual peristaltic disturbance intensity of the current gastric cavity. This avoids both insufficient locking force due to strong peristalsis, which could cause the capsule to be disturbed and pushed, and excessive magnetic field applied under gentle peristalsis, which could lead to unnecessary energy consumption and mucosal compression risks. This improves the adaptability and safety of self-locking protection under different gastric physiological states.

[0097] The following is a specific embodiment of a 5G remote magnetically controlled capsule endoscope detection and control system:

[0098] A tertiary hospital used the system described in this application to conduct cross-regional remote magnetically controlled capsule endoscopy examinations for patients in primary healthcare facilities located in remote areas. After the patient swallows the magnetically controlled capsule, the local edge control terminal collects four channels of heterogeneous data with a control cycle of 10ms. Remote experts at the city end send remote control data via a joystick, and images are transmitted back at a frame rate of 40 frames per second.

[0099] The acquisition and decoupling module analyzes the link status reported by the 5G communication module to obtain the one-way transmission delay. =40ms, network jitter time =20ms, calculate the current network latency window period. = + =60ms; determined by the image sampling frame rate =40 frames / second single frame sampling period =1 / =25ms. Because... =60ms greater than =25ms, the system divides the extrapolated time domain reference interval into two discrete time slices with a step size of 25ms according to K=⌊60 / 25⌋=2.

[0100] The spatial drift prediction module extracts historical 3D coordinates and attitude quaternions from the most recent 8 control cycles to construct local motion differential equations; it also analyzes the gastric cavity visual feature data to obtain the average optical flow velocity. Corresponding environmental disturbance acceleration variable With a single frame sampling period The two-step time-domain integration is performed with a step size of 25ms to output the predicted spatial position and predicted spatial attitude corresponding to the second discrete time slice, and the predicted spatial drift vector d is obtained. Its magnitude is ||d||=1.8mm and its direction is biased towards the greater curvature of the stomach cavity.

[0101] The feedforward compensation module projects the capsule's rigid body geometric 3D model onto a plane perpendicular to the current direction of motion based on the predicted spatial attitude, and calculates the predicted fluid collision area. =78mm 2 The system calculates the current magnetic moment vector direction m. It then projects the predicted spatial drift vector d in the opposite direction to the desired force direction, obtaining the projected scalar value. =1.5mm; take the proportionality coefficient k=1.0(T / m) / mm, let =80mm 2 Calculate the magnetic field gradient compensation amount 1.46 T / m was used as the feedforward magnetic field compensation parameter.

[0102] The servo feedback module performs feature point matching on the transmitted image to obtain the current two-dimensional pixel coordinates (u,v)=(1032,540) of the target lesion center and the coordinates of the image field center. =(960,540), target center pixel offset Δe=[72,0] pixels; visual tracking position deviation vector is obtained by mapping through the visual Jacobian inverse matrix. Its modulus is 0.9 mm. The system calculates the mean square error of the edge gradient magnitude. After normalization, the fuzzy quantification value is obtained as b = 0.2; take... =0.6、 =0.25, calculate =0.95, || ||=0.513T / m. Since b=0.2 is relatively small, the correction amount only decreases to 95% of the original, indicating that the current image is clear and the correction is reliable.

[0103] The instruction fusion module integrates remote desired control data, feedforward magnetic field compensation parameter ΔG, and feedback error correction parameter. Superimposed to generate target drive commands; during magnetic saturation constraint verification, the maximum value of the target excitation current of each coil. =11A, exceeding the maximum physical current threshold. =10A, system calculates global attenuation coefficient =10 / 11≈0.91, the magnetic field strength scalar of the target drive command is proportionally reduced to 0.91 times the original value, and a safe target drive command is output, so that the current of each coil drops back to within 10A.

[0104] During a base station handover, the security arbitration module calculated the real-time data packet loss rate using a sliding observation window (window length 1000ms). =7%, exceeding the preset packet loss rate threshold. =5%, and network latency rises to 170ms, exceeding the latency threshold. =150ms, indicating the network has exceeded the communication security range. Simultaneously, the control instability prediction module detects that the predicted spatial drift vector magnitude exceeds the spatial drift threshold for six consecutive cycles. =2.0mm, exceeding the instability detection counting threshold. =5, outputting an instability trigger signal. The safety arbitration module uses this signal to block the target drive command.

[0105] Before switching to self-locking protection, the system extracts 24 consecutive frames of gastric visual feature data prior to the network mutation, and obtains the dominant peristaltic frequency through optical flow field time-domain spectral analysis. =0.25Hz, peak energy Take the calibration factor α and calculate the minimum stiffness scalar. Based on this, the steady-state control current in the local steady-state control parameter set is... The signal is raised to generate a self-locking protection control signal, causing the capsule to hover in place under the constraint of the locking magnetic field. The remote control is then returned after the network is restored to a safe communication range.

[0106] This embodiment demonstrates the complete workflow of the system in a cross-regional remote gastroscopy scenario. Through the synergistic effects of network latency window extrapolation, feedforward magnetic field compensation, visual servo feedback, magnetic saturation safety constraints, communication security arbitration, control instability prediction, and dynamic configuration of self-locking stiffness, the system can maintain the capsule's trajectory close to the desired trajectory and stably track the target lesion in the field of view even under non-ideal conditions such as latency fluctuations and base station switching in the 5G wide area network. Furthermore, it can safely switch to local in-situ hovering when the network deteriorates. This provides a basis for achieving both accuracy and safety in the clinical application of remote magnetically controlled capsule endoscopy under complex network conditions.

[0107] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A 5G remote magnetically controlled capsule endoscope detection and control system, characterized in that, include: The acquisition and decoupling module is used to periodically acquire network quality characteristic data of the wide area network transmission link, capsule pose time sequence data and gastric visual characteristic data of the capsule endoscope, as well as remote desired control data output by the remote control terminal. The spatial drift prediction module is used to calculate the extrapolated time-domain reference interval of the current network delay window based on the network quality characteristic data. Based on the historical trend of the capsule pose time series data, extrapolation prediction is performed on the spatial position change of the in vivo capsule endoscope within the extrapolation time domain reference interval to generate spatial drift prediction data. The feedforward compensation module is used to calculate the predicted spatial drift vector based on the spatial drift prediction data, and generate a magnetic field gradient compensation amount to offset the time delay interference based on the direction and magnitude of the predicted spatial drift vector, as a feedforward magnetic field compensation parameter. The servo feedback module is used to extract the target center pixel offset and image edge blur from the gastric cavity visual feature data, and to perform reverse compensation calculation based on the motion deviation of the capsule endoscope relative to the target lesion to generate feedback error correction parameters. The instruction fusion module is used to superimpose and correct the remote desired control data, the feedforward magnetic field compensation parameters and the feedback error correction parameters to generate target driving instructions; The security arbitration module is used to determine whether the network quality characteristic data is within a preset communication security range; if it is within the communication security range, the target driving instruction is output. If the communication security range is exceeded, the target driving command is blocked, and a self-locking protection control signal is generated and output by calling the preset local steady-state control parameter set.

2. The 5G remote magnetically controlled capsule endoscope detection and control system according to claim 1, characterized in that, The specific steps for calculating the extrapolated time-domain reference interval of the current network latency window based on the network quality characteristic data include: The network quality feature data is analyzed to extract one-way transmission delay and network jitter time, and the sum of the one-way transmission delay and the network jitter time is determined as the current network delay window period; Obtain the current image sampling frame rate of the capsule endoscope inside the body, and calculate the reciprocal of the current image sampling frame rate to obtain the single frame sampling period; If the current network delay window period is greater than or equal to the single frame sampling period, then the current network delay window period is divided into K equally spaced discrete time slices with the single frame sampling period as the step size, forming the extrapolated time domain reference interval. If the current network latency window is less than the single-frame sampling period, then the current network latency window is directly treated as a discrete time slice.

3. The 5G remote magnetically controlled capsule endoscope detection and control system according to claim 2, characterized in that, The specific steps for generating spatial drift prediction data include: Extract a preset number of historical three-dimensional coordinate points and attitude quaternions from the capsule pose time series data, and construct local motion differential equations; The visual feature data of the gastric cavity is analyzed to extract the motion vector field of the gastric wall mucosa, and the environmental disturbance acceleration variable is generated based on the motion vector field mapping. Within the extrapolated time-domain reference interval, the environmental disturbance acceleration variable is injected as an external physical constraint into the local motion differential equation for time-domain integration and recursion, outputting the predicted spatial position and predicted spatial attitude corresponding to the next discrete time slice, thus forming the spatial drift prediction data.

4. The 5G remote magnetically controlled capsule endoscope detection and control system according to claim 1, characterized in that, Before generating the feedforward magnetic field compensation parameters, a real-time calculation step is performed to determine the effective force-bearing area and magnetic moment orientation of the capsule inside the body. This includes: Extract the predicted spatial attitude from the spatial drift prediction data and retrieve the pre-stored capsule rigid body geometric 3D model; The current direction of motion is calculated based on the capsule pose time sequence data. Based on the predicted spatial pose, the rigid body geometric three-dimensional model of the capsule is projected onto a plane perpendicular to the current direction of motion, and the predicted fluid collision area of ​​the capsule at the leading edge of motion is calculated. The current magnetic moment vector direction of the permanent magnet built into the capsule endoscope is calculated based on the predicted spatial attitude, and the predicted fluid collision area and the current magnetic moment vector direction are input into the feedforward compensation module as additional features.

5. The 5G remote magnetically controlled capsule endoscope detection and control system according to claim 4, characterized in that, The specific calculation methods for generating feedforward magnetic field compensation parameters include: The difference between the predicted spatial position in the spatial drift prediction data and the current true spatial position in the capsule pose time series data is calculated to obtain the predicted spatial drift vector; The remote desired control data is converted into a desired magnetic field vector, and the desired force direction is calculated based on the desired magnetic field vector and the current magnetic moment vector direction; the predicted spatial drift vector is spatially projected along the opposite direction of the desired force direction to obtain a projected scalar value; Based on the projected scalar value, according to the formula Calculate the magnetic field gradient compensation amount, where This is the magnetic field gradient compensation amount. This is a preset proportional coefficient. The predicted fluid collision area. This is the preset reference airflow cross-sectional area. The projection scalar value; The magnetic field gradient compensation amount is used as the output of the feedforward magnetic field compensation parameter.

6. The 5G remote magnetically controlled capsule endoscope detection and control system according to claim 1, characterized in that, The specific steps for generating feedback error correction parameters include: Obtain the preset coordinates of the image field of view center, perform feature point matching on the gastric cavity visual feature data, and calculate the current two-dimensional pixel coordinates of the geometric center of the target lesion in the current image coordinate system; Calculate the algebraic difference between the current two-dimensional pixel coordinates and the coordinates of the image field of view center to generate the target center pixel offset; Using the inverse of a preset visual Jacobian matrix, the target center pixel offset is converted into a visual tracking position deviation vector in physical three-dimensional space. The image edge blurriness in the gastric cavity visual feature data is extracted, and the mean square error of the pixel gradient magnitude at the image edge is calculated as the quantization value of the blurriness. According to the formula Calculate the feedback error correction parameters, where, For feedback error correction parameters, The visual tracking position deviation vector, This is the quantized value of the ambiguity. , These are the preset scaling factor and ambiguity attenuation factor.

7. The 5G remote magnetically controlled capsule endoscope detection and control system according to claim 1, characterized in that, After generating the target driving instructions, the process also includes magnetic saturation constraint and actuator physical red line safety verification steps, specifically including: The target drive command is converted into the target excitation current parameters of each coil in the external magnetic control actuator array, and the maximum physical current threshold of the external actuator is retrieved. If any target excitation current parameter exceeds the maximum physical current threshold, the global attenuation coefficient is calculated based on the over-limit ratio. The magnetic field strength scalar in the target drive command is proportionally reduced using the global attenuation coefficient, and a safe target drive command that meets the hardware physical constraints is output to replace the original target drive command output to the external magnetic control actuator.

8. The 5G remote magnetically controlled capsule endoscope detection and control system according to claim 1, characterized in that, Before the security arbitration module performs its judgment, it also includes a multi-route timestamp alignment and communication packet loss rate sliding window statistics step, specifically including: The network quality feature data, the capsule pose timing data, the gastric cavity visual feature data, and the remote desired control data are timestamped with a local clock reference, and pushed into the local edge cache queue in the order of the timestamps, and historical expired data that exceeds the preset time retention threshold is removed. Set a sliding observation window with a preset time axis length, count the number of data packets that do not arrive on time in the wide area network transmission link within the sliding observation window, calculate the ratio of the number of data packets to the expected total number of data packets within the sliding observation window, and generate the real-time data packet loss rate; If the real-time data packet loss rate is greater than a preset packet loss rate threshold, or if the latency in the network quality characteristic data is greater than a preset latency threshold, then the network quality characteristic data is determined to have exceeded the preset communication security range.

9. A 5G remote magnetically controlled capsule endoscope detection and control system according to claim 1, characterized in that, It also includes a control instability prediction module, used for: Extract the predicted spatial drift vector generated by the feedforward compensation module within multiple consecutive control cycles; The predicted spatial drift vector within each control cycle is compared with a preset spatial drift threshold, and the number of cycles that continuously exceed the spatial drift threshold is counted. If the number of consecutive exceeded cycles is greater than the preset instability judgment count threshold, the capsule endoscope is determined to be in a state of control instability, and an instability trigger signal is output to the safety arbitration module. After receiving the instability trigger signal, the safety arbitration module directly blocks the target driving command and calls the preset local steady-state control parameter set to generate a self-locking protection control signal output, so that the capsule endoscope in the body is suspended in place under magnetic field locking.

10. A 5G remote magnetically controlled capsule endoscope detection and control system according to claim 1, characterized in that, Before activating the preset local steady-state control parameter set to generate the self-locking protection control signal, a dynamic configuration step for the self-locking stiffness based on gastric peristalsis energy spectrum analysis is also performed, specifically including: Extract the gastric cavity visual feature data of a preset number of consecutive frames before the network quality feature data suddenly breaks through the communication security range; Optical flow field temporal spectrum analysis was performed on the visual feature data of the gastric cavity to calculate the dominant peristaltic frequency and kinetic energy distribution spectral density of the gastric wall muscles under the current pathological state. The minimum stiffness scalar value of the local locking magnetic field is calculated based on the dominant creep frequency and kinetic energy distribution spectral density. The steady-state control current in the local steady-state control parameter set is dynamically corrected based on the minimum stiffness scalar value.

Images