A shadowless lamp intelligent control system based on multi-data coupling

Abstract

The application relates to the technical field of medical devices, and discloses a shadowless lamp intelligent control system based on multi-data coupling. The shadowless lamp intelligent control system based on multi-data coupling provided by the application cooperates with a surgical field sensing and visual angle alignment module, establishes the spatial geometric relationship between the line of sight of a surgeon and the optical axis of a lamp head, eliminates the deviation between the optical axis and the line of sight of the surgeon, reduces the deep tissue glare, converts the smoke compensation into active prediction through a light field pre-rehearsal assembly, pre-adjusts the light spectrum before the smoke is generated, eliminates the visual interruption, pre-adjusts the lamp head posture through a depth pre-driving assembly, eliminates the deep cavity switching light spot jumping, monitors the baseline and the incident surface angle through a binocular parallax constraint unit, adjusts the edge lamp head through a cooperative control module, reduces the binocular parallax, and makes the shadowless lamp upgrade from single tracking to multi-physical field cooperative prediction control through the cooperative control module for weighted coupling of multiple parameters, so that the progressive unified output of visual angle alignment, process prediction and space prediction is realized.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to an intelligent control system for shadowless lamps based on multi-data coupling. Background Technology

[0002] Surgical shadowless lamps are the core lighting equipment in the operating room, and their lighting quality directly affects the surgeon's visual efficiency and surgical safety. Existing intelligent control systems for shadowless lamps have a certain degree of automation, mainly including the following technical approaches: capturing the position of the light spot in the surgical field through an image acquisition unit and driving the lamp head to move to achieve spot tracking; detecting the smoke concentration in the surgical field through a smoke sensor and triggering a compensation algorithm to increase illuminance or adjust the color temperature; and preset deep cavity lighting modes through the control panel, with the lamp head adjusting its posture according to a fixed program during switching.

[0003] The existing technologies described above have the following drawbacks: First, existing systems only track the position of the light spot and do not establish a spatial relationship between the surgeon's viewpoint and the axis of the shadowless lamp. There is often a spatial deviation between the direction of the shadowless lamp's axis and the surgeon's actual observation direction, leading to specular reflection glare on the surface of deep tissues or shadows forming at the edges of the surgical field, reducing the visibility of deep cavity operations. Second, existing smoke compensation is a passive response mechanism. The system only triggers brightness compensation after the smoke sensor detects smoke spreading to the sensing area, creating a visual interruption window. At the initial stage of smoke generation, the contrast of the surgical field has already significantly decreased, forcing the surgeon to interrupt the operation and wait for the light field to recover, affecting the continuity of the surgery. Third, existing deep cavity illumination modes are statically preset. When the surgeon switches to a deep cavity mode, the lamp head repositions according to a fixed program, causing the light spot to jump on the surgical field surface, requiring readjustment to the lighting environment. Furthermore, the preset angle cannot match the real-time operational intent, resulting in positioning lag. In summary, existing shadowless lamp control systems have not yet achieved surgeon's viewpoint alignment, active smoke prediction and compensation, and dynamic pre-adjustment of deep operating areas. There is a lack of effective coupling between lighting control and the surgical process, surgeon behavior, and the physical environment of the surgical field. Summary of the Invention

[0004] This application proposes an intelligent control system for shadowless lamps based on multi-data coupling. This system possesses advantages such as active alignment of the surgeon's viewpoint, early prediction of smoke diffusion, smooth transition of deep illumination, and collaborative constraint of binocular parallax across multiple lamp heads. It addresses technical problems in existing shadowless lamp control systems, including deviations between the optical axis and the surgeon's line of sight, visual interruptions due to smoke compensation, light spot jitter during deep cavity switching, and binocular parallax caused by differences in incident light from multiple lamp heads. The system identifies the main operator and evolves the line of sight direction through a surgical field perception module, establishes the spatial geometric registration relationship between the line of sight and the optical axis through a viewpoint alignment module, pre-simulates smoke diffusion and extrapolates the deep surgical area through a process extrapolation module, and dynamically allocates weights and couples outputs based on the surgical process stages through a collaborative control module. This achieves collaborative prediction and active adaptation of illumination parameters with the surgical process, surgeon behavior, surgical field physical environment, and human visual characteristics.

[0005] To achieve the above objectives, this application adopts the following technical solution: a surgical field perception module, a viewpoint alignment module, a process inference module, and a collaborative control module; The surgical field perception module includes a regional camera component, a panoramic camera component, and an operation behavior perception component. The regional camera component and the panoramic camera component are used to acquire surgical field images, identify the main operator, reconstruct the position evolution line of sight, and determine the binocular baseline. The operation behavior perception component is used to identify the type of surgical instruments and the range of motion entering the surgical field. The line of sight alignment module is connected to the surgical field perception module. A local reference is established with the main operation target as the origin. The line of sight is corrected and the optical axis is registered. The angle is calculated to generate the angle correction parameters. The lamp head is driven to align with the surgeon's line of sight. The angle between the baseline of both eyes and the incident surface of the lamp head is calculated to generate the binocular disparity constraint parameters. The process simulation module is connected to the surgical field perception module and the line of sight alignment module. It includes a light field pre-simulation component and a depth pre-driving component. Based on the instrument type and the motion amplitude analysis stage, it simulates the effect of surgical field smoke diffusion on light scattering coefficient and outputs spectral pre-modulation parameters. It simulates the deep position and generates pre-modulation instructions. The collaborative control module is connected to the line-of-sight alignment module and the process simulation module. It dynamically allocates the weights of the three paths according to the stage of the surgical process, performs coupled calculations on the three parameters, generates driving instructions, and outputs them.

[0006] Furthermore, the regional camera component is installed to the side of the operating lamp head to acquire images of the surgical field, extract the spatial coordinates of the main operating target, and monitor instrument obstruction and smoke disturbance information in the surgical field. The panoramic camera assembly includes two cameras installed diagonally on the ceiling of the operating room. The optical axes of the two cameras are arranged intersecting on the horizontal plane, and the overlapping area of ​​the field of view covers the operating table surface and the upper body area of ​​the surgical personnel. The panoramic camera assembly is based on a binocular stereo vision observation frame to detect and identify personnel in the operating room.

[0007] Furthermore, when the field of view of a single camera is obstructed, the panoramic camera component switches to monocular mode for the unobstructed camera and maintains eye tracking through filtering prediction; when the field of view of both cameras is obstructed or the face is completely obscured, extrapolation prediction is performed based on the historical trend of head posture changes, and binocular observation is restored after the obstruction is removed.

[0008] Furthermore, within the local spatial reference frame, the line-of-sight alignment module establishes the spatial geometric relationship between the corrected main viewing angle direction vector and the lamp head optical axis reference vector, and calculates the spatial angle between the two relative to the main operating target to quantify the azimuth deviation; based on the sensitivity coefficient of the lamp head mechanical structure, the azimuth deviation is converted into a viewing angle difference correction parameter, and the lamp head is driven to align the optical axis with the actual observation direction of the main operator. The sensitivity coefficient is determined through offline calibration experiments.

[0009] Furthermore, the gaze alignment module receives the binocular baseline vector, obtains the lamp head optical axis reference vector and the local axis direction vector of the local spatial reference frame, establishes the lamp head incident plane and calculates its normal vector, calculates the angle between the binocular baseline vector and the lamp head incident plane, and when the angle exceeds the allowable range, generates binocular disparity constraint parameters and outputs them to the collaborative control module.

[0010] Furthermore, the light field pre-simulation component obtains the instrument type and motion amplitude from the surgical field perception module, analyzes the current surgical process stage, queries the smoke production rate benchmark value based on the instrument type, and corrects it in combination with the motion speed, mapping the surgical process stage to fluid dynamics boundary conditions, and mapping the corrected smoke production rate to fluid dynamics source terms; it calls the fluid dynamics simplified model to deduce the impact of surgical field smoke diffusion on the spatiotemporal distribution of light scattering coefficient, and calculates the predicted value of light scattering coefficient; before the actual smoke is generated, it modulates the light source driving current ratio according to the predicted value of light scattering coefficient, increases the output ratio of short-wavelength blue light component, and outputs spectral pre-modulation parameters.

[0011] Furthermore, the depth pre-drive component receives the spatial coordinates and motion amplitude of the main operation target, obtains the view difference correction parameters and the process stage judgment output by the light field pre-simulation component, establishes a target spatial migration model, calculates the spatial migration trend vector and migration rate of the operation target to the deep region, combines the current operation target coordinates and the deep probability distribution of the process stage, and uses a time-series prediction method to deduce the spatial position of the deep operation area at the next moment, calculates the pre-deflection reference angle of the edge lamp head aligning with the deep region, and embeds the pitch compensation amount in the view difference correction parameters as a correction term, determines whether there is a possibility of large-angle movement based on the operation amplitude to determine the turning angle gain, generates attitude pre-adjustment commands, and drives the edge lamp head to complete a continuous and smooth attitude transition in a pre-drive manner before the operator actually goes deep.

[0012] Furthermore, the collaborative control module is connected to the viewpoint alignment module and the process prediction module, and gathers viewpoint difference correction parameters, binocular disparity constraint parameters, spectral premodulation parameters and spatial position of deep operation area. It receives the current surgical process stage judgment, dynamically allocates the three weight coefficients of viewpoint alignment, process prediction and spatial premodulation according to the current stage, and generates the final driving command.

[0013] Furthermore, the collaborative control module establishes a mapping relationship between the surgical process stage and the lighting demand weight, receives the current surgical process stage judgment, and dynamically allocates three weight coefficients for viewpoint alignment, process prediction and spatial pre-adjustment based on the current stage. When switching stages, the weight coefficients are smoothly transitioned through a first-order inertial filter.

[0014] Furthermore, before outputting the final driving command, the collaborative control module adjusts the target posture of the edge lamp heads according to the binocular disparity constraint parameters to reduce binocular depth perception deviation, performs conflict detection on the driving commands of each lamp head, and adopts a priority scheduling strategy to prioritize the posture adjustment of the main lighting lamp head when there is a risk of mechanical interference in the target posture of multiple lamp heads.

[0015] The beneficial effects of this invention are as follows: This application provides an intelligent control system for shadowless lamps based on multi-data coupling. Through the collaboration of surgical field perception and viewpoint alignment modules, it establishes the spatial geometric relationship between the surgeon's line of sight and the lamp head's optical axis, eliminating deviations between the optical axis and the surgeon's line of sight and reducing glare in deep tissues. The light field pre-simulation component transforms smoke compensation into active prediction, pre-adjusting the spectrum before smoke generation to eliminate visual interruptions. The depth pre-driving component pre-adjusts the lamp head's posture to eliminate spot jumps during deep cavity switching. The binocular parallax constraint unit monitors the angle between the baseline and the incident plane, and the collaborative control module adjusts the edge lamp head to reduce binocular parallax. Through the collaborative control module, multi-path parameter weighted coupling upgrades the shadowless lamp from single tracking to multi-physics field collaborative predictive control, achieving a progressive and unified output of viewpoint alignment, process prediction, and spatial prediction. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort: Figure 1 This is the system flowchart for this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] In surgical lighting control, there is a strong coupling relationship between surgeon's visual ergonomics and lighting quality. With the development of computer vision and intelligent control technology, intelligent control systems for shadowless lamps have become an important development direction for operating room equipment. However, most existing systems adopt unidirectional tracking logic, where the surgeon can only passively accept the position of the light spot generated by the system. The system cannot actively predict changes in lighting needs based on the surgeon's behavior, and it lacks multi-lamp collaborative constraints tailored to the surgeon's binocular visual characteristics, resulting in glare, shadows, and binocular parallax during deep procedures.

[0019] refer to Figure 1 This application provides an intelligent control system for shadowless lamps based on multi-data coupling, including a surgical field perception module, a viewing angle alignment module, a process deduction module, and a collaborative control module.

[0020] The surgical field perception module includes a regional camera component, a panoramic camera component, and a surgical behavior perception component.

[0021] The regional camera module employs an industrial-grade area scan camera, mounted on a side bracket of the surgical lamp head. Its optical axis is parallel to the main optical axis of the surgical lamp, with a fixed vertical distance of 150mm between them. The module is equipped with a fixed-focus lens with a focal length of 16mm, covering the effective projection range of the surgical lamp's illumination spot. The image resolution is 1920×1080 pixels, meeting the sub-pixel-level positioning accuracy requirements for feature points in the surgical field. The regional camera module connects to the system's main control unit via a gigabit Ethernet interface, acquiring data at a frame rate of 30 frames per second. It uses an image registration algorithm combining corner detection and feature descriptors to extract the spatial coordinates of the main operating target. Simultaneously, a dynamic detection strategy combining background subtraction and optical flow methods is employed to monitor instrument occlusion and smoke disturbance information in the surgical field. The background update rate for background subtraction is once every 5 frames, and the calculation window for optical flow is 15×15 pixels.

[0022] The panoramic camera assembly includes a first panoramic camera and a second panoramic camera, both employing industrial-grade area scan cameras equipped with motorized zoom lenses and a focal length adjustment range of 8mm to 24mm. The first and second panoramic cameras are mounted on pre-embedded brackets at diagonal positions on the ceiling of the operating room, at the same height (3.5m above the ground) and higher than the maximum lifting height of the shadowless lamp head (3.0m) to avoid obstruction of the camera's field of view by the movement of the shadowless lamp's robotic arm. The optical axes of the first and second panoramic cameras are arranged intersecting on a horizontal reference plane, with the intersection area covering the operating table and its surrounding operating space. The optical axes are tilted downwards relative to the horizontal reference plane, with the downward tilt angle determined based on the geometric relationship between the installation height and the operating table height. With an operating table height of 0.8m, the downward tilt angle is arctan(2.7 / 5.0) = 28.3°, ensuring that the overlapping area of ​​the two cameras' fields of view covers the operating table surface and the upper body area of ​​personnel standing around the operating table. Both cameras have an image resolution of 2560×1440 pixels, a frame rate of 25 frames / s, and a synchronization error of less than 1ms.

[0023] During the system initialization phase, the intrinsic and extrinsic parameters of the first and second panoramic cameras were calibrated to obtain their intrinsic parameter matrices, distortion coefficients, and relative pose relationships, thus establishing a binocular stereo vision observation framework. The intrinsic and extrinsic parameter calibration employed a checkerboard calibration method, with a checkerboard grid of 9×6 corner points and a single grid side length of 30mm. During calibration, a world coordinate system for the operating room was established with the geometric center of the operating table as the origin. The major axis of the operating table was designated as the X-axis, the minor axis as the Y-axis, and the vertically upward direction as the Z-axis. The baseline length for binocular stereo vision was the Euclidean distance between the two camera installation positions on the horizontal plane, typically 7.0m in a typical operating room layout.

[0024] The panoramic camera component simultaneously acquires images of the operating room scene through a first panoramic camera and a second panoramic camera, and performs the following steps under the binocular stereo vision observation framework to determine the main viewpoint direction vector.

[0025] Step 1: Personnel Detection and Identification. A lightweight human detection network is used to detect personnel in the images captured by the first and second panoramic cameras, obtaining the 2D bounding boxes and coordinates of key human points for each person. Key human points include the top of the head, neck, both shoulders, both elbows, both wrists, both hips, both knees, and both ankles, totaling 15 key points. The confidence threshold for the 2D bounding boxes is set to 0.6; targets below this threshold are filtered out. Based on a database of surgical personnel clothing features, the color, style, and accessories of each person's clothing are identified to distinguish the identities of the chief surgeon, assistant surgeons, and scrub nurses. The surgical personnel attire feature database pre-stores attire feature templates corresponding to different identity categories: the chief surgeon wears a green surgical gown with a chromaticity range of H:120-150, S:40-100, and V:30-90; the assistant surgeon wears a blue surgical gown with a chromaticity range of H:200-240, S:40-100, and V:30-90; and the scrub nurse wears a pink surgical gown with a chromaticity range of H:320-360, S:30-80, and V:40-100. The confidence threshold for identity recognition is set to 0.75.

[0026] Step Two: Primary Operator Locking. Based on the planar distance between the center coordinates of each operator's 2D bounding box and the center coordinates of the operating table (0, 0, 0), and combined with each operator's identity category, the current primary operator is determined. The rules for determining the primary operator are as follows: priority is given to the surgeon closest to the center of the operating table; if no surgeon is detected, the assistant surgeon closest to the center of the operating table is selected; if neither the surgeon nor the assistant surgeon is in the field of view, it is determined that there is no valid primary operator, and the system maintains the viewing angle alignment state of the previous cycle or switches to the default lighting mode. The default lighting mode is that the main optical axis of the shadowless lamp is vertically downward, and the illuminance is maintained at the baseline value.

[0027] Step 3: Position Information Extraction. Based on a binocular stereo vision observation framework, the 3D coordinates of the main operator's ankle key points are reconstructed using the triangulation principle. Triangulation is based on the 2D image coordinates of the same ankle key point from two cameras and the projection matrix obtained through calibration. The 3D coordinates are solved using the least squares method. The main operator's position coordinates in the operating room world coordinate system are determined. The update frequency of the position coordinates is consistent with the frame rate of the panoramic camera component, which is 25Hz.

[0028] Step 4: Head Pose Estimation. The facial region of the main operator is cropped and enlarged. Facial region cropping is based on the neck and top head keypoints output by the human detection network, extending upwards by 0.5 times the head length, downwards by 0.3 times the head length, and left and right by 0.4 times the head width to form the region of interest. A facial mesh detection model is used to extract key facial feature points, including the inner corners of the eyes, the tip of the nose, the corners of the mouth, and the chin, totaling six feature points. The two-dimensional coordinates of these key facial feature points in the binocular image are directly output by the facial mesh detection model, while the three-dimensional coordinates are reconstructed through binocular triangulation. The perspective n-point algorithm is used to calculate the head pose rotation matrix Rh. The iteration termination condition for the perspective n-point algorithm is a reprojection error of less than 1.0 pixel or an iteration count of 100. The head pose rotation matrix Rh describes the rotation relationship between the head coordinate system and the camera coordinate system. It is obtained by multiplying the rotation matrices around the X-axis, Y-axis, and Z-axis in sequence. The rotation matrix around the X-axis is determined by the pitch angle, the rotation matrix around the Y-axis is determined by the yaw angle, and the rotation matrix around the Z-axis is determined by the roll angle.

[0029] Step 5: Evolution of Gaze Direction and Determination of Binocular Baselines. Based on head pose estimation, the eye region image is further extracted. Eye region cropping is based on the inner corners of the left and right eyes, extending 15 pixels outwards and 10 pixels upwards and downwards to form the region of interest (ROI). An eye localization algorithm is used to locate the pupil center coordinates, achieving sub-pixel accuracy with a positioning error of less than 0.5 pixels. When both eyes are visible, the left pupil center Pl and the right pupil center Pr are located separately, and the bi-eye baseline vector vb = Pr - Pl is calculated. When one eye is occluded, based on the visible pupil center and facial symmetry assumptions, the position of the occluded eye pupil center is calculated using the nose tip as the center of symmetry, thus determining the bi-eye baseline vector vb. Based on the relative position of the pupil center in the head coordinate system and the head pose rotation matrix Rh, the operator's gaze direction vector vs is evolved. This gaze direction vector is obtained by multiplying the head pose rotation matrix by the unit direction vector of the pupil center relative to the eyeball center. When both eyes are visible, the average of the gaze direction vectors of both eyes is taken as the final gaze direction vector vs. When one eye is occluded, the gaze direction vector of the visible eye is taken as the final gaze direction vector vs., and the visibility recovery of the occluded eye is continuously monitored in subsequent frames. Visibility recovery is determined when the pupil center localization confidence of the region of interest of the eye exceeds 0.8.

[0030] Step Six: Capture Angle Correction. Based on the relative position of the main operator's standing coordinates Pstand and the operating table center coordinates Ptable=(0,0,0), calculate the standing offset angle δ. δ is the angle between the projection of the line connecting the main operator and the operating table center onto the horizontal reference plane and the major axis of the operating table, calculated from the components of the standing coordinates on the horizontal plane using the arctangent function. The viewing direction vector vs is corrected based on the standing offset angle δ to eliminate the viewing angle capture deviation caused by the main operator's position deviating from directly in front of the operating table. The corrected main viewing direction vector is obtained by rotating the viewing direction vector around the Z-axis by a negative standing offset angle. When the main operator is directly in front of the operating table, the standing offset angle δ is zero, and the main viewing direction vector does not require correction. The update frequency of the standing offset angle correction is 25Hz, consistent with the frame rate of the panoramic camera component.

[0031] Step 7: Field of View Occlusion Handling. When the field of view of either the first or second panoramic camera is obstructed by the robotic arm of the shadowless lamp, the crane, or other equipment, the panoramic camera assembly automatically switches to monocular mode for the unobstructed camera. In monocular mode, based on the historical frame sequence of the camera and the assumption of head posture continuity in the current frame, the operator's gaze direction vector is predicted through filtering. The filtering prediction uses a Kalman filter, with the state variables being the Euler angle representation of the head posture rotation matrix and the gaze direction vector, and the observation variables being the Euler angles of the head posture in the current frame. Both the process noise covariance matrix and the observation noise covariance matrix are diagonal matrices, with diagonal elements set according to the typical statistical characteristics of head posture changes. When both cameras are obstructed or the operator's face is completely covered by surgical drapes or a mask, the system switches to head posture extrapolation mode. Based on the historical trend of the head posture rotation matrix Rh, the current gaze direction vector is predicted using linear extrapolation, with an extrapolation time window of 5 frames, or 200ms. After the occlusion is removed, the system restores the normal line-of-sight evolution process under the binocular stereo vision observation framework, and corrects the prediction error accumulated during the extrapolation period with the data of the first frame after restoration. The correction method is a weighted average of the extrapolated vector and the measured vector after restoration, with a weight of 0.7:0.3.

[0032] The operation behavior perception component employs an instance segmentation network to segment surgical field images acquired by the regional camera component. The input image size of the instance segmentation network is 512×512 pixels, and the output includes instrument category labels and pixel-level masks. Instrument categories include electrosurgical units, ultrasonic scalpels, forceps, needle holders, and hemostatic forceps, totaling five categories. The confidence threshold for the instance segmentation network is set to 0.7; detection results below this threshold are filtered out. The operation behavior perception component detects the motion amplitude of surgical instruments using a continuous image frame difference method. Displacement is obtained by calculating the Euclidean distance between the instrument centroid coordinates in adjacent frames, and velocity is calculated as the ratio of displacement to the frame interval, which is 33.3 ms. The instrument centroid coordinates are the average of all pixel coordinates within the pixel-level mask.

[0033] The viewpoint alignment module is connected to the surgical field perception module, including a spatial association establishment unit, a viewpoint difference calculation unit, and a binocular disparity constraint unit.

[0034] The spatial association establishment unit receives the spatial position coordinates Ptarget of the main operating target and the corrected main view direction vector output by the surgical field perception module. A local spatial reference frame Σ is constructed with Ptarget as the dynamic geometric origin O: the Z-axis direction of the operating room world coordinate system is taken as the local Z-axis direction, and the corrected main viewpoint direction vector is used. The projection direction on the horizontal reference plane is the local X-axis direction, and the cross product direction of the local X-axis and the local Z-axis is the local Y-axis direction, forming a right-handed orthogonal coordinate system. The surgical field feature points and lamp reference coordinates are mapped to the frame Σ using a coordinate transformation matrix. This matrix consists of the rotation matrix of the local spatial reference frame relative to the operating room world coordinate system and the origin translation vector. Simultaneously, the corrected main viewpoint direction vector vs' is received, and the lamp optical axis reference vector VL is extracted. Both are mapped to the frame Σ using the coordinate transformation matrix, decoupling the orientation deviation between the line of sight and the optical axis, establishing spatial coplanar constraints between the surgeon and the lamp, and completing the registration of the line of sight and optical axis vectors.

[0035] The viewing angle difference calculation unit establishes the spatial geometric relationship between the corrected main viewing angle direction vector vs' and the lamp head optical axis reference vector VL within the local spatial reference frame Σ. It calculates the spatial angle θ between the two vectors relative to the main operating target through spatial vector operations. This spatial angle is calculated using the inverse cosine function of the product of the dot product and the modulus of the two vectors. This angle quantifies the azimuth deviation of the optical axis relative to the surgeon's observation direction. The viewing angle difference calculation unit establishes a mapping relationship between the lamp head's mechanical structure and the optical axis deflection sensitivity, converting the azimuth deviation into the viewing angle difference correction parameters required for lamp head operation. This ensures that the optical axis, after correction, is aligned with the actual observation direction of the main operator. Viewpoint difference correction parameters. Including horizontal rotation compensation h and pitch compensation ,satisfy:

[0036]

[0037] Wherein, kh and kp are the mechanical sensitivity coefficients of the lamp head in the horizontal and pitch directions, respectively, and φ is the projection angle of the azimuth deviation on the horizontal reference plane of the frame Σ, which is calculated by the arctangent function from the projection components of the azimuth deviation on the local X-axis and local Y-axis. The mechanical sensitivity coefficients kh and kp were determined through offline calibration experiments. The calibration method is as follows: within the frame Σ, standard attitudes with spatial angles of 1°, 2°, 3°, 5°, 8°, and 10° are set, and the ratio of the actual response angle of the lamp head to the standard attitude angle is recorded. kh and kp are obtained by linear fitting using the least squares method, with a goodness of fit of not less than 0.95. The update frequency of the angle difference calculation parameters is 25Hz, consistent with the frame rate of the panoramic camera component.

[0038] The binocular disparity constraint unit receives the binocular baseline vector vb output by the surgical field perception module, and obtains the lamp head optical axis reference vector VL and the local X-axis direction vector ex of the local spatial reference frame Σ. It establishes the lamp head incident plane spanned by the lamp head optical axis reference vector and the local X-axis direction vector, and calculates the normal vector of this plane, which is obtained by the cross product of the two vectors. The angle between the binocular baseline vector and the lamp head incident plane is calculated, which is obtained by dividing the absolute value of the dot product of the binocular baseline vector and the plane normal vector by the product of the magnitudes of the two vectors and then taking the arcsine function. The binocular disparity constraint unit compares the calculated angle with the binocular disparity tolerance angle, which is determined based on the multi-lamp head geometry of the shadowless lamp and the typical surgical observation distance. The shadowless lamp uses a multi-lamp head ring arrangement, typically with 5 to 7 lamp heads, a ring diameter of 0.6m, a surgeon's interocular distance of 0.065m, a typical surgical observation distance of 0.5m, and a binocular disparity tolerance angle of [missing information]. When the calculated angle exceeds 7.4°, binocular disparity constraint parameters are generated and output to the collaborative control module. These parameters include the directional characteristics of the binocular baseline vectors and the quantized value of the angle, guiding the collaborative control module's adjustment strategy for the incident angle of the edge lamp heads. The update frequency of the binocular disparity constraint parameters is 25Hz.

[0039] The process simulation module is connected to the surgical field perception module and the viewpoint alignment module, including the light field pre-simulation component and the depth pre-driving component.

[0040] The light field pre-simulation component acquires instrument type and motion amplitude from the surgical field perception module and analyzes the current surgical process stage. The surgical process stages include tissue incision, vascular dissection, deep anastomosis, and wound hemostasis, totaling four stages. The instrument types and motion amplitudes differ in each stage: The typical instrument for tissue incision is an electrocautery scalpel, with a high-frequency, small-amplitude vibration amplitude of 1–3 Hz and an amplitude of 2–5 mm; the typical instrument for vascular dissection is forceps, with a low-frequency, medium-amplitude displacement amplitude of 0.2–0.5 Hz and a single displacement of 10–30 mm; the typical instrument for deep anastomosis is a needle holder, with a low-frequency, precise displacement amplitude of 0.1–0.3 Hz and a single displacement of 1–5 mm; and the typical instrument for wound hemostasis is a hemostat, with an unstable, intermittent displacement amplitude, an irregular displacement frequency, and a single displacement of 5–20 mm. The process stage analysis is based on the joint determination of device type and motion amplitude. When the confidence level of device type identification exceeds 0.8 and the motion amplitude is within the above typical range, it is determined to be the corresponding stage.

[0041] The light field pre-simulation component incorporates an instrument-smoke production mapping table. This table, established based on clinical trial data, records baseline smoke production rates for different instrument types under varying amplitudes of motion. For example, the baseline smoke production rate for an electrosurgical unit under high-frequency, low-amplitude vibration is 150 mg / min; for forceps under low-frequency, medium-amplitude displacement, it's 20 mg / min; for a needle holder under low-frequency, precise displacement, it's 5 mg / min; and for hemostats under unstable, intermittent displacement, it's 30 mg / min. The light field pre-simulation component queries the instrument-smoke production mapping table based on the current instrument type to obtain the baseline smoke production rate, and then corrects this baseline value based on the movement speed to arrive at the actual smoke production rate (SL). The correction method is as follows: When the amplitude of movement exceeds a threshold determined according to the device type, the smoke generation rate is positively correlated with the amplitude of movement. The actual smoke generation rate SL is equal to the baseline smoke generation rate multiplied by the sum of the products of speed and speed correction coefficient. The speed correction coefficient is determined according to the device type: 0.1 s / mm for electrosurgical units, 0.05 s / mm for forceps, 0.02 s / mm for needle holders, and 0.08 s / mm for hemostats. The unit of speed is mm / s. When the amplitude of movement is less than or equal to this threshold, the actual smoke generation rate SL is equal to the baseline smoke generation rate. The amplitude of movement threshold is determined according to the device type: 5 mm for electrosurgical units, 30 mm for forceps, 5 mm for needle holders, and 20 mm for hemostats.

[0042] The light field pre-simulation component maps the surgical procedure stages to fluid dynamics boundary conditions and the actual smoke generation rate SL to fluid dynamics source terms, forming the input parameter set of a simplified fluid dynamics model. This simplified model is then used to deduce the impact of surgical field smoke diffusion on the spatiotemporal distribution of the light scattering coefficient. The simplified fluid dynamics model is a single-phase flow model, neglecting particle aggregation effects. It uses the surgical procedure stages as boundary conditions, the actual smoke generation rate SL as the source term, and the spatiotemporal distribution of the light scattering coefficient as the output. Its governing equations are:

[0043] Where C represents the smoke concentration field, t represents time, u represents the airflow velocity field in the surgical field, and D represents the smoke particle diffusion coefficient. For gradient operators, Here, SL is the Laplace operator, and SL is the actual smoke generation rate. The airflow velocity field in the surgical field is set according to the nominal wind speed of the laminar flow ventilation system in the operating room, with a typical value of 0.25 m / s, and the direction is a horizontal unidirectional flow from the head side of the operating table to the foot side. The smoke particle diffusion coefficient is determined by calculating the average particle size of the smoke particles using the Stokes-Einstein equation. The average particle size of the smoke particles is 0.5 μm, and the diffusion coefficient is... The light field pre-simulation component uses the finite difference method to numerically solve the aforementioned governing equations. The computational grid is divided according to the spatial scale of the surgical field, with a typical field size of 0.3m × 0.3m × 0.2m. The grid is divided into 30 × 30 × 20 elements, with each element measuring 10mm × 10mm × 10mm. The time step is determined based on numerical stability conditions, and typically satisfies these conditions, with a time step of 10ms. The calculation cycle updates the prediction field every 100ms, meaning a prediction result is output every 10 time steps.

[0044] The light field pre-simulation component calculates the predicted value of the light scattering coefficient. The light scattering coefficient is proportional to the smoke concentration field. The proportionality coefficient is determined by scattering theory based on the complex refractive index and particle size distribution of the smoke particles. The complex refractive index is 1.5-0.1i, the particle size distribution is log-normal, the geometric mean diameter is 0.5μm, and the geometric standard deviation is 1.5. The calculated proportionality coefficient is... The light field pre-simulation component modulates the light source driving current ratio based on the predicted light scattering coefficient. The modulation strategy is as follows: when the predicted light scattering coefficient is greater than a threshold value... At that time, the output ratio of short-wavelength blue light component is increased. The increase in the output ratio of blue light component is linearly related to the predicted value of light scattering coefficient, and the linearity coefficient is [value missing]. The maximum increase in the blue light component output ratio is 30%; when the predicted light scattering coefficient is less than or equal to the threshold... During this process, the current driving current ratio of the light source is maintained. The spectral pre-modulation parameters include the driving current adjustment amount for each light emission channel, which includes a blue light channel, a green light channel, a red light channel, and a warm white light channel, totaling four channels, and are output to the collaborative control module. The light field pre-simulation component outputs the current surgical process stage judgment to the collaborative control module. The stage judgment update frequency is 10Hz, consistent with the frame rate of the regional camera component.

[0045] The depth pre-drive component receives the spatial coordinates and motion amplitude of the main operating target output by the surgical field perception module, acquires the viewpoint difference correction parameters output by the viewpoint alignment module and the process stage judgment output by the light field pre-simulation component, and establishes a target spatial migration model. The method for establishing the target spatial migration model is as follows: taking the current spatial coordinates of the main operating target as the starting point, the instrument movement direction as the migration direction, and the instrument movement speed as the migration rate, a spatial migration trend vector and migration rate are constructed. The migration direction is determined by moving average filtering based on the displacement direction of the instrument centroid coordinates in multiple consecutive frames of images. The moving average window is 5 frames, or 150ms. The migration rate is determined based on the instrument movement amplitude and the probability distribution of deep operations in the process stage. The probability distribution of deep operations in the process stage is established based on statistical analysis of clinical surgical data: the probability of deep migration in the tissue incision stage is 0.1, the probability of deep migration in the vascular dissection stage is 0.3, the probability of deep migration in the deep anastomosis stage is 0.8, and the probability of deep migration in the wound hemostasis stage is 0.2.

[0046] The depth pre-drive component receives migration trend data and, combined with the spatial coordinates of the current primary target and the depth probability distribution at each stage of the process, establishes a depth inference model. The depth inference model employs a time-series prediction method based on a Hidden Markov Model (HMM). The state space is a discrete grid representing the spatial location of the deep operation area, and the observation sequence consists of the spatial coordinates of the current primary target and the migration trend vector. A forward algorithm is used to calculate the posterior probability distribution of the spatial location of the deep operation area at the next moment. The position corresponding to the maximum probability is taken as the inference result to determine the probability of deep operation. The state transition probability of the HMM is set according to the depth operation probability distribution at each stage of the process, and the observation probability is set according to the matching degree between the spatial migration trend vector and the state transition direction. The depth pre-drive component calls the viewpoint difference correction parameters to establish an attitude pre-adjustment model, calculating the pre-deflection reference angle for the edge lamp head aligning with the deep region. The pre-deflection reference angle is obtained by dividing the horizontal distance between the deep operation area and the current target by the installation height of the shadowless lamp, taking the arctangent function, and adding the pitch adjustment compensation amount from the viewpoint difference correction parameters. The typical installation height of the shadowless lamp is 2.5m. The depth pre-drive component determines whether a large-angle movement is possible based on the operational amplitude. The determination method is as follows: when the instrument's movement speed is greater than the speed threshold of 10 mm / s, and the displacement direction change angle for 5 consecutive frames is less than the angle threshold of 15°, a large-angle movement is considered possible, and the angle gain coefficient is 1.5; otherwise, the angle gain coefficient is 1.0. The target angle in the pre-deflection command is equal to the angle gain coefficient multiplied by the pre-deflection reference angle. Before the operator actually enters the deep region, the depth pre-drive component drives the edge lamp head to complete a continuous and smooth attitude transition in a pre-drive manner. The pre-drive lead is 3 seconds, that is, when the simulation results indicate that the operator will enter the deep region in 3 seconds, the pre-drive control is initiated. The pre-drive control uses an S-curve acceleration and deceleration planning algorithm to plan the motion trajectory of the edge lamp head from the current attitude to the target attitude. The acceleration of the S-curve is a constant value, with a typical value of [value missing]. This allows for continuous changes in the lamp head's angular velocity and angular acceleration, preventing light spot flickering. The depth pre-drive component outputs attitude pre-adjustment commands and the spatial position of the deep operating area to the collaborative control module, with an update frequency of 10Hz.

[0047] The collaborative control module is connected to the perspective alignment module and the process inference module, and includes a weight allocation unit and a coupling operation unit.

[0048] The weight allocation unit aggregates the viewpoint difference correction parameters and binocular disparity constraint parameters output by the viewpoint alignment module, the spectral premodulation parameters output by the process prediction module, and the spatial location of the deep operation area. It also receives the current surgical process stage judgment output by the process prediction module and establishes a mapping relationship between the surgical process stage and the lighting requirement weights. The stage-to-lighting requirement weight mapping is pre-set based on the lighting requirement characteristics of each surgical process stage, including weight configurations for four stages: For the tissue incision stage, the weight configuration is viewpoint alignment weight 0.2, process prediction weight 0.5, and spatial premodulation weight 0.3; for the vascular dissection stage, the weight configuration is viewpoint alignment weight 0.3, process prediction weight 0.3, and spatial premodulation weight 0.4; for the deep anastomosis stage, the weight configuration is viewpoint alignment weight 0.2, process prediction weight 0.3, and spatial premodulation weight 0.5; and for the wound hemostasis stage, the weight configuration is viewpoint alignment weight 0.5, process prediction weight 0.2, and spatial premodulation weight 0.3. The weight allocation unit dynamically allocates the three weight coefficients according to the current stage, and the sum of the three weight coefficients is 1. During phase switching, a first-order inertial filter is used to smoothly transition the weight coefficients, avoiding sudden changes in lighting parameters caused by abrupt weight changes. The filtering time constant of the first-order inertial filter is determined based on the human eye's perception threshold for changes in illumination, with a typical value of 0.5s, meaning that the human eye cannot perceive the illuminance jump caused by weight switching within 0.5s.

[0049] The coupling operation unit receives weighting coefficients and input parameters from various channels to establish a multiphysics weighted coupling operation model. This model includes a viewing angle alignment drive function, a spectral modulation drive function, and a spatial pre-adjustment drive function. The viewing angle alignment drive function maps the viewing angle difference correction parameter to the lamp head angle adjustment amount. The mapping relationship is linear, with a proportionality coefficient of 1.0, meaning the lamp head angle adjustment amount equals the viewing angle difference correction parameter. The spectral modulation drive function maps the spectral pre-modulation parameter to the drive current value of each light-emitting channel. The mapping relationship is determined based on the photoelectric characteristic curve of the light-emitting element. The drive current of the blue, green, red, and warm white light channels has an approximately linear relationship with the output luminous flux, with a linearity of no less than 95%. The spatial pre-adjustment drive function maps the spatial position of the deep operating area to the pre-deflection angle of the edge lamp head. The mapping relationship is a geometric projection relationship. The pre-deflection angle is equal to the arctangent function of the horizontal distance between the deep operating area and the current operating target divided by the installation height of the shadowless lamp. The coupling operation unit performs weighted coupling operations on the three input parameters to calculate the angle, illuminance, and spectral drive of each lamp head of the shadowless lamp, generating the final drive command. The final drive command is obtained by weighted summation of the three input parameters after mapping through their corresponding drive functions, with the weights being the weight coefficients allocated to the three input parameters in the current stage. The update frequency of the coupling operation unit is 25Hz, consistent with the frame rate of the panoramic camera component.

[0050] Before outputting the final drive command, the collaborative control module adjusts the target attitude of the edge lamp head based on the binocular disparity constraint parameters. When the binocular disparity constraint parameters indicate that the angle between the binocular baseline vector and the lamp head's incident plane exceeds the allowable range... To reduce the angle difference between the optical axis of each edge lamp head and the baseline vector of the eyes, the adjustment strategy is as follows: calculate the angle between the optical axis of each edge lamp head and the baseline vector of the eyes, sort them from largest to smallest angle, prioritize adjusting the edge lamp head with the largest angle, and adjust by 30% of the angle difference each time, iterating until the angle between the optical axis of all edge lamp heads and the baseline vector of the eyes is less than 30%. This can be repeated up to a maximum of 10 iterations. This ensures that the projection components of the incident optical axes of each lamp head onto the baseline direction of both eyes are consistent, reducing binocular depth perception bias.

[0051] Simultaneously, conflict detection is performed on the drive commands for each lamp head. When there is a risk of mechanical interference in the target postures of multiple lamp heads, a priority scheduling strategy is adopted to prioritize the posture adjustment of the main lighting lamp head. The criteria for conflict detection include the minimum safe distance between lamp heads, the limits of the robotic arm joint angles, and the degree of spatiotemporal overlap of the lamp head motion trajectories. The minimum safe distance is the minimum spacing between the lamp head housing contours, typically 50mm. The limits of the robotic arm joint angles include the horizontal rotation joint angle range of... to The pitch joint angle range is to The spatiotemporal overlap of the lamp head motion trajectories is determined by predicting the motion trajectories of each lamp head within the next 100ms and detecting whether the trajectory envelopes intersect. When a conflict risk is detected, the main lighting lamp head executes the original drive command, while the auxiliary lighting lamp head adjusts its target orientation to avoid the interference area while maintaining the lighting coverage. The adjustment amount is 50% of the original drive command, and the direction is away from the conflict area.

[0052] After the final drive command output by the coordinated control module drives the lamp head to execute, the lamp head attitude feedback information is returned to the viewing angle alignment module and the process inference module via the lamp head encoder to update the reference state, forming a closed-loop control. The lamp head encoder feedback includes the current horizontal rotation angle, pitch angle, and lamp head spatial coordinates, with a feedback frequency of 100Hz, consistent with the lamp head servo control cycle. The viewing angle alignment module updates the lamp head optical axis reference vector based on the feedback lamp head spatial coordinates, and the process inference module updates the current attitude reference of the depth pre-drive based on the feedback lamp head attitude, with an update frequency of 25Hz.

[0053] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A shadowless lamp intelligent control system based on multi-data coupling, characterized in that, It includes a surgical field perception module, a viewpoint alignment module, a process deduction module, and a collaborative control module; The surgical field perception module includes a regional camera component, a panoramic camera component, and an operation behavior perception component. The regional camera component and the panoramic camera component are used to acquire surgical field images, identify the main operator, reconstruct the position evolution line of sight, and determine the binocular baseline. The operation behavior perception component is used to identify the type of surgical instruments and the range of motion entering the surgical field. The line of sight alignment module is connected to the surgical field perception module. A local reference is established with the main operation target as the origin. The line of sight is corrected and the optical axis is registered. The angle is calculated to generate the angle correction parameters. The lamp head is driven to align with the surgeon's line of sight. The angle between the baseline of both eyes and the incident surface of the lamp head is calculated to generate the binocular disparity constraint parameters. The process simulation module is connected to the surgical field perception module and the line of sight alignment module. It includes a light field pre-simulation component and a depth pre-driving component. Based on the instrument type and the motion amplitude analysis stage, it simulates the effect of surgical field smoke diffusion on light scattering coefficient and outputs spectral pre-modulation parameters. It simulates the deep position and generates pre-modulation instructions. The collaborative control module is connected to the line-of-sight alignment module and the process simulation module. It dynamically allocates the weights of the three paths according to the stage of the surgical process, performs coupled calculations on the three parameters, generates driving instructions, and outputs them.

2. The intelligent control system for shadowless lamps based on multi-data coupling according to claim 1, characterized in that, The regional camera component is installed to the side of the operating lamp head to acquire images of the surgical field, extract the spatial coordinates of the main operating target, and monitor instrument obstruction and smoke disturbance information in the surgical field. The panoramic camera assembly includes two cameras installed diagonally on the ceiling of the operating room. The optical axes of the two cameras are arranged intersecting on the horizontal plane, and the overlapping area of ​​the field of view covers the operating table surface and the upper body area of ​​the surgical personnel. The panoramic camera assembly is based on a binocular stereo vision observation frame to detect and identify personnel in the operating room.

3. The intelligent control system for shadowless lamps based on multi-data coupling according to claim 2, characterized in that, When the field of view of a single camera is obstructed, the panoramic camera component switches to monocular mode for the unobstructed camera and maintains eye tracking through filtering prediction. When the field of view of both cameras is obstructed or the face is completely obscured, extrapolation prediction is performed based on the historical trend of head posture changes, and binocular observation is restored after the obstruction is removed.

4. The intelligent control system for shadowless lamps based on multi-data coupling according to claim 3, characterized in that, The line-of-sight alignment module establishes the spatial geometric relationship between the corrected main viewing angle direction vector and the lamp head optical axis reference vector within the local spatial reference frame, and calculates the spatial angle between the two relative to the main operating target to quantify the azimuth deviation; based on the sensitivity coefficient of the lamp head mechanical structure, the azimuth deviation is converted into a viewing angle difference correction parameter, and the lamp head is driven to align the optical axis with the actual observation direction of the main operator. The sensitivity coefficient is determined through offline calibration experiments.

5. The intelligent control system for shadowless lamps based on multi-data coupling according to claim 4, characterized in that, The gaze alignment module receives the binocular baseline vector, obtains the lamp head optical axis reference vector and the local axis direction vector of the local spatial reference frame, establishes the lamp head incident plane and calculates its normal vector, calculates the angle between the binocular baseline vector and the lamp head incident plane, and when the angle exceeds the allowable range, generates binocular disparity constraint parameters and outputs them to the collaborative control module.

6. The intelligent control system for shadowless lamps based on multi-data coupling according to claim 5, characterized in that, The light field pre-simulation component obtains the instrument type and motion amplitude from the surgical field perception module, analyzes the current surgical process stage, queries the smoke production rate benchmark value according to the instrument type, and makes corrections in combination with the motion speed, mapping the surgical process stage to fluid dynamic boundary conditions, and mapping the corrected smoke production rate to fluid dynamic source terms. The influence of field smoke diffusion on the spatiotemporal distribution of light scattering coefficient is simulated by using a simplified fluid dynamics model, and the predicted value of light scattering coefficient is calculated. Before the actual generation of smoke, the driving current ratio of the light source is modulated according to the predicted value of light scattering coefficient to increase the output ratio of short-wavelength blue light component and output spectral pre-modulation parameters.

7. The intelligent control system for shadowless lamps based on multi-data coupling according to claim 6, characterized in that, The depth pre-drive component receives the spatial coordinates and motion amplitude of the main operating target, obtains the view difference correction parameters and the process stage judgment output by the light field pre-simulation component, establishes a target spatial migration model, calculates the spatial migration trend vector and migration rate of the operating target to the deep region, combines the current operating target coordinates and the deep probability distribution of the process stage, and uses a time-series prediction method to deduce the spatial position of the deep operating area at the next moment, calculates the pre-deflection reference angle of the edge lamp head aligned with the deep region, and embeds the pitch compensation amount in the view difference correction parameters as a correction term in the calculation of the pre-deflection reference angle. Based on the operation amplitude, it determines whether there is a possibility of large-angle movement to determine the turning angle gain, generates attitude pre-adjustment commands, and drives the edge lamp head to complete a continuous and smooth attitude transition in a pre-drive manner before the operator actually goes deep.

8. The intelligent control system for shadowless lamps based on multi-data coupling according to claim 7, characterized in that, The collaborative control module is connected to the viewpoint alignment module and the process prediction module. It gathers viewpoint difference correction parameters, binocular disparity constraint parameters, spectral premodulation parameters and spatial location of deep operation areas, receives the current surgical process stage judgment, and dynamically allocates the three weight coefficients of viewpoint alignment, process prediction and spatial premodulation according to the current stage to generate the final driving command.

9. The intelligent control system for shadowless lamps based on multi-data coupling according to claim 8, characterized in that, The collaborative control module establishes a mapping relationship between the surgical process stage and the lighting demand weight. It receives the current surgical process stage judgment and dynamically allocates three weight coefficients for viewpoint alignment, process prediction and spatial pre-adjustment based on the current stage. When switching stages, the weight coefficients are smoothly transitioned through a first-order inertial filter.

10. The intelligent control system for shadowless lamps based on multi-data coupling according to claim 9, characterized in that, Before outputting the final drive command, the collaborative control module adjusts the target posture of the edge lamp heads according to the binocular disparity constraint parameters to reduce binocular depth perception deviation. It also performs conflict detection on the drive commands of each lamp head. When there is a risk of mechanical interference in the target posture of multiple lamp heads, a priority scheduling strategy is adopted to prioritize the posture adjustment of the main lighting lamp head.

Images