An aerial imaging shadowless lamp control system fusing an infrared VCSEL array

Abstract

The application relates to the technical field of aerial imaging, and discloses an aerial imaging shadowless lamp control system fusing an infrared VCSEL array, which comprises an aerial real image interaction module, a physician visual angle tracking module, a multi-joint mechanical arm positioning module, a multi-mode interaction management module and an interaction abnormality protection module. Infrared structured light is generated by using an infrared VCSEL array light source of a specific wave band, and an aerial real image interaction interface is formed via a negative refractive index flat lens group. The light source wave band is specially configured to have anti-interference to common illumination light sources in an operating room, so that the interference of ambient light on the aerial interface display effect can be reduced, the display stability and operation recognition reliability of the real image interaction interface under a complex surgical light environment are ensured, the interaction difficulty problem caused by unclear interface is solved, and the accuracy and fluency of intraoperative control are improved.

Description

Technical Field

[0001] This invention relates to the field of aerial imaging technology, specifically to an aerial imaging shadowless lamp control system that integrates an infrared VCSEL array. Background Technology

[0002] Aerial imaging technology is an advanced display technology that uses the principle of light field modulation based on micro-nano structures to form three-dimensional images in the air without the need for a physical screen.

[0003] Currently, due to the complex environment of the operating room and the stringent requirements for aseptic operation, the display stability of the interactive interface generated in the air during intraoperative equipment control is easily affected by ambient light. When the brightness and contrast of the interface are insufficient, it will cause difficulties for doctors to identify and operate, affecting the reliability of the interaction.

[0004] Therefore, a control system for aerial imaging shadowless lamps that integrates an infrared VCSEL array is proposed to solve the above problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an aerial imaging shadowless lamp control system that integrates an infrared VCSEL array, solving the problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an aerial imaging shadowless lamp control system integrating an infrared VCSEL array, the system comprising: The aerial real-image interaction module is integrated into the lamp head assembly of the shadowless lamp. It consists of an infrared VCSEL array light source, a negative refractive index flat lens group and a single-sided infrared interactive sensor. It forms a real-image interaction interface in the air without a medium and detects non-contact operations, generating interactive command data. The physician's perspective tracking module, connected to the aerial real-image interaction module, includes a face tracking camera and a time-of-flight distance sensor, which captures the physician's facial features and distance information in the surgical area in real time and generates perspective tracking data. The multi-joint robotic arm positioning module is connected to the physician's perspective tracking module and the aerial real-image interaction module. It receives the perspective tracking data and drives the multi-joint robotic arm to move through the built-in transmission device. It adjusts the spatial pose of the shadowless lamp head assembly and the aerial real-image interaction module so that the real-image interaction interface automatically follows the physician's perspective. The multi-mode interaction management module is connected to the aerial real-image interaction module and the physician's perspective tracking module. It receives the interaction command data and the perspective tracking data, identifies the identities of multiple surgical personnel, assigns interaction permission priorities, and generates interface management commands. The interaction anomaly protection module is connected to the multi-mode interaction management module, the physician's perspective tracking module, and the multi-joint robotic arm positioning module. It monitors the environmental status of the surgical area in real time. When a preset abnormal scenario is detected, it triggers the protection mechanism and generates system behavior control instructions.

[0007] Preferably, the aerial real-image interaction module includes a light source generation subunit, a real-image generation subunit, and an interactive perception subunit; The light source generating subunit is an infrared VCSEL array light source that emits infrared structured light in a specific wavelength band; The real image generation subunit is a negative refractive index flat lens group that receives and modulates the infrared structured light to form a real image interactive interface that can be observed with the naked eye at a preset position in the air. This unit determines the apparent position of the real image interactive interface formed in the air based on the following optical relationship: ; in, This represents the apparent distance from the real-image interactive interface to the optical center of the negative refractive index flat lens group. This represents the equivalent focal length of the negative refractive index flat lens group. This represents the actual object distance from the infrared VCSEL array light source to the optical center of the negative refractive index flat lens group; The interactive sensing subunit is a one-sided infrared interactive sensor that detects non-contact gestures performed by the operator within a preset spatial domain of the real-image interactive interface and generates the interactive command data.

[0008] Preferably, the physician's perspective tracking module includes a facial information acquisition unit, a spatial distance perception unit, and a tracking drive unit; The facial information acquisition unit is a high-resolution face tracking camera that captures the doctor's facial feature points and calculates the direction of the doctor's gaze point in real time. The spatial distance sensing unit is a time-of-flight distance sensor that works in conjunction with the facial information acquisition unit to measure the real-time distance between the physician’s face and the shadowless lamp head assembly. The tracking drive unit is a servo transmission device integrated into the positioning module of the multi-joint robotic arm. It receives data generated by the facial information acquisition unit and the spatial distance perception unit, synthesizes the viewpoint tracking data, and drives the transmission device.

[0009] Preferably, the multi-joint robotic arm positioning module includes a mechanical support unit, a multi-degree-of-freedom motion unit, and a pose calculation unit; The mechanical support unit is a series and parallel multi-joint robotic arm structure that physically supports the shadowless lamp head assembly, the aerial real-image interaction module, and the physician's perspective tracking module. The multi-degree-of-freedom motion unit is composed of multiple servo motors and the transmission device. It receives the drive signal from the tracking drive unit and realizes the flexible multi-degree-of-freedom motion of the mechanical support unit in three-dimensional space. The pose calculation unit, equipped with a built-in inverse kinematics algorithm, receives information about the target pose from the view tracking data, calculates and outputs control parameters for each joint motor in the multi-degree-of-freedom motion unit, ensuring that the real-image interactive interface is stably maintained within a preset optimal viewing area. This unit calculates the control parameters for each joint using the following inverse kinematics model: ; ; in, This indicates the angle that the multi-joint robotic arm needs to rotate. This represents the inverse kinematics solution function designed for the specific configuration of the multi-joint robotic arm. It is a transformation matrix. It is a rotation matrix. It is a position vector. This represents the transpose of the zero vector. This is the transpose operator.

[0010] Preferably, the multi-mode interaction management module includes a multi-user identification unit, a permission arbitration unit, and an interface scheduling unit; The multi-user identification unit is connected to the physician's perspective tracking module, and identifies and distinguishes the identities of multiple operators in the surgical area based on the facial feature information in the perspective tracking data. The permission arbitration unit is connected to the multi-user identification unit and has pre-stored interaction permission priority rules based on surgical roles. When the multi-user identification unit identifies multiple people, it arbitrates and determines the attending physician with the highest interaction permission according to the rules. The interface scheduling unit is connected to the permission arbitration unit and the aerial real-image interaction module. Based on the output result of the permission arbitration unit, it generates the interface management instruction and controls the aerial real-image interaction module to activate and update the real-image interaction interface only for the operator with the highest interaction authority.

[0011] Preferably, the interaction anomaly protection module includes an environmental monitoring unit, an anomaly scene determination unit, and an instruction circuit breaker unit; The environmental monitoring unit is connected to the physician's perspective tracking module and the multi-joint robotic arm positioning module to acquire real-time information on sudden movement of personnel in the surgical area, the motion status of the robotic arm positioning module, and system load data. The abnormal scene determination unit has pre-set abnormal scene logic including rapid near-field intrusion, robotic arm movement exceeding limits, and system overload. It compares the data obtained by the environmental monitoring unit with the pre-set logic, and determines it as an abnormal scene when the match is successful. The instruction circuit breaker unit is connected to the abnormal scene determination unit, the multi-mode interaction management module, and the multi-joint robotic arm positioning module. When the abnormal scene determination unit outputs a determination signal, it triggers a protection mechanism to generate the system behavior control instruction. The instruction includes: freezing the real-image interaction interface, interrupting the current non-contact operation response, and controlling the multi-joint robotic arm positioning module to decelerate and pause movement.

[0012] Preferably, the infrared VCSEL array light source in the light source generating subunit emits infrared structured light with a wavelength range configured to resist interference from common interfering light sources in the operating room, and matches the working band of the single-sided infrared interactive sensor, so as to maintain stable gesture interaction detection in complex light environments.

[0013] Preferably, the facial information acquisition unit and the spatial distance sensing unit are arranged coaxially in space. The acquired facial feature point coordinates and the measured distance information are fused together by a data fusion algorithm to calculate the position and orientation of the physician's perspective in three-dimensional space and generate high-precision perspective tracking data.

[0014] Preferably, the multi-user identification unit integrates a continuous learning model to learn the facial feature changes of the identified physician during a single surgery. The permission arbitration unit is also connected to a manual priority overlay interface to receive temporary permission allocation instructions from the surgeon and update the current interaction permission priority according to the instructions.

[0015] Preferably, the "rapid near-field intrusion" scenario preset in the abnormal scenario determination unit is defined as a person without the current interaction permission rapidly entering and blocking the common field of view of the time-of-flight distance sensor and the face tracking camera within a preset time threshold. When this scenario is determined, the system behavior control instruction generated by the instruction circuit breaker unit further includes: sending a signal to the multi-mode interaction management module to temporarily lock the interface scheduling unit and prevent the switching of interaction permissions.

[0016] Compared with the prior art, the present invention provides an aerial imaging shadowless lamp control system that integrates an infrared VCSEL array, which has the following beneficial effects: 1. In this invention, during surgical interactive control, infrared structured light is generated by using an infrared VCSEL array light source with a specific wavelength, and a real image interactive interface is formed in the air via a negative refractive index flat lens group. The wavelength of this light source is specially configured to be anti-interference against common lighting sources in the operating room, thereby reducing the interference of ambient light on the display effect of the air interface, ensuring the display stability and operation recognition reliability of the real image interactive interface in complex surgical lighting environments, solving the problem of interaction difficulties caused by unclear interfaces, and improving the accuracy and smoothness of intraoperative control.

[0017] 2. In this invention, during multi-user collaborative surgical operations, the multi-mode interaction management module identifies the identities of personnel within the surgical area in real time and determines a unique high-authority interaction subject based on preset permission rules. Simultaneously, the interaction anomaly protection module monitors for rapid near-field intrusion anomalies by unauthorized personnel and triggers interface locking and permission switching prevention mechanisms. This enables the system to intelligently manage interaction permissions, eliminate control command conflicts caused by multiple accidental touches and simultaneous operations, and ensure the clarity of control rights and the orderliness of the operation process.

[0018] 3. In this invention, during the process of controlling the multi-joint robotic arm positioning module to achieve automatic tracking of the surgeon's perspective by the shadowless lamp and aerial interface, the surgeon's posture is obtained in real time through the surgeon's perspective tracking module, and the interaction anomaly protection module continuously monitors the movement status of the robotic arm and sudden movement of personnel. When an abnormal scenario is determined to be a rapid near-field intrusion that will cause a collision, the command circuit breaker mechanism is immediately triggered to restrict the movement of the robotic arm. This enables the system to achieve accurate and automatic posture tracking while having active safety protection capabilities, which can prevent accidental contact and collision between the robotic arm and personnel in the surgical area, and ensure the physical safety of the surgical environment. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the aerial imaging shadowless lamp control system integrating an infrared VCSEL array, as described in this invention. Figure 2 This is a structural diagram of the physician's perspective tracking module of the present invention; Figure 3 This is a diagram illustrating the architecture of the multi-mode interactive management module of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figures 1-3 This is a control system for an aerial imaging shadowless lamp that integrates an infrared VCSEL array. The system includes: The aerial real-image interaction module is integrated into the lamp head assembly of the shadowless lamp. It consists of an infrared VCSEL array light source, a negative refractive index flat lens group and a single-sided infrared interactive sensor. It forms a real-image interaction interface in the air without a medium and detects non-contact operations, generating interactive command data. The physician's perspective tracking module, connected to the aerial real-image interaction module, includes a face tracking camera and a time-of-flight distance sensor to capture the physician's facial features and distance information in the surgical area in real time and generate perspective tracking data. The multi-joint robotic arm positioning module is connected to the physician's view tracking module and the aerial real-image interaction module. It receives view tracking data and drives the multi-joint robotic arm to move through the built-in transmission device. It adjusts the spatial pose of the shadowless lamp head assembly and the aerial real-image interaction module so that the real-image interaction interface automatically follows the physician's view. The multi-mode interactive management module connects with the aerial real-image interaction module and the physician's perspective tracking module. It receives interactive command data and perspective tracking data, identifies the identities of multiple surgical personnel, assigns interaction permission priorities, and generates interface management commands. The interactive anomaly protection module is connected to the multi-mode interactive management module, the physician's perspective tracking module, and the multi-joint robotic arm positioning module. It monitors the environmental status of the surgical area in real time. When a preset abnormal scenario is detected, the protection mechanism is triggered and system behavior control instructions are generated.

[0022] The aerial real-image interaction module includes a light source generation subunit, a real-image generation subunit, and an interactive perception subunit; The light source generating subunit is an infrared VCSEL array light source that emits infrared structured light in a specific wavelength band; The real image generation subunit is a negative refractive index flat lens group that receives and modulates infrared structured light to form a real image interactive interface that can be observed with the naked eye at a preset position in the air. This unit determines the apparent position of the real image interactive interface formed in the air based on the following optical relationship: ; in, This represents the apparent distance from the real-image interactive interface to the optical center of the negative refractive index flat lens group. This represents the equivalent focal length of a negative refractive index flat lens group. This represents the actual object distance from the infrared VCSEL array light source to the optical center of the negative refractive index flat lens group. The specific modes and optimal operating distance of infrared structured light are as follows: Mode: Vertical cavity surface emission laser arrays typically operate in floodlight and speckle modes, combined with a "single-sided infrared interactive sensor" for gesture detection. Here, "structured light" refers to light that encodes a light spot and specific texture to facilitate the sensor's calculation of depth and recognition of gestures. It also emphasizes that its wavelength is resistant to interference from common operating room light sources, which means that it uses narrow-band filtering technology. Optimal working distance: The system's "optimal working distance" is actually determined by the apparent distance of the real image. The decision is made by the designers working backward from the position where doctors typically stand in a surgical setting to determine the required parameters. and lens focal length Therefore, the working distance can be adjusted through optical parameter configuration, and is not a fixed value.

[0023] The interactive sensing subunit is a one-sided infrared interactive sensor that detects non-contact gestures performed by the operator within a preset spatial domain of the real-image interactive interface and generates interactive command data.

[0024] The physician's perspective tracking module includes a facial information acquisition unit, a spatial distance perception unit, and a tracking drive unit; The facial information acquisition unit is a high-resolution face tracking camera that captures the doctor's facial feature points. A specific image processing algorithm extracts the 3D coordinates of key facial feature points. This algorithm utilizes a computer vision library to locate the pixel coordinates of feature points such as the eyes, nose tip, and corners of the mouth from the 2D image captured by the camera. Based on this, the system uses a pinhole camera model to describe the mapping relationship between 3D spatial points and the 2D image plane, thereby calculating the doctor's gaze direction in real time. The specific imaging geometry is described by the following formula: ; in, This represents a non-zero scaling factor. These represent the two-dimensional pixel coordinates of the feature points on the camera image plane. This represents the internal parameter matrix of a high-resolution face tracking camera. Represents the rotation matrix. Represents the translation vector. This represents the transpose of the zero vector. This represents the absolute position coordinates of the physician's facial feature points and gaze point in a three-dimensional world coordinate system. It is a transpose operator; Using the above formula, the system can back-project the captured two-dimensional image feature points back into three-dimensional space, thereby calculating the relative positional relationship between the doctor's eyes and the orientation vector of the face, and finally obtaining the direction of the doctor's gaze point. The spatial distance sensing unit, a time-of-flight distance sensor, works in conjunction with the facial information acquisition unit to measure the real-time distance between the physician's face and the shadowless lamp head assembly. The specific distance calculation is described by the following formula: ; in, This represents the straight-line distance between the physician's face and the shadowless lamp head assembly, measured and output by the spatial distance sensing unit. This represents the speed of light in a vacuum. This represents the round-trip time delay from the transmission of an optical signal to its reception. This represents the angle between the incident ray and the normal to the surface of the shadowless lamp head assembly; Based on the above formula, the system can obtain high-precision spatial distance data between the doctor's face and the device in real time, according to the time-of-flight principle. The tracking drive unit is a servo transmission device integrated into the positioning module of the multi-joint robotic arm. It receives data generated by the facial information acquisition unit and the spatial distance perception unit, synthesizes the view tracking data, and drives the transmission device.

[0025] The multi-joint robotic arm positioning module includes a mechanical support unit, a multi-degree-of-freedom motion unit, and a pose calculation unit; The mechanical support unit is a series and parallel multi-joint robotic arm structure that physically supports the shadowless lamp head assembly, the aerial real-image interaction module, and the physician's perspective tracking module. The multi-degree-of-freedom motion unit consists of multiple servo motors and transmission devices. It receives drive signals from the tracking drive unit and realizes the flexible multi-degree-of-freedom motion of the mechanical support unit in three-dimensional space. The pose calculation unit, with a built-in inverse kinematics algorithm, receives information about the target pose from view tracking data, calculates and outputs control parameters for the joint motors in the multi-degree-of-freedom motion unit, ensuring that the real-image interactive interface remains stably within a preset optimal viewing area. This unit calculates the control parameters for each joint using the following inverse kinematics model: ; ; in, This indicates the angle that the multi-joint robotic arm needs to rotate. This represents the inverse kinematics solution function for a specific configuration design of a multi-joint robotic arm. It is a transformation matrix. It is a rotation matrix. It is a position vector. This represents the transpose of the zero vector. This is the transpose operator.

[0026] The multi-mode interaction management module includes a multi-user identification unit, a permission arbitration unit, and an interface scheduling unit; The multi-user identification unit is connected to the physician's perspective tracking module. Based on facial feature information in the perspective tracking data, it identifies and distinguishes the identities of multiple operators in the surgical area. The permission arbitration unit, connected to the multi-user identification unit, pre-stores interaction permission priority rules based on surgical roles. When the multi-user identification unit identifies multiple personnel, it arbitrates according to the rules and determines the attending physician with the highest interaction permission. The decision logic of this unit can be formalized as a priority function. When multiple surgical personnel are identified, the system prioritizes the first-level surgeon. Each person is assigned a priority score. ; in, It is based on the fixed base weights assigned by the preset surgical role. It is the output of the multi-user recognition unit based on a facial feature continuous learning model, in time. The confidence score for identification and These are the coefficients for role weight and confidence weight, respectively. Indicates time, Indicates an index variable; System Selection The user with the highest value is the primary physician with the highest interaction authority. The interface scheduling unit generates instructions based on this and activates the interface only for that user. The interface scheduling unit is connected to the permission arbitration unit and the airborne real-image interaction module. Based on the output of the permission arbitration unit, it generates interface management instructions and controls the airborne real-image interaction module to activate and update the real-image interaction interface only for the operator with the highest interaction authority.

[0027] The interactive anomaly protection module includes an environmental monitoring unit, an anomaly scenario determination unit, and an instruction circuit breaker unit; The environmental monitoring unit connects the physician's perspective tracking module and the multi-joint robotic arm positioning module to acquire real-time information on sudden personnel movement in the surgical area, the motion status of the robotic arm positioning module, and system load data. The abnormal scene determination unit has pre-set abnormal scene logic including rapid near-field intrusion, robotic arm movement exceeding limits, and system overload. It compares the data obtained by the environmental monitoring unit with the pre-set logic, and determines an abnormal scene when a match is found. The specific comprehensive determination logic is described by the following logical operation formula: ; In this formula, Indicates that the system is in The final abnormal state output at time step, Represents logical operators, Indicates an index variable. This indicates the total number of preset abnormal scenarios. Indicates the first Weighting coefficients for various abnormal scenarios Indicates an indicator function, Indicates that the environmental monitoring unit is in Real-time multidimensional state vectors collected at all times. Indicates the first The logical condition range corresponding to the various abnormal scenarios. and These represent the Boolean values ​​"true" and "false" respectively. Indicates time; Using the above formula, the system can transform complex, multi-dimensional physical state monitoring into a standardized logical judgment process. Once real-time data... Falling into any pre-defined danger zone If the indicator function outputs true, the system will immediately lock that moment as an abnormal scenario and activate the corresponding protection mechanism after the logic is processed. Specific threshold settings for determining rapid near-field intrusion: These types of thresholds are typically stored as system parameters, and the usual setting logic is as follows: Time threshold: Set an extremely short time window within which the signal changes beyond the normal value; Spatial threshold: Combining distance data from the ToF sensor, it determines whether an object suddenly appears within the robotic arm's range of motion; The instruction circuit breaker unit is connected to the abnormal scene determination unit, the multi-mode interaction management module, and the multi-joint robotic arm positioning module. When the abnormal scene determination unit outputs a determination signal, it triggers the protection mechanism to generate system behavior control instructions. The instructions include: freezing the real-image interaction interface, interrupting the current non-contact operation response, and controlling the multi-joint robotic arm positioning module to decelerate and pause movement.

[0028] The infrared VCSEL array light source in the light source generation subunit emits infrared structured light wavelengths that are configured to resist interference from common interfering light sources in the operating room and match the operating band of the single-sided infrared interactive sensor, thus maintaining stable gesture interaction detection in complex lighting environments.

[0029] The facial information acquisition unit and the spatial distance perception unit are arranged coaxially in space. The acquired facial feature point coordinates and measured distance information are fused together by a data fusion algorithm to calculate the position and orientation of the doctor's perspective in three-dimensional space and generate high-precision perspective tracking data.

[0030] The multi-user identification unit integrates a continuous learning model that learns the changes in the facial features of the identified physician during a single surgery. The permission arbitration unit is also connected to a manual priority overriding interface, which receives temporary permission allocation instructions from the surgeon and updates the current interaction permission priority according to the instructions.

[0031] The "rapid near-field intrusion" scenario preset in the abnormal scenario determination unit is defined as a person without the current interaction permission rapidly entering and blocking the common field of view of the time-of-flight distance sensor and the face tracking camera within a preset time threshold. When this scenario is determined, the system behavior control instructions generated by the instruction circuit breaker unit also include: sending a signal to the multi-mode interaction management module to temporarily lock the interface scheduling unit and prevent the switching of interaction permissions.

[0032] The operation steps of an aerial imaging shadowless lamp control system integrating an infrared VCSEL array are as follows: Step 1: Generate an interactive interface for aerial real-image display: The system first emits infrared structured light of a specific wavelength from an infrared VCSEL array light source integrated into the operating lamp head. This wavelength is specially configured to avoid interference from common ambient light in the operating room. The infrared light is received and modulated by a negative refractive index flat lens group. According to the lens imaging formula in geometric optics, the system calculates and forms a real-image interactive interface in the air at a preset position without any physical screen, which can be directly observed by the doctor with the naked eye, by precisely setting the actual object distance between the light source and the lens and using the fixed focal length of the lens. A single-sided infrared interactive sensor is deployed in the airspace near this virtual interface to detect the doctor's non-contact gesture operation in this area and convert the recognized gesture into interactive command data.

[0033] Step 2: Real-time tracking of the physician's perspective: To ensure the interactive interface always follows the doctor, the system synchronously tracks the doctor's perspective in real time. This function is accomplished by a high-resolution face tracking camera and a time-of-flight distance sensor working together. The camera captures images of the doctor's face, extracts facial feature points from the two-dimensional image using a built-in pinhole camera model and image processing algorithms, and calculates the position of the doctor's eyes in three-dimensional space and the direction vector of the gaze point. At the same time, the time-of-flight distance sensor emits modulated light pulses toward the doctor's face, and by measuring the time delay of the light pulses' round trip, it calculates the precise distance between the doctor's face and the operating light head in real time based on the principle of the constancy of the speed of light. Facial orientation and distance information are fused to generate a complete set of perspective tracking data.

[0034] Step 3: Drive the robotic arm to achieve interface tracking: After acquiring the viewpoint tracking data, the system's multi-joint robotic arm positioning module begins to work. Its internal pose calculation unit receives viewpoint data containing the target position and orientation, and expresses it as a homogeneous transformation matrix describing the target pose at the end of the robotic arm. Subsequently, the system calls the inverse kinematics algorithm designed for the robotic arm configuration to decompose and solve this target pose matrix, obtaining the specific values ​​of the angles that each joint of the robotic arm needs to rotate and move. These control parameters are sent to the multi-degree-of-freedom motion unit composed of multiple servo motors to drive the overall movement of the multi-joint robotic arm, thereby precisely adjusting the spatial position and attitude of the shadowless lamp head and its integrated aerial imaging module, ultimately achieving stable suspension of the real-image interactive interface in the air and automatic movement following the doctor's viewpoint.

[0035] Step 4: Managing multi-user interaction permissions: In scenarios involving multiple surgical personnel working together, the system initiates multi-mode interactive management. The multi-user identification unit continuously analyzes facial feature information in the perspective tracking data to distinguish and identify the identities of different personnel within the surgical area. The permission arbitration unit calculates a dynamic priority score for each identified person based on preset surgical role permission rules and recognition confidence. The system automatically selects the person with the highest priority score and arbitrates them as the "lead surgeon" currently in control. The interface scheduling unit then generates instructions based on the arbitration result, ensuring that the aerial real-image interaction interface is only activated for and responds to the gesture operations of this lead surgeon, thereby reducing control command conflicts.

[0036] Step 5: Real-time monitoring and security protection: Throughout the system's operation, the interactive anomaly protection module performs parallel safety monitoring, while the environmental monitoring unit continuously collects multi-dimensional data on personnel movement speed in the surgical area, the real-time motion status of the robotic arm, and the system's computational load. The anomaly scenario determination unit compares this real-time data with preset anomaly logic conditions such as "rapid near-field intrusion," "robotic arm movement exceeding limits," and "system overload." Once any anomaly condition is determined to be triggered, the unit immediately outputs an anomaly determination signal. Upon receiving the signal, the instruction circuit breaker unit immediately triggers the protection mechanism, executing a series of system behavior control instructions, including freezing the air interaction interface, interrupting the current gesture response, controlling the robotic arm to decelerate and pause movement, and temporarily locking the permission switching function. This measure aims to instantly place the system in a safe state, preventing any potential risks of misoperation and physical collision, and ensuring the safety of the surgical site.

[0037] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0038] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An aerial imaging shadowless lamp control system fusing infrared VCSEL arrays, characterized in that, The system includes: The aerial real-image interaction module is integrated into the lamp head assembly of the shadowless lamp. It consists of an infrared VCSEL array light source, a negative refractive index flat lens group and a single-sided infrared interactive sensor. It forms a real-image interaction interface in the air without a medium and detects non-contact operations, generating interactive command data. The physician's perspective tracking module, connected to the aerial real-image interaction module, includes a face tracking camera and a time-of-flight distance sensor, which captures the physician's facial features and distance information in the surgical area in real time and generates perspective tracking data. The multi-joint robotic arm positioning module is connected to the physician's perspective tracking module and the aerial real-image interaction module. It receives the perspective tracking data and drives the multi-joint robotic arm to move through the built-in transmission device. It adjusts the spatial pose of the shadowless lamp head assembly and the aerial real-image interaction module so that the real-image interaction interface automatically follows the physician's perspective. The multi-mode interaction management module is connected to the aerial real-image interaction module and the physician's perspective tracking module. It receives the interaction command data and the perspective tracking data, identifies the identities of multiple surgical personnel, assigns interaction permission priorities, and generates interface management commands. The interaction anomaly protection module is connected to the multi-mode interaction management module, the physician's perspective tracking module, and the multi-joint robotic arm positioning module. It monitors the environmental status of the surgical area in real time. When a preset abnormal scenario is detected, it triggers the protection mechanism and generates system behavior control instructions.

2. The over-the-air imaging shadowless lamp control system for a fused infrared VCSEL array of claim 1, wherein, The aerial real-image interaction module includes a light source generation subunit, a real-image generation subunit, and an interactive perception subunit. The light source generating subunit is an infrared VCSEL array light source that emits infrared structured light in a specific wavelength band; The real image generation subunit is a negative refractive index flat lens group that receives and modulates the infrared structured light to form a real image interactive interface that can be observed with the naked eye at a preset position in the air. This unit determines the apparent position of the real image interactive interface formed in the air based on the following optical relationship: ； in, This represents the apparent distance from the real-image interactive interface to the optical center of the negative refractive index flat lens group. This represents the equivalent focal length of the negative refractive index flat lens group. This represents the actual object distance from the infrared VCSEL array light source to the optical center of the negative refractive index flat lens group; The interactive sensing subunit is a one-sided infrared interactive sensor that detects non-contact gestures performed by the operator within a preset spatial domain of the real-image interactive interface and generates the interactive command data.

3. The overhead imaging shadowless lamp control system of claim 1, wherein, The physician's perspective tracking module includes a facial information acquisition unit, a spatial distance perception unit, and a tracking drive unit; The facial information acquisition unit is a high-resolution face tracking camera that captures the doctor's facial feature points and calculates the doctor's gaze direction in real time. The spatial distance sensing unit is a time-of-flight distance sensor that works in conjunction with the facial information acquisition unit to measure the real-time distance between the physician’s face and the shadowless lamp head assembly. The tracking drive unit is a servo transmission device integrated into the positioning module of the multi-joint robotic arm. It receives data generated by the facial information acquisition unit and the spatial distance perception unit, synthesizes the viewpoint tracking data, and drives the transmission device.

4. The over-the-air imaging shadowless lamp control system for a fused infrared VCSEL array of claim 3, wherein, The multi-joint robotic arm positioning module includes a mechanical support unit, a multi-degree-of-freedom motion unit, and a pose calculation unit. The mechanical support unit is a series and parallel multi-joint robotic arm structure that physically supports the shadowless lamp head assembly, the aerial real-image interaction module, and the physician's perspective tracking module. The multi-degree-of-freedom motion unit is composed of multiple servo motors and the transmission device. It receives the drive signal from the tracking drive unit and realizes the flexible multi-degree-of-freedom motion of the mechanical support unit in three-dimensional space. The pose calculation unit, equipped with a built-in inverse kinematics algorithm, receives information about the target pose from the view tracking data, calculates and outputs control parameters for each joint motor in the multi-degree-of-freedom motion unit, ensuring that the real-image interactive interface is stably maintained within a preset optimal viewing area. This unit calculates the control parameters for each joint using the following inverse kinematics model: ； ； in, This indicates the angle that the multi-joint robotic arm needs to rotate. This represents the inverse kinematics solution function designed for the specific configuration of the multi-joint robotic arm. It is a transformation matrix. It is a rotation matrix. It is a position vector. This represents the transpose of the zero vector. This is the transpose operator.

5. The aerial imaging shadowless lamp control system integrating an infrared VCSEL array according to claim 1, characterized in that, The multi-mode interaction management module includes a multi-user identification unit, a permission arbitration unit, and an interface scheduling unit; The multi-user identification unit is connected to the physician's perspective tracking module, and identifies and distinguishes the identities of multiple operators in the surgical area based on the facial feature information in the perspective tracking data. The permission arbitration unit is connected to the multi-user identification unit and has pre-stored interaction permission priority rules based on surgical roles. When the multi-user identification unit identifies multiple people, it arbitrates and determines the attending physician with the highest interaction permission according to the rules. The interface scheduling unit is connected to the permission arbitration unit and the aerial real-image interaction module. Based on the output result of the permission arbitration unit, it generates the interface management instruction and controls the aerial real-image interaction module to activate and update the real-image interaction interface only for the operator with the highest interaction authority.

6. The over-the-air imaging shadowless lamp control system for a fused infrared VCSEL array of claim 5, wherein, The interaction anomaly protection module includes an environmental monitoring unit, an anomaly scene determination unit, and an instruction circuit breaker unit. The environmental monitoring unit is connected to the physician's perspective tracking module and the multi-joint robotic arm positioning module to acquire real-time information on sudden movement of personnel in the surgical area, the motion status of the robotic arm positioning module, and system load data. The abnormal scene determination unit has pre-set abnormal scene logic including rapid near-field intrusion, robotic arm movement exceeding limits, and system overload. It compares the data obtained by the environmental monitoring unit with the pre-set logic, and determines it as an abnormal scene when the match is successful. The instruction circuit breaker unit is connected to the abnormal scene determination unit, the multi-mode interaction management module, and the multi-joint robotic arm positioning module. When the abnormal scene determination unit outputs a determination signal, it triggers a protection mechanism to generate the system behavior control instruction. The instruction includes: freezing the real-image interaction interface, interrupting the current non-contact operation response, and controlling the multi-joint robotic arm positioning module to decelerate and pause movement.

7. The overhead imaging shadowless lamp control system of claim 2, wherein, The infrared VCSEL array light source in the light source generation subunit emits infrared structured light with a wavelength range configured to counteract common interference light sources in operating rooms and to match the operating band of the single-sided infrared interactive sensor, thus maintaining stable gesture interaction detection in complex lighting environments.

8. The overhead imaging shadowless lamp control system of claim 3, wherein, The facial information acquisition unit and the spatial distance sensing unit are arranged coaxially in space. The acquired facial feature point coordinates and the measured distance information are fused together by a data fusion algorithm to generate high-precision view tracking data.

9. The overhead imaging shadowless lamp control system of claim 5, wherein, The multi-user identification unit integrates a continuous learning model, which learns the changes in the facial features of the identified physician during a single surgery. The permission arbitration unit is also connected to a manual priority overlay interface, which receives temporary permission allocation instructions from the surgeon and updates the current interaction permission priority according to the instructions.

10. The overhead imaging shadowless lamp control system for fusion infrared VCSEL array according to claim 6, characterized in that, The "rapid near-field intrusion" scenario preset in the abnormal scenario determination unit is defined as a person without the current interaction permission rapidly entering and blocking the common field of view of the time-of-flight distance sensor and the face tracking camera within a preset time threshold. When this scenario is determined, the system behavior control instruction generated by the instruction circuit breaker unit also includes: sending a signal to the multi-mode interaction management module to temporarily lock the interface scheduling unit.

Images