An ultrasonic array haptic feedback device and method for non-destructive testing in a nuclear environment

Abstract

This invention discloses an ultrasonic array tactile feedback device and method for non-destructive testing in nuclear environments, relating to the field of remote operation and non-destructive testing technology in the nuclear industry. The device includes a visual positioning module, a remote tooling status module, a main control computing unit, a gate driving unit, and an ultrasonic transducer array. The visual positioning module captures the operator's hand movements and calculates the fingertip coordinates; the remote tooling status module acquires the probe echo signal, calculates the coupling efficiency value, and transmits it back; the main control computing unit has a built-in adaptive dual-modal sound field reconstruction engine, which automatically switches between real-time parameterized generation mode and static holographic reconstruction mode according to the current interaction scenario; the gate driving unit performs dual-buffered time-division multiplexing control to apply low-frequency modulation to the ultrasonic carrier and convert it into a high-voltage driving signal; the ultrasonic transducer array superimposes a tactile sound field with distinguishable shapes in the air, allowing the operator to identify the probe's contact status through fingertip touch, improving the accuracy and safety of non-destructive testing.

Description

Technical Field

[0001] This invention relates to the field of remote operation and non-destructive testing technology in the nuclear industry, specifically an ultrasonic array tactile feedback device and method for non-destructive testing in nuclear environments. Background Technology

[0002] The long-term safe operation of nuclear energy facilities depends on regular non-destructive testing of core components such as reactor pressure vessels and primary circuit piping. Because these devices are exposed to high-dose radiation environments for extended periods, the testing work cannot be carried out by personnel directly entering the site. Instead, a master-slave remote-operated robotic system must be used, where an operator located in a safe area remotely controls a robotic arm carrying an ultrasonic probe to scan weld seams through a monitoring screen.

[0003] However, existing remote-operated inspection systems in the nuclear industry face a significant lack of tactile information. Operators at remote control consoles can only observe the robotic arm's posture and the A-scan or B-scan waveforms of the ultrasonic instrument through a two-dimensional monitoring screen. This singular visual feedback results in a lack of intuitive perception of the contact status between the remote probe and the workpiece surface. Especially in inspection scenarios involving complex structures such as small-diameter pipes and curved welds, crucial information such as whether the probe perfectly fits the workpiece surface and whether there are any slight warping or tilting cannot be accurately determined visually. Operators often need to repeatedly adjust the robotic arm's posture and observe subtle changes in the ultrasonic echo waveform to barely infer the probe's contact status. This haphazard, "blind man and elephant" approach is not only inefficient but also highly susceptible to data corruption or missed detections due to poor coupling.

[0004] Meanwhile, when operators control the robotic arm to move the probe, they cannot sense changes in the reaction force or friction of the workpiece surface on the probe. Unlike handheld inspection, they cannot use touch to determine whether the probe is in place or whether the clamping force is appropriate. This lack of tactile feedback forces operators to frequently pause and repeatedly confirm, severely impacting inspection efficiency and operational continuity. Although existing technologies attempt to provide tactile guidance through wearable force feedback gloves or force feedback master hands, wearable devices are prone to causing operator fatigue during prolonged operation, and high-precision force feedback devices are expensive and complex to maintain, making widespread application in nuclear industry environments difficult. Summary of the Invention

[0005] The purpose of this invention is to provide an ultrasonic array tactile feedback device and method for non-destructive testing in nuclear environments, in order to solve the above-mentioned problems.

[0006] The technical solution of this invention is: An ultrasonic array tactile feedback device for non-destructive testing in nuclear environments includes: a visual positioning module, located on a remote operating console, configured to acquire depth images of the operator's hand in real time and calculate the three-dimensional spatial coordinates of the fingertips, outputting target tactile feedback point position data; a remote tooling status module, located at the end of a remote inspection robot and connected to an ultrasonic probe, configured to acquire probe echo signals in real time, calculate and transmit the coupling efficiency value between the probe and the surface of the workpiece under test; and a main control computing unit, connected to both the visual positioning module and the remote tooling status module, internally configured with an adaptive dual-modal sound field reconstruction engine, the adaptive dual-modal sound field reconstruction engine being configured to: activate a real-time parameterized generation mode when the fingertip movement speed is greater than a set threshold, and use an iterative optimization algorithm to solve for the focused phase hologram in real time; and activate a mode when the fingertip movement speed is less than a set threshold, ... When a threshold is set and valid coupling efficiency data is received, the static holographic reconstruction mode is activated, and the corresponding static phase hologram is retrieved from the pre-stored database according to the coupling efficiency value; the engine outputs the focused phase hologram or static phase hologram as the driving phase hologram of the ultrasonic transducer array; the gate driving unit, connected to the main control computing unit, is configured to receive the driving phase hologram, execute double-buffered time-division multiplexing control logic, convert low-voltage logic signals into multiple high-voltage driving signals, and apply a low-frequency amplitude modulation envelope to the ultrasonic carrier; the ultrasonic transducer array includes multiple piezoelectric ceramic transducer elements arranged in a predetermined topology, connected to the gate driving unit, and radiates and superimposes a tactile feedback sound field in the air in response to the high-voltage driving signal, the tactile feedback sound field having a distinguishable tactile primitive shape.

[0007] Furthermore, the pre-stored static phase hologram in the static holographic reconstruction mode is pre-calculated and generated by a weighted GS iterative algorithm combined with a sub-pixel physical constraint mechanism. The weighted GS iterative algorithm is configured as follows: Define a low-resolution grid for the physical array plane and a high-resolution simulation grid for the target sound field plane, wherein the spatial sampling density of the high-resolution simulation grid is higher than that of the low-resolution grid; The source plane complex amplitude is propagated to the target plane in the forward direction using the angular spectrum propagation operator to obtain the target complex amplitude. The phase is kept unchanged and the amplitude is replaced with the preset target tactile pattern amplitude to obtain the update field. The updated field is propagated back to the source plane to obtain the updated source plane complex amplitude, and a phase unification constraint is applied to multiple sub-pixels corresponding to the same physical transducer aperture region, that is, the phase of the multiple sub-pixels is forcibly unified to the phase angle of the complex amplitude vector sum of the region. Repeat the above forward propagation, amplitude constraint, backward propagation and sub-pixel constraint steps until convergence, and extract the final phase matrix as the static phase hologram.

[0008] Furthermore, the visual positioning module also includes: a hand-eye transformation matrix storage unit, configured to store a pre-calibrated hand-eye transformation matrix for mapping the observation coordinates in the camera coordinate system to the target coordinates in the acoustic array coordinate system; and an extended Kalman filter prediction unit, connected to the hand-eye transformation matrix storage unit, configured to establish a state vector including fingertip position and velocity. The state vector is represented as: The system uses a state transition matrix to predict the prior position of the fingertip when the sound wave is emitted at the next moment; the coordinate output unit is connected to the extended Kalman filter prediction unit and is configured to send the predicted coordinates as the target tactile feedback point position data to the main control computing unit to offset the total transmission delay of the system from visual recognition to acoustic excitation.

[0009] Furthermore, the tactile primitive shape includes: a first shape, corresponding to a first coupling efficiency range, used to indicate a probe suspended or severely tilted state; a second shape, corresponding to a second coupling efficiency range, used to indicate a probe in contact but not flat; and a third shape, corresponding to a third coupling efficiency range, used to indicate a probe in perfect contact state; wherein the first shape, second shape, and third shape have different spatial sound field distributions, used to prompt the operator to identify the coupling state of the remote probe through fingertip tactile sensation.

[0010] Furthermore, the gate driving unit includes: a dual-buffered memory architecture comprising a first buffer and a second buffer, wherein the first buffer is configured to store a focused phase hologram representing a tactile excitation state, and the second buffer is configured to store a defocused or zero-amplitude hologram representing a tactile silence state; a hardware timer configured to generate a periodic switching signal at a frequency sensitive to human skin touch; and a switching circuit connected to and communicating with the hardware timer and the dual-buffered memory architecture, configured to alternately select the read address of the first buffer or the second buffer according to the periodic switching signal, thereby causing the output data to cycle between the first buffer and the second buffer, so that the output sound field alternately jumps between the excitation state and the silence state, applying the low-frequency amplitude modulation envelope to the ultrasonic carrier, and generating a vibrational tactile sensation at the modulation frequency on the surface of human skin.

[0011] Furthermore, the gate driving unit also includes a push-pull power amplifier circuit, configured to split each logic signal into a positive-phase driving signal and an inverted-phase driving signal, which respectively drive the power MOSFETs connected to both ends of the ultrasonic transducer, thereby forming a peak voltage difference twice the power supply voltage across the transducer to enhance the acoustic radiation power.

[0012] Furthermore, the gate driving unit also includes a distributed I / O expansion architecture, which includes: multiple shift register chips connected to the main control FPGA via a high-speed SPI bus to form a distributed I / O expansion architecture; and a global latch signal line connected to all the shift register chips, configured to trigger parallel output simultaneously after all register chips complete the reception of a frame of data, ensuring that all transducers update phase at the same time. The ultrasonic transducer array adopts a scalable board-level cascaded architecture, with multiple gate drive unit boards interconnected in a daisy chain through high-speed inter-board connectors, sharing the same global latch signal line to eliminate inter-board phase jitter.

[0013] A method for generating tactile feedback using an ultrasonic array for non-destructive testing in a nuclear environment, applied to the aforementioned device, includes the following steps: The visual positioning module acquires depth images of the operator's hand in real time, calculates the three-dimensional spatial coordinates of the fingertips, and outputs the target tactile feedback point position data; The remote tooling status module collects the echo signal of the remote ultrasonic probe in real time, calculates the coupling efficiency value between the probe and the surface of the workpiece under test, and transmits it back. The main control computing unit switches strategies based on the received state of fingertip movement speed and coupling efficiency value: When the fingertip movement speed exceeds the set threshold, the real-time parameterized generation mode is activated, and the iterative optimization algorithm is used to solve the focused phase hologram in real time. When the fingertip movement speed is lower than a set threshold and valid coupling efficiency data is received, the static holographic reconstruction mode is activated, and the corresponding static phase hologram is retrieved from the pre-stored database according to the coupling efficiency value. The focused phase hologram or static phase hologram is output as the driving phase hologram of the ultrasonic transducer array; The driving phase hologram is subjected to double-buffered time-division multiplexing control logic. Low-frequency amplitude modulation envelope is applied to the ultrasonic carrier by cyclically switching between the first buffer and the second buffer, and the low-voltage logic signal is converted into multiple high-voltage driving signals. The high-voltage driving signal is applied to multiple piezoelectric ceramic transducer array elements arranged in a predetermined topology, which radiate in the air and superimpose to form a tactile feedback sound field with distinguishable tactile primitive shapes.

[0014] Furthermore, the static phase hologram is pre-calculated and generated by the following steps: Define a low-resolution grid for the physical array plane and a high-resolution simulation grid for the target sound field plane, wherein the spatial sampling density of the high-resolution simulation grid is higher than that of the low-resolution grid; The source plane complex amplitude is propagated to the target plane in the forward direction using the angular spectrum propagation operator to obtain the target complex amplitude. The phase is kept unchanged and the amplitude is replaced with the preset target tactile pattern amplitude to obtain the update field. The updated field is propagated back to the source plane to obtain the updated source plane complex amplitude, and a phase unification constraint is applied to multiple sub-pixels corresponding to the same physical transducer aperture region, that is, the phase of the multiple sub-pixels is forcibly unified to the phase angle of the complex amplitude vector sum of the region. Repeat the above forward propagation, amplitude constraint, backward propagation and sub-pixel constraint steps until convergence, and extract the final phase matrix as the static phase hologram.

[0015] Furthermore, the visual positioning step also includes: The pre-calibrated hand-eye transformation matrix is ​​stored to map the observation coordinates in the camera coordinate system to the target coordinates in the acoustic array coordinate system; Establish a state vector containing fingertip position and velocity. The state vector is represented as: .

[0016] Using extended Kalman filtering and a state transition matrix, the prior position of the fingertip at the next moment when the sound wave is emitted is predicted based on the state vector, and used as the predicted coordinates. The predicted coordinates are used as the target tactile feedback point location data to offset the total transmission delay from visual recognition to acoustic excitation in the system.

[0017] Compared with the prior art, the beneficial effects of the present invention are: This invention uses a visual positioning module to capture the operator's hand movements in real time and map them into remote robot control commands, achieving intuitive remote operation mapping; the remote tooling status module synchronously collects the echo signal of the ultrasonic probe, calculates the coupling efficiency value reflecting the probe's fit, and transmits it back in real time. The adaptive dual-modal acoustic field reconstruction engine inside the main control computing unit intelligently switches working modes according to the operation scenario: when the hand moves quickly to carry the probe, the real-time parameterized generation mode is activated to quickly solve the focused phase hologram, ensuring that the tactile feedback closely follows the fingertip movement; when the hand slows down to perform fine scanning and receives effective coupling efficiency data, it switches to static holographic reconstruction mode, and retrieves the tactile primitives corresponding to different contact states according to the coupling efficiency value, so that the operator can intuitively judge the status of the remote probe by the touch of the fingertips. The gate drive unit cycles between the excited state and the silent state through dual-buffered time-division multiplexing control, applies the low-frequency modulation most sensitive to the human body to the ultrasonic carrier, and converts the weak continuous sound field into a clear and distinguishable vibration tactile sensation, so that the operator can perceive the contact status of the remote probe in real time by the touch of the fingertips in the safe area, which greatly reduces the cognitive load and the risk of misoperation, while avoiding wearing constraints, making it suitable for long-term operation in the nuclear industry, and significantly improving the operation accuracy, efficiency and safety of remote operation non-destructive testing in nuclear radiation environment. Attached Figure Description

[0018] Figure 1 This is a functional block diagram of the present invention.

[0019] Figure 2 This is a structural schematic diagram for a specific application scenario.

[0020] The components include: 1. Remote control console; 2. Visual synchronization display; 3. Ultrasonic transducer array; 4. Aerial tactile interaction platform; and 5. Depth camera. Detailed Implementation

[0021] The following is combined Figures 1 to 2 The specific embodiments of the present invention will be described in detail below.

[0022] Example like Figure 2 As shown, an ultrasonic array tactile feedback device for non-destructive testing in a nuclear environment includes: a visual positioning module, a remote tooling status module, a main control computing unit, a gate driving unit, and an ultrasonic transducer array.

[0023] The visual positioning module is located in the human-computer interaction area of ​​the remote control console. It is configured to acquire real-time depth images of the operator's hand, calculate the 3D spatial coordinates of the fingertips, and output the target tactile feedback point location data. Specifically, depth camera 5 acquires real-time depth images of the operator's hand. Depth camera 5 is positioned below the aerial tactile interaction platform 4 for easy recording of the hand's state. During the initialization phase, the transducer array and the 3D coordinate system of the visual positioning module are physically mapped. The skeletal keypoint recognition algorithm built into the visual positioning module extracts the 3D spatial coordinates of the index fingertip from the depth image. To achieve precise alignment between visual and acoustic coordinates, the module stores a pre-calibrated hand-eye transformation matrix. It is used to map the observation coordinates in the camera coordinate system to the target coordinates in the acoustic array coordinate system. It outputs the target haptic feedback point location data, including timestamps;

[0024] The visual positioning module includes a visual synchronization display 2 and a depth camera 5. The skeletal keypoint recognition algorithm is configured within the visual positioning module itself. The visual synchronization display 2, ultrasonic transducer array 3, and depth camera 5 are installed on the external operating table 1 of the detection chamber in the simulated nuclear radiation environment; according to... Figure 1 The diagram shows the functional block diagram of the ultrasonic array tactile feedback device for non-destructive testing in nuclear environments, realizing specific scenarios of the aerial tactile interaction platform 4, including manual movement scenarios, and coupling discrimination elements (conical, rectangular, spherical) of probe and weld fit.

[0025] Depth camera 5 is used to check the hand's status and provide its spatial coordinates. Depth camera 5 is connected to the industrial control software system. The mapping and alignment are then performed by the industrial control software system within the entire core environment. It's worth noting that the industrial control software system is an existing system. The system further transmits the indoor robotic arm's viewpoint back to the visual synchronization display 2, enabling the indoor robotic arm to move accurately when the human hand is operated and moved on the aerial tactile interaction platform 4.

[0026] The visual synchronization display 2 serves more than just the functions shown above; it can also be considered the default front-end interface of the industrial control software system, similar to a car infotainment system display. Since the current task is remote operation within a kernel environment, upon entering this task, it will begin initializing the camera and performing a series of other operations. The industrial control software system is an existing system, and this embodiment will not elaborate further on it.

[0027] The remote tooling status module is set at the end of the remote inspection robot and connected to the ultrasonic probe. It is configured to collect the probe echo signal in real time, calculate and transmit the coupling efficiency value between the probe and the surface of the workpiece to be tested. The remote tooling status module has a built-in FPGA signal preprocessing unit and executes the coupling efficiency calculation process; specifically, the module pre-stores the reference echo amplitude values ​​measured on the standard test block. and minimum environmental noise threshold Based on the wall thickness and sound velocity of the workpiece under test (such as a small-diameter pipe), the module dynamically sets the detection gate on the time axis of the ultrasonic A-scan signal. (The time window in which the bottom echo is expected to appear). Real-time capture of the maximum peak value of the echo signal within the gate. According to the formula Solve for the current state coupling efficiency; and use the calculated results The value is encapsulated into the data packet.

[0028] To overcome the multipath effect of metal structures within a nuclear radiation environment, the module employs the UWB (Ultra-Wideband) industrial wireless protocol to calculate... The value is quantized into an 8-bit or 16-bit integer format, and then transmitted in nanosecond-level narrow pulses. The values ​​are transmitted back to the main control computing unit of the external console with low latency, so that the console main control can further generate sound field primitives with differentiated spatial shapes.

[0029] The main control computing unit is connected to both the visual positioning module and the remote tooling status module. It is internally equipped with an adaptive dual-modal sound field reconstruction engine. The main control computing unit receives the target coordinates predicted by the visual positioning module. Next, the interaction intent is determined to decide which sound field generation strategy to use and to define the fingertip movement speed. , The threshold-based determinations correspond to both real-time and static operation scenarios.

[0030] The adaptive dual-modal sound field reconstruction engine is configured as follows: when the fingertip movement speed is greater than a set threshold, the real-time parameterized generation mode is activated, and the focused phase hologram is solved in real time using an iterative optimization algorithm; when the fingertip movement speed is lower than the set threshold and valid coupling efficiency data is received, the static holographic reconstruction mode is activated, and the corresponding static phase hologram is retrieved from the pre-stored database according to the coupling efficiency value; the engine outputs either a focused phase hologram or a static phase hologram as the driving phase hologram of the ultrasonic transducer array 3. ; The main control computing unit is integrated into the industrial control software system. As long as the task described in this embodiment is being performed, the industrial control software system will automatically call the two algorithms mentioned above, and determine whether to switch to the corresponding algorithm based on the state of the human hand acquired by the depth camera 5.

[0031] The gate driving unit is connected to the main control computing unit and is configured to receive and drive the phase hologram, execute double-buffered time-division multiplexing control logic, convert low-voltage logic signals into multiple high-voltage driving signals, and apply a low-frequency amplitude modulation envelope with a frequency of to the 40kHz ultrasonic carrier. The ultrasonic transducer array 3 is the final propagation carrier of the industrial control software system. It includes multiple piezoelectric ceramic transducer elements arranged in a predetermined topology and connected to the gate driving unit. It responds to the high voltage driving signal, radiates in the air, and superimposes to generate a tactile feedback sound field. The tactile feedback sound field has a distinguishable tactile primitive shape.

[0032] like Figure 2 As shown, the predetermined topology is: the ultrasonic transducers are arranged in a 16×16 array.

[0033] The ultrasonic transducer array 3 is responsible for emitting ultrasonic waves. Its own state is fixed. It is responsible for receiving 16×16 signals sent to it by the gate drive unit.

[0034] Real-time parametric generation mode in the adaptive bimodal sound field reconstruction engine: When detected When the sound pressure level exceeds a set threshold and the system is in dynamic tracking mode, the main control computing unit invokes the real-time parameterized generation mode in the built-in adaptive dual-modal sound field reconstruction engine. This mode is suitable for single-point following or standard geometric trap generation; it employs the BFGS quasi-Newton optimization algorithm based on the Java environment, using the maximization of the sound pressure at the target point as the objective function for iterative solution, and reads the converged phase solution of the previous time frame as the initial value for the current step to reduce the number of iterations and ensure phase continuity.

[0035] Specifically, the BFGS algorithm is constructed based on The non-convex optimization function that maximizes the sound pressure amplitude at the target point is defined as the phase vector to be optimized. Let be the phase vector to be optimized, and construct the objective function. The negative square of the sound pressure amplitude at the target focal point is: .

[0036] in, The total number of transducers. For the first The emission amplitude of each transducer For wave number, For the first The distance from each transducer to the target focus is then calculated; subsequently, the objective function is calculated with respect to each phase variable. analytic gradient vector Its component form is: .

[0037] Iteratively update the inverse matrix approximation of the Hessian matrix using the BFGS formula. ,according to Determine the search direction and the step size through line search. Update phase Until convergence.

[0038] The real-time parametric generation mode is also equipped with special acoustic trap generation logic based on analytical formulas, used for BFGS-optimized focusing phase. To create a differentiated tactile feel, topological phase factors are superimposed on topological phase factors, such as superimposing spiral phase terms. Generate vortex traps carrying orbital angular momentum, where The topological load is an integer. The transducer azimuth angle is used to generate a rotational shearing tactile sensation; the formula is: .

[0039] or superimposed The phase jump term generates a twin trap for a two-lobed sound field, used to produce a directional gripping sensation. The formula is: .

[0040] in It is a step function. For dividing the plane normal vector.

[0041] In the static holographic reconstruction mode of the adaptive dual-modal sound field reconstruction engine, the pre-stored static phase hologram is pre-calculated and generated by the weighted GS iterative algorithm combined with the sub-pixel physical constraint mechanism. The weighted GS iterative algorithm is configured as follows: When detected Below the threshold and receiving valid coupling efficiency data When the system determines that it has entered the fine inspection stage of the weld, the main control computing unit switches to the static holographic reconstruction mode, judges and classifies the coupling efficiency of different intervals, and generates corresponding complex custom sound field patterns or specific texture surfaces. This strategy calls the weighted GS iterative algorithm based on the MATLAB environment, combined with the sub-pixel physical constraint mechanism, to pre-calculate and store high-quality static phase holograms on the super-resolution simulation grid, and directly calls them through table lookup to generate complex sound field primitives.

[0042] Specifically, the weighted GS iterative algorithm in the static holographic reconstruction mode is configured to perform the following computational process: defining a low-resolution mesh for the physical array plane. High-resolution simulation mesh with the target sound field plane ,in Spatial sampling density is of times ( Using the angular spectrum propagation operator to measure the complex amplitude of the source plane Propagation forward to the target plane yields Maintaining the phase unchanged and replacing the amplitude with the preset target tactile pattern amplitude. Later, an updated field was obtained. ;Will Backpropagation back to the source plane yields and to Perform sub-pixel physical constraint operations, that is, operations corresponding to the same physical transducer aperture region... The phase of each sub-pixel is forcibly unified to the phase angle of the sum of the complex amplitude vectors in that region. This generates a high-fidelity hologram, using the following formula: .

[0043] Repeat the above forward propagation, amplitude constraint, backward propagation, and sub-pixel constraint steps until convergence, and extract the final result. The matrix serves as the driving phase hologram for the physical array.

[0044] In some embodiments, the visual positioning module further includes: a hand-eye transformation matrix storage unit, an extended Kalman filter prediction unit, and a coordinate output unit.

[0045] The hand-eye transformation matrix storage unit is configured to store a pre-calibrated hand-eye transformation matrix, which is used to map the observation coordinates in the camera coordinate system to the target coordinates in the acoustic array coordinate system. The extended Kalman filter prediction unit is connected to the hand-eye transformation matrix storage unit and configured to establish a state vector containing fingertip position and velocity. The state vector is represented as: And the state transition matrix is ​​used to predict the prior position of the fingertip when the sound wave is emitted at the next moment; where, x , y and z This represents the position coordinates of the fingertip in three-dimensional space. v x , v y and v z Indicates fingertips at x , y and z The velocity components in three directions.

[0046] The coordinate output unit is connected to the extended Kalman filter prediction unit and is configured to send the predicted coordinates as the target tactile feedback point location data to the main control computing unit to offset the total transmission delay of the system from visual recognition to acoustic excitation.

[0047] In some embodiments, the main control computing unit bases its decisions on the returned coupling efficiency data. It calls high-fidelity sound field primitives pre-computed based on the weighted GS iterative algorithm to realize the tactile characteristics of the objective state, which are defined as three states: Decoupling state The system retrieves and generates spherical sound field primitives, providing intuitive prompts to the operator that the probe is suspended or too far away, and that further adjustments should be made. Adjustment state The system retrieves and generates rectangular sound field primitives, indicating to the operator that the primitives have been touched but not flattened, and that the angle still needs to be finely adjusted. Locked state The system retrieves and generates cone-shaped sound field primitives as an objective confirmation signal of a relatively perfect fit, informing the operator to maintain the current posture for detection.

[0048] The shapes of tactile primitives include a first shape, a second shape, and a third shape, which correspond to decoupled states, respectively. Adjustment state and locked state .

[0049] In some embodiments, the gate drive unit includes: a dual-buffered memory architecture, a hardware timer, a switching circuit, a push-pull power amplifier circuit, and a distributed I / O expansion architecture.

[0050] The dual-buffered memory architecture includes a first buffer and a second buffer. The first buffer is configured to store a focused phase hologram representing the tactile excitation state, and the second buffer is configured to store a defocused or zero-amplitude hologram representing the tactile quiescent state. The hardware timer is configured to generate a periodic switching signal based on the frequency sensitive to human skin touch. The switching circuit is connected and communicates with the hardware timer and dual-buffer memory architecture. It is configured to alternately select the read address of the first buffer or the second buffer according to the periodic switching signal, so that the output data is cyclically switched between the first buffer and the second buffer, so that the output sound field alternately jumps between the excited state and the silent state, and applies a low-frequency amplitude modulation envelope to the ultrasonic carrier to generate a vibration sensation at the modulation frequency on the surface of human skin.

[0051] The push-pull power amplifier circuit is configured to split each logic signal into a positive-phase drive signal and an inverted-phase drive signal, which drive the power MOSFETs connected to both ends of the ultrasonic transducer respectively, forming a peak voltage difference twice the power supply voltage across the transducer to enhance the acoustic radiation power.

[0052] The distributed I / O expansion architecture includes: multiple shift register chips and a global latch signal line. The multiple shift register chips are connected to the main control FPGA via a high-speed SPI bus to form a distributed I / O expansion architecture. The global latch signal line is connected to all shift register chips and is configured to trigger parallel output simultaneously after all register chips have completed receiving a frame of data, ensuring that all transducers update their phase at the same time. The ultrasonic transducer array 3 adopts a scalable board-level cascaded architecture. Multiple gate drive unit boards are interconnected in a daisy chain through high-speed inter-board connectors, sharing the same global latch signal line and eliminating inter-board phase jitter.

[0053] Specifically, the phase discretization and hardware calibration steps are as follows: the main control computing unit reads the transducer phase calibration table stored in the non-volatile memory and compensates for the inherent phase deviation of each channel caused by manufacturing tolerances. and obtain the first The inherent phase deviation of each channel due to manufacturing tolerances Calculate the corrected phase Based on the clock division accuracy of the gate drive unit The continuous phase quantity is converted using the following formula. Level Discrete Integer Index : .

[0054] For high-channel fan-out operation, the FPGA sends the calculated phase hologram data as a serial bit stream to the distributed register network via a high-speed SPI bus. After a frame of data transmission is completed, a global latch signal is sent to cause all register chips to output logic control signals in parallel. Each logic signal output by the registers is split into positive and negative drive signals by logic gates, which respectively drive the push-pull power amplifier circuits connected to both ends of the ultrasonic transducer. This creates a peak voltage difference across the transducer that is twice the power supply voltage. This significantly enhances the sound radiation power.

[0055] Simultaneously, the processed focused phase hologram is written to the FPGA's internal buffer, initiating dual-buffered time-division multiplexing logic to divide the Block RAM into a first buffer (Buffer A) and a second buffer (Buffer B). The main control computing unit writes the focused phase hologram representing the tactile excitation state to the first buffer and the defocused or zero-amplitude hologram representing the tactile silence state to the second buffer. The FPGA utilizes an internal hardware timer with a user-configurable modulation period. (Typical value is 5ms, corresponding to 200Hz tactile sensitive frequency) Generates a switching trigger signal, automatically switches between reading data from the first buffer and the second buffer and drives the external register to update the output, so that the output sound field can alternate between focused and defocused states, producing a strong texture vibration on the surface of human skin.

[0056] In some embodiments, the single plates of the ultrasonic transducer array 3 are tiled. To accommodate the potential future expansion of the ultrasonic array's channel count, a scalable board-level cascade architecture was designed for the transducer structure. Multiple gate drive unit boards are interconnected in a daisy chain via high-speed inter-board connectors, enabling cascaded pass-through in the shift register chains of each board. This ensures that transducers distributed on different boards update their phase at the same microsecond, eliminating inter-board phase jitter.

[0057] Furthermore, the communication protocol stack between the main control computing unit and the gate drive unit also includes an adaptive bandwidth compression mechanism. Before sending the next frame of data, the system performs differential detection. If it detects that the amplitude of all transducers remains at full power while only the phase changes, it automatically constructs a simplified data packet containing a phase-only update instruction header, thereby increasing the communication bandwidth utilization to 200%.

[0058] like Figure 1 As shown, an ultrasonic array tactile feedback generation method for non-destructive testing in nuclear environments, applied to the aforementioned device, includes the following steps: The visual positioning module acquires depth images of the operator's hand in real time, calculates the three-dimensional spatial coordinates of the fingertips, and outputs the target tactile feedback point position data; The remote tooling status module collects the echo signal of the remote ultrasonic probe in real time, calculates the coupling efficiency value between the probe and the surface of the workpiece under test, and transmits it back. The main control computing unit switches strategies based on the received state of fingertip movement speed and coupling efficiency value: When the fingertip movement speed exceeds the set threshold, the real-time parameterized generation mode is activated, and the iterative optimization algorithm is used to solve the focused phase hologram in real time. When the fingertip movement speed is lower than the set threshold and valid coupling efficiency data is received, the static holographic reconstruction mode is activated, and the corresponding static phase hologram is retrieved from the pre-stored database according to the coupling efficiency value. Output a focused phase hologram or a static phase hologram as the driving phase hologram for the ultrasonic transducer array 3; The driving phase hologram is subjected to double-buffered time-division multiplexing control logic. The low-frequency amplitude modulation envelope is applied to the ultrasonic carrier by cyclically switching between the first buffer and the second buffer, and the low-voltage logic signal is converted into multiple high-voltage driving signals. A high-voltage driving signal is applied to multiple piezoelectric ceramic transducer array elements arranged in a predetermined topology, which radiate in the air and superimpose to form a tactile feedback sound field with distinguishable tactile primitive shapes.

[0059] The static phase hologram is pre-calculated and generated by the following steps: Define a low-resolution mesh for the physical array plane and a high-resolution simulation mesh for the target sound field plane, wherein the spatial sampling density of the high-resolution simulation mesh is higher than that of the low-resolution mesh; The source plane complex amplitude is propagated to the target plane in the forward direction using the angular spectrum propagation operator to obtain the target complex amplitude. The phase is kept unchanged and the amplitude is replaced with the preset target tactile pattern amplitude to obtain the update field. The updated field is propagated back to the source plane to obtain the updated source plane complex amplitude, and a phase unification constraint is applied to multiple sub-pixels corresponding to the same physical transducer aperture region, that is, the phase of multiple sub-pixels is forced to be unified to the phase angle of the complex amplitude vector sum of the region. Repeat the above forward propagation, amplitude constraint, backward propagation and sub-pixel constraint steps until convergence, and extract the final phase matrix as a static phase hologram.

[0060] The visual localization steps also include: The pre-calibrated hand-eye transformation matrix is ​​stored to map the observation coordinates in the camera coordinate system to the target coordinates in the acoustic array coordinate system; Establish a state vector containing fingertip position and velocity. The state vector is represented as: .

[0061] Using extended Kalman filtering and state transition matrix, the prior position of the fingertip at the next moment of sound wave emission is predicted based on the state vector, and used as the prediction coordinate; The predicted coordinates are used as the target tactile feedback point location data to offset the total transmission delay from visual recognition to acoustic excitation.

[0062] In this embodiment, the visual positioning module captures the operator's hand movements in real time and maps them to remote robot control commands, realizing intuitive remote operation mapping; the remote tooling status module synchronously collects the ultrasonic probe echo signal, calculates the coupling efficiency value reflecting the probe's fit, and transmits it back in real time. The adaptive dual-modal acoustic field reconstruction engine inside the main control computing unit intelligently switches working modes according to the operation scenario: when the hand moves quickly to carry the probe, the real-time parameterized generation mode is activated to quickly solve the focused phase hologram, ensuring that the tactile feedback closely follows the fingertip movement; when the hand slows down to perform fine scanning and receives effective coupling efficiency data, it switches to static holographic reconstruction mode, and retrieves the tactile primitives corresponding to different contact states according to the coupling efficiency value, so that the operator can intuitively judge the status of the remote probe by the touch of the fingertips. The gate drive unit cycles between the excited state and the silent state through dual-buffered time-division multiplexing control, applies the low-frequency modulation most sensitive to the human body to the ultrasonic carrier, and converts the weak continuous sound field into a clear and distinguishable vibration tactile sensation, so that the operator can perceive the contact status of the remote probe in real time by the touch of the fingertips in the safe area, which greatly reduces the cognitive load and the risk of misoperation, while avoiding wearing constraints, making it suitable for long-term operation in the nuclear industry, and significantly improving the operation accuracy, efficiency and safety of remote operation non-destructive testing in nuclear radiation environment.

[0063] The above-disclosed embodiments are merely preferred embodiments of the present invention. However, the embodiments of the present invention are not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. An ultrasonic array tactile feedback device for non-destructive testing in nuclear environments, characterized in that, include: The visual positioning module, located on the remote control console, is configured to acquire depth images of the operator's hand in real time, calculate the three-dimensional spatial coordinates of the fingertips, and output the target tactile feedback point location data. The remote tooling status module is set at the end of the remote inspection robot and connected to the ultrasonic probe. It is configured to collect the probe echo signal in real time, calculate and transmit the coupling efficiency value between the probe and the surface of the workpiece to be tested. The main control computing unit is connected to the visual positioning module and the remote tooling status module, respectively. It is internally configured with an adaptive dual-modal sound field reconstruction engine. This engine is configured to: activate a real-time parameterized generation mode when the fingertip movement speed exceeds a set threshold, using an iterative optimization algorithm to solve for the focused phase hologram in real time; and activate a static holographic reconstruction mode when the fingertip movement speed is below the set threshold and valid coupling efficiency data is received, retrieving the corresponding static phase hologram from a pre-stored database based on the coupling efficiency value. The engine outputs the focused phase hologram or the static phase hologram as the driving phase hologram for the ultrasonic transducer array. The gate driving unit, connected to the main control computing unit, is configured to receive the driving phase hologram, execute double-buffered time-division multiplexing control logic, convert low-voltage logic signals into multiple high-voltage driving signals, and apply a low-frequency amplitude modulation envelope to the ultrasonic carrier. An ultrasonic transducer array includes multiple piezoelectric ceramic transducer elements arranged in a topology, connected to the gate driving unit, which radiates in the air in response to the high-voltage driving signal and superimposes to generate a tactile feedback sound field, wherein the tactile feedback sound field has a distinguishable tactile primitive shape.

2. The ultrasonic array tactile feedback device for non-destructive testing in a nuclear environment according to claim 1, characterized in that, The pre-stored static phase hologram in the static holographic reconstruction mode is pre-calculated and generated by a weighted GS iterative algorithm combined with a sub-pixel physical constraint mechanism. The weighted GS iterative algorithm is configured as follows: Define a low-resolution grid for the physical array plane and a high-resolution simulation grid for the target sound field plane, wherein the spatial sampling density of the high-resolution simulation grid is higher than that of the low-resolution grid; The source plane complex amplitude is propagated to the target plane in the forward direction using the angular spectrum propagation operator to obtain the target complex amplitude. The phase is kept unchanged and the amplitude is replaced with the preset target tactile pattern amplitude to obtain the update field. The updated field is propagated back to the source plane to obtain the updated source plane complex amplitude, and phase uniform constraints are applied to multiple sub-pixels corresponding to the same physical transducer aperture region. Repeat the above forward propagation, amplitude constraint, backward propagation and sub-pixel constraint steps until convergence, and extract the final phase matrix as the static phase hologram.

3. The ultrasonic array tactile feedback device for non-destructive testing in a nuclear environment according to claim 1, characterized in that, The visual positioning module also includes: The hand-eye transformation matrix storage unit is configured to store a pre-calibrated hand-eye transformation matrix, which is used to map the observation coordinates in the camera coordinate system to the target coordinates in the acoustic array coordinate system. The extended Kalman filter prediction unit, connected to the hand-eye transformation matrix storage unit, is configured to establish a state vector containing fingertip position and velocity. The state vector is represented as: And use the state transition matrix to predict the prior position of the fingertip when the sound wave is emitted at the next moment; The coordinate output unit is connected to the extended Kalman filter prediction unit and is configured to send the predicted coordinates as target tactile feedback point location data to the main control calculation unit.

4. The ultrasonic array tactile feedback device for non-destructive testing in a nuclear environment according to claim 1, characterized in that, The shapes of the tactile primitives include: The first shape, corresponding to the first coupling efficiency range, is used to indicate whether the probe is suspended or severely tilted. The second shape, corresponding to the second coupling efficiency range, is used to indicate the probe contact but not flattened state. The third shape, corresponding to the third coupling efficiency range, is used to indicate the perfect fit of the probe. The first, second, and third shapes have different spatial sound field distributions, which are used to prompt the operator to identify the coupling status of the remote probe through fingertip touch.

5. The ultrasonic array tactile feedback device for non-destructive testing in a nuclear environment according to claim 1, characterized in that, The gate driving unit includes: The dual-buffer memory architecture includes a first buffer and a second buffer. The first buffer is configured to store a focused phase hologram representing a tactile excitation state, and the second buffer is configured to store a defocused or zero-amplitude hologram representing a tactile quiescent state. Hardware timer, configured to generate periodic switching signals based on the frequency sensitive to human skin touch; The switching circuit is connected and communicates with the hardware timer and the dual-buffered memory architecture, and is configured to alternately select the read address of the first buffer or the second buffer according to the periodic switching signal, so that the output data is cyclically switched between the first buffer and the second buffer.

6. The ultrasonic array tactile feedback device for non-destructive testing in a nuclear environment according to claim 1, characterized in that, The gate driving unit further includes: The push-pull power amplifier circuit is configured to split each logic signal into a positive-phase drive signal and an inverted-phase drive signal, which respectively drive the power MOSFETs connected to both ends of the ultrasonic transducer.

7. The ultrasonic array tactile feedback device for non-destructive testing in a nuclear environment according to claim 1, characterized in that, The gate driving unit further includes a distributed I / O extension architecture, which includes: Multiple shift register chips are connected to the main control FPGA via a high-speed SPI bus; A global latch signal line is connected to all the aforementioned shift register chips and is configured to trigger parallel output simultaneously after all register chips have completed receiving a frame of data, ensuring that all transducers update their phases at the same time.

8. A method for generating tactile feedback using an ultrasonic array for non-destructive testing in a nuclear environment, applied to the apparatus described in any one of claims 1 to 7, characterized in that, Includes the following steps: The visual positioning module acquires depth images of the operator's hand in real time, calculates the three-dimensional spatial coordinates of the fingertips, and outputs the target tactile feedback point position data; The remote tooling status module collects the echo signal of the remote ultrasonic probe in real time, calculates the coupling efficiency value between the probe and the surface of the workpiece under test, and transmits it back. The main control computing unit switches strategies based on the received state of fingertip movement speed and coupling efficiency value: When the fingertip movement speed exceeds the set threshold, the real-time parameterized generation mode is activated, and the iterative optimization algorithm is used to solve the focused phase hologram in real time. When the fingertip movement speed is lower than a set threshold and valid coupling efficiency data is received, the static holographic reconstruction mode is activated, and the corresponding static phase hologram is retrieved from the pre-stored database according to the coupling efficiency value. The focused phase hologram or static phase hologram is output as the driving phase hologram of the ultrasonic transducer array; The driving phase hologram is subjected to double-buffered time-division multiplexing control logic, a low-frequency amplitude modulation envelope is applied to the ultrasonic carrier, and the low-voltage logic signal is converted into multiple high-voltage driving signals; The high-voltage driving signal is applied to multiple piezoelectric ceramic transducer array elements arranged in a predetermined topology, which radiate in the air and superimpose to form a tactile feedback sound field with distinguishable tactile primitive shapes.

9. The method for generating tactile feedback using an ultrasonic array for non-destructive testing in a nuclear environment according to claim 8, characterized in that, The static phase hologram is pre-calculated and generated by the following steps: Define a low-resolution grid for the physical array plane and a high-resolution simulation grid for the target sound field plane, wherein the spatial sampling density of the high-resolution simulation grid is higher than that of the low-resolution grid; The source plane complex amplitude is propagated to the target plane in the forward direction using the angular spectrum propagation operator to obtain the target complex amplitude. The phase is kept unchanged and the amplitude is replaced with the preset target tactile pattern amplitude to obtain the update field. The updated field is propagated back to the source plane to obtain the updated source plane complex amplitude, and a phase unification constraint is applied to multiple sub-pixels corresponding to the same physical transducer aperture region, that is, the phase of the multiple sub-pixels is forcibly unified to the phase angle of the complex amplitude vector sum of the region. Repeat the above forward propagation, amplitude constraint, backward propagation and sub-pixel constraint steps until convergence, and extract the final phase matrix as the static phase hologram.

10. The method for generating tactile feedback using an ultrasonic array for non-destructive testing in a nuclear environment according to claim 8, characterized in that, The visual positioning step also includes: The pre-calibrated hand-eye transformation matrix is ​​stored to map the observation coordinates in the camera coordinate system to the target coordinates in the acoustic array coordinate system; Establish a state vector containing fingertip position and velocity. Using extended Kalman filtering and state transition matrix, the prior position of the fingertip at the next moment when the sound wave is emitted is predicted based on the state vector, and used as the prediction coordinate; The predicted coordinates are used as the target tactile feedback point location data.

Images