Visual sampling interactive probe remote control system

Abstract

The invention provides a visual sampling interactive probe remote control system, and relates to the technical field of probe control, and the system comprises a visual target input module; the view field consistency correction module is used for performing view field change correction on the initial target position data according to the current pose state data returned by the sampling probe to generate view field correction target position data; the delay displacement compensation module is used for predicting the spatial displacement of the sampling probe in the display time difference according to the display time difference of the real-time video, and performing displacement compensation processing on the view field correction target position data; the target pose analysis module is used for analyzing target pose parameters required by the sampling probe for reaching the corresponding sampling target and generating target pose data; the control instruction construction module is used for constructing a probe control instruction for controlling the sampling probe to rotate, stretch out and draw back and adjusting the sampling posture; according to the invention, the autonomy and accuracy of the visual sampling interactive probe are improved.

Description

Technical Field

[0001] This invention relates to the field of probe control technology, and in particular to a remote control system for a visual sampling interactive probe. Background Technology

[0002] In existing technologies, solutions for remotely monitoring the operating status of sampling probes typically rely on fixed video monitoring modules coupled with wired control units. In a typical system, a miniature camera is positioned at the front end of the sampling probe, and real-time video is transmitted to a host computer interface via a transmission line. The operator manually adjusts the probe's extension / retraction, motor rotation, or sampling posture based on the video feed and sensor readings. Furthermore, the control terminal often employs a separate button panel or an industrial controller with a simple graphical interface. Communication between the controller and the probe typically uses RS485 or direct Ethernet connection to issue remote commands. While such systems possess basic remote control and status feedback capabilities, the interaction methods are fragmented, requiring switching between "video interface observation" and "control interface operation" for sampling actions, representing a traditional, non-integrated visual control mode.

[0003] In applications requiring point-by-point sampling along narrow channels or enclosed cavities, the existing systems described above may lead to accumulated probe positioning errors. Specifically, because the video feed and control interface are separate, operators cannot directly manipulate the target position on the screen while observing the image; they must instead input corresponding angle or displacement commands on the control interface. When there are multiple branches or angle changes within the channel, operators often need to rely on memory to determine the probe's relative position. If the display delay of approximately 150–300ms (a common encoding and transmission delay in existing industrial cameras) is inconsistent with the channel's spatial geometry, the probe may deviate from the target sampling point or even enter the wrong branch, resulting in accumulated errors. These errors directly affect the spatial accuracy of sampling, rather than simply being an efficiency or cost issue; they represent a technical flaw that is difficult to avoid in existing separate visualization and control models. Summary of the Invention

[0004] The purpose of this invention is to provide a remote control system for a visual sampling interactive probe, which aims to solve the problems mentioned in the background art.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] A remote control system for a visual sampling interactive probe, the system comprising:

[0007] The visualization target input module is used to acquire real-time video collected by the front end of the sampling probe and generate an interactive target selection interface in the real-time video to receive the sampling target position specified by the operator in the video screen and form initial target position data.

[0008] The field-of-view consistency correction module is used to correct the field-of-view changes of the initial target position data based on the current pose state data returned by the sampling probe, and generate field-of-view corrected target position data.

[0009] The delay displacement compensation module is used to predict the spatial displacement of the sampling probe within the display time difference based on the real-time video display time difference, and to perform displacement compensation processing on the field of view correction target position data to generate time calibration target position data.

[0010] The target pose analysis module is used to calibrate the target position data according to time, analyze the target pose parameters required for the sampling probe to reach the corresponding sampling target, and generate target pose data.

[0011] The control command construction module is used to construct probe control commands for controlling the rotation, extension, and sampling posture adjustment of the sampling probe based on the target pose data.

[0012] The remote execution feedback module is used to send probe control commands to the actuator of the sampling probe and obtain the pose status data of the sampling probe after execution, so that the field of view consistency correction module can continuously update and process it.

[0013] Preferably, the field-of-view consistency correction module includes:

[0014] The pose state sequence construction unit is used to acquire pose state data returned by the sampling probe in multiple consecutive control cycles and construct pose state sequence data in chronological order.

[0015] The pose change feature extraction unit is used to perform adjacent cycle comparison processing on pose state sequence data, extract the motion direction and motion amplitude features of the sampling probe in continuous control cycle, and generate pose change feature data.

[0016] The field-of-view change mapping unit is used to map pose change feature data to corresponding field-of-view change feature data based on the structural parameters of the sampling probe and the camera viewing angle parameters.

[0017] The field of view offset determination unit is used to determine the field of view offset data caused by the movement of the sampling probe in the video frame based on the field of view change characteristic data;

[0018] The target position correction generation unit is used to perform reverse correction processing on the initial target position data based on the field of view offset data to generate field of view corrected target position data.

[0019] Preferably, the time-delay displacement compensation module includes:

[0020] The display time difference acquisition unit is used to acquire the display time difference data corresponding to the time difference from the completion of real-time video acquisition by the sampling probe to the completion of display on the visualization interface.

[0021] The historical pose data caching unit is used to cache the pose state data of the sampling probe in multiple control cycles within the time range corresponding to the display time difference, forming a historical pose data set;

[0022] The pose change rate calculation unit is used to perform time correlation analysis on the historical pose data set to determine the pose change rate data of the sampling probe per unit time.

[0023] The time-delay displacement range prediction unit is used to predict the spatial displacement range data generated by the sampling probe during the display time difference based on the display time difference data and the pose change rate data.

[0024] The target position displacement compensation unit is used to perform displacement compensation processing on the field-of-view corrected target position data based on the spatial displacement range data, and generate time-calibrated target position data.

[0025] Preferably, the target pose resolution module includes:

[0026] The spatial relative relationship construction unit is used to construct the spatial relative relationship data between the current position of the sampling probe and the sampling target position based on the time calibration target position data and the current pose state data.

[0027] The pose adjustment requirement decomposition unit is used to decompose the pose adjustment requirements required for the sampling probe to reach the sampling target into rotation adjustment requirement data, telescopic adjustment requirement data, and sampling posture adjustment requirement data based on spatial relative relationship data.

[0028] The pose execution sequence generation unit is used to plan the sequence of rotation adjustment requirement data, telescopic adjustment requirement data and sampling posture adjustment requirement data according to the motion constraints of the sampling probe, and generate pose execution sequence data.

[0029] The target pose data generation unit is used to combine and process the pose adjustment requirement data according to the pose execution sequence data to generate the target pose data.

[0030] Preferably, the pose change feature extraction unit includes:

[0031] The pose difference sequence generation subunit is used to compare the pose state data corresponding to adjacent control cycles in the pose state sequence data one by one to generate pose difference sequence data that characterizes the pose change between each control cycle.

[0032] The difference stability filtering subunit is used to filter pose changes that recur within multiple consecutive control cycles based on pose difference sequence data, and generate stable pose difference data.

[0033] The dominant change direction determination subunit is used to determine the dominant change direction of the sampling probe in the current time period based on the stable pose difference data, and generate dominant direction data.

[0034] The variation amplitude level classification subunit is used to classify the variation amplitude according to the distribution of the magnitude of the variation in the stable pose difference data, and generate variation amplitude level data.

[0035] The pose change feature construction subunit is used to combine the dominant direction data with the change amplitude level data to generate pose change feature data that characterizes the motion state of the sampling probe.

[0036] Preferably, the field-of-view change mapping unit includes:

[0037] The mapping parameter standardization subunit is used to obtain the structural parameters of the sampling probe and the camera viewing angle parameters, and perform uniform scaling to generate standardized mapping parameter data;

[0038] The pose-field association construction subunit is used to establish a correspondence between pose change feature data and field of view changes in video images based on standardized mapping parameter data, and generate pose-field association data.

[0039] The mapping weight adjustment subunit is used to adjust the weight of the pose-field association data according to the difference in the degree of influence of the sampling probe on the video field of view under different pose change directions and different change amplitude levels, and generate weighted association data.

[0040] The field-of-view change feature generation subunit is used to convert pose change feature data into field-of-view change feature data corresponding to the video frame based on weighted correlation data.

[0041] The field-of-view change continuity verification subunit is used to verify the continuity of the field-of-view change feature data generated within a continuous control cycle, and generate stable field-of-view change feature data.

[0042] Preferably, the pose change rate calculation unit includes:

[0043] The time series marker subunit is used to generate time markers for the pose state data corresponding to each control cycle in the historical pose data set, forming pose time series data.

[0044] The pose change sequence generation subunit is used to compare and process the pose state data corresponding to adjacent time markers in the pose time series data to generate pose change sequence data.

[0045] The rate candidate generation subunit is used to generate multiple candidate pose change rate data based on the pose change sequence data and the corresponding time interval data.

[0046] The rate fluctuation suppression subunit is used to perform continuous analysis on candidate pose change rate data, suppress abnormal change rates, and generate smooth rate data.

[0047] The effective rate determination subunit is used to determine the pose change rate data characterizing the current motion state of the sampling probe from the smoothed rate data.

[0048] Preferably, the time-delay displacement range prediction unit includes:

[0049] The display delay interval parsing subunit is used to parse the corresponding video display delay interval data based on the display time difference data.

[0050] The delay interval pose trend construction subunit is used to construct the pose change trend data of the sampling probe within the delay interval based on the video display delay interval data and pose change rate data;

[0051] The displacement change cumulative estimation subunit is used to accumulate and calculate the displacement changes that the sampling probe may produce within the delay interval based on the pose change trend data, and generate cumulative displacement data.

[0052] The displacement range constraint correction subunit is used to constrain and correct the cumulative displacement data according to the structural limitations and motion constraints of the sampling probe, and generate spatial displacement range data.

[0053] Preferably, the pose-field-of-view association construction subunit includes:

[0054] The field-of-view partitioning subunit is used to divide the video image into multiple field-of-view regions based on the imaging range and display structure of the video image, and generate field-of-view region data.

[0055] The pose change component analysis subunit is used to perform component analysis on pose change feature data, extract the direction change component and amplitude change component corresponding to the field of view data, and generate pose change component data.

[0056] The region association rule generation subunit is used to establish the correspondence rules between the pose change component data and each field of view region based on the standardized mapping parameter data, and generate region association rule data.

[0057] The multi-region influence overlay subunit is used to overlay the corresponding regional association rule data when pose change components act on multiple field of view regions simultaneously, and generate comprehensive association data.

[0058] The pose-field-of-view correlation data generation subunit is used to determine the manifestation of field-of-view changes caused by pose change feature data in the video frame based on the comprehensive correlation data, and generate pose-field-of-view correlation data.

[0059] Preferably, the delay interval pose trend construction subunit includes:

[0060] The delay interval time segmentation sub-unit is used to divide the delay interval into multiple consecutive time segments based on the video display delay interval data, and generate delay time segment data;

[0061] The rate change interval matching subunit is used to match the rate change within the corresponding time period from the pose change rate data based on the delay time period data, and generate interval rate data.

[0062] The dominant change trend extraction sub-unit is used to compare and analyze interval rate data, extract the dominant pose change trend that persists within the delay interval, and generate dominant trend data.

[0063] The trend continuity verification subunit is used to verify the continuity of the dominant trend data between adjacent time delay periods, eliminate discontinuous trend changes, and generate continuous trend data.

[0064] The pose change trend data generation subunit is used to construct pose change trend data to characterize the overall motion direction of the sampling probe within the delay interval based on continuous trend data.

[0065] The above-described solution of the present invention has at least the following beneficial effects:

[0066] First, by completing the selection of sampling targets and the generation of control commands in the same visual interface, operators can directly perform interactive operations based on the target positions in the real-time video screen, avoiding frequent switching between the video observation interface and the independent control interface. This changes the operation process from the existing separate visualization and control mode, which relies on memory and manual calculation, and provides an integrated interactive foundation for the remote control of sampling probes.

[0067] Furthermore, by introducing a field-of-view consistency correction mechanism during the control process, the pose change of the sampling probe is associated with the field-of-view change of the video image. This allows the sampling target position selected by the operator in the video image to be automatically corrected as the probe's posture changes. Thus, when the probe turns, extends, or adjusts its posture, the target position in the video image remains consistent with the actual observation field of view, reducing the impact of image offset caused by probe movement on target positioning.

[0068] Based on this, by analyzing the time delay in the real-time video display process and combining the pose change of the sampling probe during the delay, displacement compensation is performed on the target position. This allows the system to still calibrate the actual spatial position of the sampling target even with encoding and transmission delays. This avoids the probe gradually deviating from the expected sampling path in narrow channels or complex cavities due to video display lag, thereby suppressing the accumulation of positioning errors.

[0069] Furthermore, by resolving the target position after field-of-view correction and delay compensation into a target pose that meets the motion constraints of the sampling probe, and constructing corresponding control commands accordingly, the rotation, extension, and sampling attitude adjustment processes of the probe have a clear execution sequence and executability, reducing the control instability problem caused by directly issuing abstract displacement or angle commands.

[0070] Finally, by continuously feeding back the pose status of the probe after execution and participating in subsequent field-of-view correction and target update processing, a complete remote control closed loop is formed. This enables the sampling probe to continuously correct its own position and target orientation during continuous operation when sampling point by point along narrow channels or closed cavities. This improves the spatial consistency and controllability of the sampling position in complex spatial environments and effectively overcomes the problem of positioning error accumulation that is prone to occur in existing separate visualization and control schemes. Attached Figure Description

[0071] Figure 1 This is an architecture diagram of the remote control system for the visual sampling interactive probe provided in an embodiment of the present invention. Detailed Implementation

[0072] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0073] like Figure 1 As shown, embodiments of the present invention propose a remote control system for a visual sampling interactive probe, the system comprising:

[0074] The visualization target input module is used to acquire real-time video collected by the front end of the sampling probe and generate an interactive target selection interface in the real-time video to receive the sampling target position specified by the operator in the video screen and form initial target position data.

[0075] The field-of-view consistency correction module is used to correct the field-of-view changes of the initial target position data based on the current pose state data returned by the sampling probe, and generate field-of-view corrected target position data.

[0076] The delay displacement compensation module is used to predict the spatial displacement of the sampling probe within the display time difference based on the real-time video display time difference, and to perform displacement compensation processing on the field of view correction target position data to generate time calibration target position data.

[0077] The target pose analysis module is used to calibrate the target position data according to time, analyze the target pose parameters required for the sampling probe to reach the corresponding sampling target, and generate target pose data.

[0078] The control command construction module is used to construct probe control commands for controlling the rotation, extension, and sampling posture adjustment of the sampling probe based on the target pose data.

[0079] The remote execution feedback module is used to send probe control commands to the actuator of the sampling probe and obtain the pose status data of the sampling probe after execution, so that the field of view consistency correction module can continuously update and process it.

[0080] In this embodiment of the invention, by directly using the real-time video collected by the front end of the sampling probe in the visualization target input module for interactive selection of the sampling target position, the operator can intuitively determine the sampling position based on the current video screen, thereby reducing the position understanding deviation caused by relying on memory or coordinate input in traditional remote control methods, and providing a clear and unified initial target position data basis for the subsequent control process.

[0081] By setting up a field-of-view consistency correction module, the pose changes generated by the sampling probe during its movement are introduced into the target position correction process. This allows the initial target position data to be dynamically adjusted as the pose of the sampling probe changes, thereby avoiding the problem of inconsistency between the video image and the actual observation field of view caused by changes in the pose of the sampling probe. This ensures that the sampling target selected by the operator in the visualization interface always maintains a correspondence with the actual observation area of ​​the sampling probe.

[0082] The delay displacement compensation module processes the unavoidable display time difference during real-time video display. Combined with the pose change of the sampling probe during the display time difference, the target position is displacement compensated so that the corrected target position can reflect the actual spatial position change of the sampling probe during the video display lag, thereby reducing the impact of display delay on the accuracy of sampling target positioning.

[0083] The target pose analysis module transforms the target position data, after field correction and delay compensation, into specific pose adjustment requirements that the sampling probe needs to perform. It also generates target pose data based on the motion constraints of the sampling probe itself, thus transforming the control process of the sampling probe from an abstract description of the target position into an executable pose adjustment process, thereby improving the operability and stability of remote control.

[0084] Through the cooperation between the control command construction module and the remote execution feedback module, the target pose data is converted into specific probe control commands and sent to the sampling probe actuator. At the same time, the pose status data after the sampling probe is executed is acquired in real time and fed back to the field of view consistency correction module. This enables continuous updating and closed-loop adjustment during the sampling probe control process, avoiding cumulative deviations caused by a single control command.

[0085] For example, when detecting the distribution of pollutants in narrow passages inside equipment, the operator can directly select the sampling location in real-time video through a visual interface. When the sampling probe turns or extends in the passage, the system automatically corrects the field of view and compensates for the delayed displacement of the target position, and generates the corresponding target pose and control commands accordingly, so that the sampling probe can accurately reach the selected position to complete the sampling operation. This enables remote control of the sampling probe and stable execution of the sampling process in complex spatial environments.

[0086] In a preferred embodiment of the present invention, the control command construction module is used to construct probe control commands for controlling the rotation, extension, and sampling posture adjustment of the sampling probe based on the target pose data, specifically including:

[0087] Receive target pose data output by the target pose analysis module, and perform component analysis on the target pose data to distinguish between rotation adjustment requirements, scaling adjustment requirements, and sampling posture adjustment requirements.

[0088] Based on the control interface type of the sampling probe, various pose adjustment requirements are converted into corresponding control parameter forms, so that the control parameters conform to the instruction receiving specifications of the sampling probe actuator.

[0089] Various control parameters are combined sequentially to form a complete probe control command sequence, enabling rotation, extension, and sampling posture adjustment to be executed in a predetermined order.

[0090] After generating the control command sequence, the integrity of the control commands is verified to confirm that the control parameters are all within the control range allowed by the sampling probe, and the probe control commands that pass the verification are output to the remote execution feedback module.

[0091] In a preferred embodiment of the present invention, the remote execution feedback module is used to send probe control commands to the execution mechanism of the sampling probe and acquire the pose state data of the sampling probe after execution, so as to provide continuous updating and processing by the field of view consistency correction module, specifically including:

[0092] Receive probe control commands output by the control command construction module, and send the control commands to the actuator corresponding to the sampling probe via remote communication;

[0093] During the process of the sampling probe executing control commands, the system continuously receives the execution status information returned by the sampling probe and extracts the current pose status data of the sampling probe from it.

[0094] The acquired pose state data is time-stamped to distinguish the execution results corresponding to different control cycles;

[0095] The pose state data after time stamping is fed back to the field of view consistency correction module, so that the subsequent field of view correction process can be updated based on the latest execution state of the sampling probe.

[0096] In a preferred embodiment of the present invention, the field-of-view consistency correction module includes:

[0097] The pose state sequence construction unit is used to acquire pose state data returned by the sampling probe in multiple consecutive control cycles and construct pose state sequence data in chronological order.

[0098] The pose change feature extraction unit is used to perform adjacent cycle comparison processing on pose state sequence data, extract the motion direction and motion amplitude features of the sampling probe in continuous control cycle, and generate pose change feature data.

[0099] The field-of-view change mapping unit is used to map pose change feature data to corresponding field-of-view change feature data based on the structural parameters of the sampling probe and the camera viewing angle parameters.

[0100] The field of view offset determination unit is used to determine the field of view offset data caused by the movement of the sampling probe in the video frame based on the field of view change characteristic data;

[0101] The target position correction generation unit is used to perform reverse correction processing on the initial target position data based on the field of view offset data to generate field of view corrected target position data.

[0102] In this embodiment of the invention, by introducing a pose state sequence construction and pose change feature extraction mechanism into the field-of-view consistency correction module, the pose changes of the sampling probe over multiple consecutive control cycles are analyzed as a whole. This allows the correction of the target position to no longer rely on pose information at a single moment, but rather to be based on continuous motion state. Simultaneously, by utilizing the field-of-view change mapping and field-of-view offset determination process, the structural parameters of the sampling probe and the camera viewing angle parameters are incorporated into the field-of-view change analysis. This ensures that pose changes are accurately reflected as field-of-view offsets in the video image, and the target position is corrected accordingly. This maintains consistency between the sampled target position in the video image and the actual observed field of view during the sampling probe's movement.

[0103] In a preferred embodiment of the present invention, the field-of-view offset determination unit is used to determine the field-of-view offset data caused by the movement of the sampling probe in the video frame based on the field-of-view change feature data, specifically including:

[0104] The system receives field-of-view change feature data output by the field-of-view change mapping unit and performs directional component analysis on the field-of-view change feature data to determine the main direction of field-of-view offset in the video frame.

[0105] By combining the change amplitude information reflected in the field of view change feature data, the range of the field of view offset in the video frame is determined;

[0106] The determined offset direction and offset degree are combined to form field offset data, which characterizes the overall offset of the video image.

[0107] The field of view offset data is output to the target position correction generation unit as the basis for subsequent reverse correction of the target position.

[0108] In a preferred embodiment of the present invention, the target position correction generation unit is used to perform reverse correction processing on the initial target position data based on the field of view offset data to generate field of view corrected target position data, specifically including:

[0109] Receive the field of view offset data output by the field of view offset determination unit, and obtain the initial target position data corresponding to the field of view offset;

[0110] Based on the offset direction reflected by the field of view offset data, the initial target position data is adjusted in the opposite direction to counteract the video image offset caused by the movement of the sampling probe.

[0111] Based on the degree of offset reflected by the field of view offset data, the adjustment range of the initial target position data is limited so that the corrected target position is still within the effective display range of the video image;

[0112] The target position data after the reverse correction process is completed is output as the field-of-view corrected target position data and transmitted to the time-delay displacement compensation module for subsequent processing.

[0113] In a preferred embodiment of the present invention, the time-delay displacement compensation module includes:

[0114] The display time difference acquisition unit is used to acquire the display time difference data corresponding to the time difference from the completion of real-time video acquisition by the sampling probe to the completion of display on the visualization interface.

[0115] The historical pose data caching unit is used to cache the pose state data of the sampling probe in multiple control cycles within the time range corresponding to the display time difference, forming a historical pose data set;

[0116] The pose change rate calculation unit is used to perform time correlation analysis on the historical pose data set to determine the pose change rate data of the sampling probe per unit time.

[0117] The time-delay displacement range prediction unit is used to predict the spatial displacement range data generated by the sampling probe during the display time difference based on the display time difference data and the pose change rate data.

[0118] The target position displacement compensation unit is used to perform displacement compensation processing on the field-of-view corrected target position data based on the spatial displacement range data, and generate time-calibrated target position data.

[0119] In this embodiment of the invention, a delay displacement compensation module processes the display time difference generated during real-time video display and, combined with historical pose data and pose change rate analysis, predicts the spatial displacement that the sampling probe may generate during the display delay. This ensures that target position compensation is no longer a static correction but is integrated with the actual motion state of the sampling probe. In this way, the system can pre-correct the sampling target position even when there is a lag in the video display, thereby reducing the impact of display delay on the accuracy of remote sampling control and making the sampling target position seen by the operator in the visualization interface closer to the actual reachable position of the sampling probe.

[0120] In a preferred embodiment of the present invention, the display time difference acquisition unit is used to acquire display time difference data corresponding to the completion of real-time video acquisition from the sampling probe to the completion of display on the visualization interface, specifically including:

[0121] When the sampling probe completes the acquisition of a single frame of video image, a corresponding acquisition time marker is generated for that video image;

[0122] When the video image is transmitted, processed and displayed on the visualization interface, a corresponding display time stamp is generated for the video image;

[0123] The acquisition time marker and the display time marker are compared to determine the time interval from acquisition to display of the video image, and this time interval is used as the display time difference data.

[0124] The display time difference data corresponding to multiple consecutive video images is recorded to reflect the time delay in the current video transmission and display process, and the display time difference data is output to the subsequent processing unit of the delay displacement compensation module.

[0125] In a preferred embodiment of the present invention, the historical pose data caching unit is used to cache the pose state data of the sampling probe in multiple control cycles within a time range corresponding to the display time difference, forming a historical pose data set, specifically including:

[0126] During the process of the sampling probe executing control commands, it continuously receives the sampling probe pose status data returned by the remote execution feedback module and generates a corresponding time stamp for each set of pose status data.

[0127] The pose state data is stored in the cache area in chronological order to form a pose state data sequence arranged by time.

[0128] Based on the display time difference data output by the display time difference acquisition unit, determine the time range corresponding to the display time difference;

[0129] The pose state data falling within the time range are selected from the pose state data sequence to construct a historical pose data set, and the historical pose data set is output to the pose change rate calculation unit.

[0130] In a preferred embodiment of the present invention, the target position displacement compensation unit is used to perform displacement compensation processing on the field-of-view correction target position data based on the spatial displacement range data, and generate time-calibrated target position data, specifically including:

[0131] Receive spatial displacement range data output by the delayed displacement range prediction unit and obtain the corresponding field-of-view correction target position data;

[0132] Based on the displacement direction information reflected by the spatial displacement range data, determine the direction in which the field of view correction target position data needs to be compensated;

[0133] Based on the displacement range defined by the spatial displacement range data, the compensation amplitude of the field of view correction target position data is constrained so that the compensation result conforms to the actual motion possibility of the sampling probe during the display time difference.

[0134] Under the premise of satisfying the displacement direction and displacement range constraints, the displacement of the target position data for field correction is adjusted to generate time-calibrated target position data;

[0135] The time calibration target position data is output to the target pose analysis module for subsequent target pose generation processing.

[0136] In a preferred embodiment of the present invention, the target pose resolution module includes:

[0137] The spatial relative relationship construction unit is used to construct the spatial relative relationship data between the current position of the sampling probe and the sampling target position based on the time calibration target position data and the current pose state data.

[0138] The pose adjustment requirement decomposition unit is used to decompose the pose adjustment requirements required for the sampling probe to reach the sampling target into rotation adjustment requirement data, telescopic adjustment requirement data, and sampling posture adjustment requirement data based on spatial relative relationship data.

[0139] The pose execution sequence generation unit is used to plan the sequence of rotation adjustment requirement data, telescopic adjustment requirement data and sampling posture adjustment requirement data according to the motion constraints of the sampling probe, and generate pose execution sequence data.

[0140] The target pose data generation unit is used to combine and process the pose adjustment requirement data according to the pose execution sequence data to generate the target pose data.

[0141] In this embodiment of the invention, the target pose analysis module transforms the target position data, after field-of-view correction and delay compensation, into specific pose adjustment requirements that the sampling probe needs to execute. Based on the motion constraints of the sampling probe, the various pose adjustment requirements are sequentially planned, ensuring that the target pose generation process conforms to both the position requirements of the sampling target and the motion characteristics of the sampling probe itself. This approach avoids the unexecutable or unstable control problems caused by directly generating control commands based solely on the target position, making the remote control process of the sampling probe more consistent with actual execution conditions and improving the continuity and reliability of the control process.

[0142] In a preferred embodiment of the present invention, the spatial relative relationship construction unit is used to construct spatial relative relationship data between the current position of the sampling probe and the sampling target position based on the time calibration target position data and the current pose state data, specifically including:

[0143] It receives time calibration target position data output by the delay displacement compensation module and receives current pose status data of the sampling probe returned by the remote execution feedback module;

[0144] The time calibration target position data and the current pose state data are processed to unify the coordinate reference, so that the two are under the same spatial reference system;

[0145] Based on the target position under a unified coordinate reference and the current position of the sampling probe, determine the relative direction and relative distance between the two.

[0146] The relative direction relationship and the relative distance relationship are combined and processed to form spatial relative relationship data describing the current position of the sampling probe relative to the sampling target position, and the spatial relative relationship data is output to the pose adjustment requirement decomposition unit.

[0147] In a preferred embodiment of the present invention, the pose adjustment requirement decomposition unit is used to decompose the pose adjustment requirement required for the sampling probe to reach the sampling target into rotation adjustment requirement data, telescopic adjustment requirement data, and sampling posture adjustment requirement data based on spatial relative relationship data, specifically including:

[0148] Receive spatial relative relationship data output by the spatial relative relationship construction unit;

[0149] Based on the relative directional relationships reflected in the spatial relative relationship data, determine the rotation adjustment requirements of the sampling probe and generate the corresponding rotation adjustment requirement data;

[0150] Based on the relative distance relationships reflected in the spatial relative relationship data, determine the required extension and retraction adjustments of the sampling probe, and generate the corresponding extension and retraction adjustment requirement data;

[0151] Based on the current attitude state of the sampling probe, determine the sampling attitude adjustment requirements needed to maintain the stability of the sampling process, and generate the corresponding sampling attitude adjustment requirement data;

[0152] The rotation adjustment requirement data, the telescoping adjustment requirement data, and the sampled posture adjustment requirement data are output as pose adjustment requirement decomposition results to the pose execution sequence generation unit.

[0153] In a preferred embodiment of the present invention, the pose execution sequence generation unit is used to perform sequential planning on the rotation adjustment requirement data, the telescopic adjustment requirement data, and the sampling posture adjustment requirement data according to the motion constraints of the sampling probe, and generate pose execution sequence data, specifically including:

[0154] Receive rotation adjustment requirement data, telescopic adjustment requirement data, and sampled posture adjustment requirement data output by the pose adjustment requirement decomposition unit;

[0155] Obtain motion constraints related to the structural characteristics of the sampling probe, including the mutual constraint relationship between rotation and extension, as well as the sampling posture maintenance requirements;

[0156] Based on the aforementioned motion constraints, the feasibility of various pose adjustment requirements is determined, and the sequential execution relationship between different adjustment requirements is identified.

[0157] Under the premise of satisfying the motion constraints of the sampling probe, the rotation adjustment, telescopic adjustment and sampling posture adjustment are sequentially planned to generate pose execution sequence data, and the pose execution sequence data is output to the target pose data generation unit.

[0158] In a preferred embodiment of the present invention, the target pose data generation unit is used to combine and process each pose adjustment requirement data according to the pose execution sequence data to generate target pose data, specifically including:

[0159] Receive pose execution order data output by the pose execution order generation unit, as well as corresponding rotation adjustment requirement data, scaling adjustment requirement data and sampling posture adjustment requirement data;

[0160] According to the execution order determined in the pose execution sequence data, the various pose adjustment requirement data are combined and processed in sequence to form a complete pose adjustment scheme;

[0161] A consistency check is performed on the combined pose adjustment scheme to confirm that there are no conflicts between the pose adjustment requirements and that they meet the motion constraints of the sampling probe.

[0162] The pose adjustment scheme after consistency check is output as the target pose data and transmitted to the control command construction module to generate the corresponding probe control command.

[0163] In a preferred embodiment of the present invention, the pose change feature extraction unit includes:

[0164] The pose difference sequence generation subunit is used to compare the pose state data corresponding to adjacent control cycles in the pose state sequence data one by one to generate pose difference sequence data that characterizes the pose change between each control cycle.

[0165] The difference stability filtering subunit is used to filter pose changes that recur within multiple consecutive control cycles based on pose difference sequence data, and generate stable pose difference data.

[0166] The dominant change direction determination subunit is used to determine the dominant change direction of the sampling probe in the current time period based on the stable pose difference data, and generate dominant direction data.

[0167] The variation amplitude level classification subunit is used to classify the variation amplitude according to the distribution of the magnitude of the variation in the stable pose difference data, and generate variation amplitude level data.

[0168] The pose change feature construction subunit is used to combine the dominant direction data with the change amplitude level data to generate pose change feature data that characterizes the motion state of the sampling probe.

[0169] In this embodiment of the invention, by constructing the pose difference sequence and performing difference stability screening in the pose change feature extraction unit, the pose change features of the sampling probe are no longer directly determined by a single pose change, but are judged based on pose changes that occur repeatedly within multiple continuous control cycles, thereby reducing the impact of occasional pose fluctuations on the motion feature extraction results. Simultaneously, by judging and classifying the pose change direction and amplitude separately, the pose change features can simultaneously reflect the motion direction and amplitude characteristics of the sampling probe, providing more stable and distinguishable input data for subsequent field-of-view change mapping.

[0170] In a preferred embodiment of the present invention, the pose difference sequence generation subunit is used to compare the pose state data corresponding to adjacent control cycles in the pose state sequence data one by one to generate pose difference sequence data characterizing the pose changes between each control cycle, specifically including:

[0171] Receive pose state sequence data output by the pose state sequence construction unit, wherein the pose state sequence data contains the pose state of the sampling probe within multiple consecutive control cycles;

[0172] Two sets of pose state data corresponding to adjacent control cycles are selected in chronological order, and the changes in direction, position and attitude of the two are compared and analyzed.

[0173] The pose change results between adjacent control cycles are recorded as a set of pose difference data and stored sequentially in chronological order;

[0174] By repeatedly performing the above comparison process on multiple adjacent control cycles, a pose difference sequence data reflecting the pose change of the sampling probe in the continuous control cycle is formed, and the pose difference sequence data is output to the difference stability screening subunit.

[0175] In a preferred embodiment of the present invention, the difference stability screening subunit is used to screen pose changes that recur within multiple consecutive control cycles based on pose difference sequence data to generate stable pose difference data, specifically including:

[0176] Receive pose difference sequence data output by the pose difference sequence generation subunit;

[0177] Within a preset number of continuous control cycles, the pose difference sequence data are compared to identify pose differences with consistent direction and trend of change in multiple control cycles.

[0178] Persistent pose differences are classified as stable differences, while pose differences that occur only in a single or a few control cycles are classified as unstable differences.

[0179] The pose difference data corresponding to the stable difference is retained to form stable pose difference data, and the stable pose difference data is output to the dominant change direction determination subunit.

[0180] In a preferred embodiment of the present invention, the dominant change direction determination subunit is used to determine the dominant pose change direction of the sampling probe in the current time period based on stable pose difference data, and generate dominant direction data, specifically including:

[0181] Receive stable pose difference data output by the difference stability filtering subunit;

[0182] Statistical analysis is performed on the pose change direction reflected in the stable pose difference data to determine the change direction with the highest frequency or longest duration in the current time period.

[0183] The determined main change direction is taken as the dominant pose change direction of the sampling probe in that time period, and the corresponding dominant direction data is generated.

[0184] The dominant direction data is output to the change amplitude level division subunit for subsequent pose change amplitude analysis.

[0185] In a preferred embodiment of the present invention, the variation amplitude level division subunit is used to classify the pose change amplitude according to the magnitude distribution of pose change in the stable pose difference data, and generate variation amplitude level data, specifically including:

[0186] Receive stable pose difference data output by the difference stability filtering subunit;

[0187] The amplitude of pose changes reflected in the stable pose difference data is summarized and analyzed to determine the distribution of different amplitude changes in the current time period.

[0188] Based on the pre-defined amplitude range, the pose change amplitude is divided into multiple levels, and each set of stable pose difference data is assigned to the corresponding amplitude level.

[0189] Generate amplitude level data to characterize the degree of pose change of the sampling probe, and output the amplitude level data to the pose change feature construction subunit.

[0190] In a preferred embodiment of the present invention, the pose change feature construction subunit is used to combine the dominant direction data and the change amplitude level data to generate pose change feature data for characterizing the motion state of the sampling probe, specifically including:

[0191] Receive dominant direction data output by the dominant change direction determination subunit, and change magnitude level data output by the change magnitude level division subunit;

[0192] The main motion direction characteristics of the sampling probe within the current time period are determined based on the dominant direction data;

[0193] The motion intensity level of the sampling probe in the main motion direction is determined by combining the change amplitude level data;

[0194] The motion direction characteristics are combined with the motion intensity level to form a complete description of the pose change characteristics;

[0195] The resulting pose change features are output as pose change feature data and transmitted to the field of view change mapping unit for subsequent processing.

[0196] In a preferred embodiment of the present invention, the field-of-view change mapping unit includes:

[0197] The mapping parameter standardization subunit is used to obtain the structural parameters of the sampling probe and the camera viewing angle parameters, and perform uniform scaling to generate standardized mapping parameter data;

[0198] The pose-field association construction subunit is used to establish a correspondence between pose change feature data and field of view changes in video images based on standardized mapping parameter data, and generate pose-field association data.

[0199] The mapping weight adjustment subunit is used to adjust the weight of the pose-field association data according to the difference in the degree of influence of the sampling probe on the video field of view under different pose change directions and different change amplitude levels, and generate weighted association data.

[0200] The field-of-view change feature generation subunit is used to convert pose change feature data into field-of-view change feature data corresponding to the video frame based on weighted correlation data.

[0201] The field-of-view change continuity verification subunit is used to verify the continuity of the field-of-view change feature data generated within a continuous control cycle, and generate stable field-of-view change feature data.

[0202] In this embodiment of the invention, the structural parameters and camera viewing angle parameters are standardized in the field-of-view mapping unit, and the correlation between pose and field of view is constructed by combining pose change features. This allows the motion state of the sampling probe to be accurately mapped to the field-of-view changes in the video image. Simultaneously, by adjusting the weights of the mapping relationships under different pose change directions and amplitude levels, and by performing continuity verification on the field-of-view change features generated within a continuous control cycle, discontinuities or abrupt changes in the field-of-view mapping results within a short period are avoided, thereby improving the consistency between the field-of-view changes in the video image and the actual motion of the sampling probe.

[0203] In a preferred embodiment of the present invention, the mapping parameter standardization subunit is used to acquire the structural parameters and camera viewing angle parameters of the sampling probe, and perform uniform scaling to generate standardized mapping parameter data, specifically including:

[0204] Obtain structural parameter information related to the mechanical structure of the sampling probe, including the installation position relationship of the sampling probe, the position of the rotation center, and the fixed relationship of the camera device relative to the sampling probe;

[0205] Obtain the camera viewing angle parameter information corresponding to the camera device, including the coverage area and imaging direction characteristics of the camera image;

[0206] The structural parameters and camera viewpoint parameters are converted to a unified data format so that parameters from different sources and with different dimensions can be used in the same processing flow.

[0207] The parameters after format conversion are scaled to ensure that the parameters have a consistent reference standard in subsequent mapping processes, thus forming standardized mapping parameter data.

[0208] The standardized mapping parameter data is output to the pose-field association construction subunit for establishing the correspondence between pose changes and field of view changes.

[0209] In a preferred embodiment of the present invention, the mapping weight adjustment subunit is used to adjust the weight of the pose-field-of-view correlation data according to the differences in the degree of influence of the sampling probe on the video field of view under different pose change directions and different change amplitude levels, and to generate weighted correlation data, specifically including:

[0210] It receives pose-field-of-view association data output by the pose-field-of-view association construction subunit, as well as pose change direction data and change magnitude level data output by the pose change feature extraction unit;

[0211] Based on the direction of pose change, determine the relative degree to which the motion of the sampling probe affects different areas of the video image;

[0212] Based on the level of pose change, determine the strength of the influence of the sampling probe movement on the video field of view.

[0213] Based on the combined direction and magnitude of pose change, assign corresponding influence weights to each correlation in the pose-field of view data.

[0214] The pose-field-of-view correlation data after weight allocation is output as weighted correlation data to the field-of-view change feature generation subunit.

[0215] In a preferred embodiment of the present invention, the field-of-view change feature generation subunit is used to convert pose change feature data into field-of-view change feature data corresponding to the video frame based on weighted correlation data, specifically including:

[0216] It receives weighted correlation data output by the mapping weight adjustment subunit and pose change feature data output by the pose change feature extraction unit;

[0217] Based on the correlation described in the weighted correlation data, the pose change feature data are assigned to the corresponding field of view of the video frame;

[0218] Within each field of view region, the pose change features are comprehensively processed by combining the corresponding weight information to form local field of view change features that reflect the field of view changes in that region.

[0219] The local field-of-view change features generated in each field of view region are integrated to form field-of-view change feature data that describes the overall field-of-view changes of the video image;

[0220] The field-of-view change feature data is output to the field-of-view change continuity verification subunit.

[0221] In a preferred embodiment of the present invention, the field-of-view change continuity verification subunit is used to perform continuity verification on the field-of-view change feature data generated within a continuous control cycle, and to generate stable field-of-view change feature data, specifically including:

[0222] Receive field-of-view change feature data generated over multiple consecutive control cycles;

[0223] Compare the field of view change feature data corresponding to adjacent control cycles to determine whether the changes in the direction and amplitude of the field of view change remain continuous;

[0224] When a discontinuity in the field of view between adjacent control cycles is detected, the abnormal change is suppressed.

[0225] The consistent field of view change characteristics across multiple consecutive control cycles are preserved to form stable field of view change characteristic data.

[0226] The stable field of view change feature data is output to the field of view offset determination unit for subsequent video frame offset analysis.

[0227] In a preferred embodiment of the present invention, the pose change rate calculation unit includes:

[0228] The time series marker subunit is used to generate time markers for the pose state data corresponding to each control cycle in the historical pose data set, forming pose time series data.

[0229] The pose change sequence generation subunit is used to compare and process the pose state data corresponding to adjacent time markers in the pose time series data to generate pose change sequence data.

[0230] The rate candidate generation subunit is used to generate multiple candidate pose change rate data based on the pose change sequence data and the corresponding time interval data.

[0231] The rate fluctuation suppression subunit is used to perform continuous analysis on candidate pose change rate data, suppress abnormal change rates, and generate smooth rate data.

[0232] The effective rate determination subunit is used to determine the pose change rate data characterizing the current motion state of the sampling probe from the smoothed rate data.

[0233] In this embodiment of the invention, the pose change rate calculation unit performs time-series labeling and pose change analysis on historical pose data, enabling the determination of the pose change rate to reflect the actual movement of the sampling probe within a continuous control cycle. Simultaneously, by suppressing fluctuations and filtering effective rates for candidate pose change rates, interference from instantaneous abnormal changes on the rate judgment results is avoided, making the obtained pose change rate more stable and reliable, and providing a stable rate basis for subsequent prediction of the delayed displacement range.

[0234] In a preferred embodiment of the present invention, the time series marker subunit is used to generate time markers for the pose state data corresponding to each control cycle in the historical pose data set, forming pose time series data, specifically including:

[0235] During the process of the sampling probe executing remote control commands, it continuously receives pose status data returned by the remote execution feedback module;

[0236] When each set of pose state data is generated or received, a corresponding time stamp information is added to the pose state data to characterize the control cycle corresponding to the pose state data.

[0237] The pose state data with time stamps are sorted and stored in chronological order so that each pose state data can form a continuous data sequence in chronological order.

[0238] The pose state data, after being time-stamped and ordered, is output as pose time series data to the pose change sequence generation subunit.

[0239] In a preferred embodiment of the present invention, the pose change sequence generation subunit is used to compare and process the pose state data corresponding to adjacent time markers in the pose time series data to generate pose change sequence data, specifically including:

[0240] Receive pose time series data output by the time series marker subunit;

[0241] Select two sets of pose state data corresponding to adjacent time markers in chronological order;

[0242] The changes in spatial position, orientation and attitude of the adjacent pose state data are compared and analyzed to determine the pose changes of the sampling probe in adjacent control cycles.

[0243] Record the change results between each pair of adjacent pose state data as a set of pose change data;

[0244] By repeatedly performing the above comparison process on multiple adjacent time periods, a pose change sequence data reflecting the continuous movement of the sampling probe is formed, and the pose change sequence data is output to the rate candidate generation subunit.

[0245] In a preferred embodiment of the present invention, the rate candidate generation subunit is used to generate multiple candidate pose change rate data based on the pose change sequence data and the corresponding time interval data, specifically including:

[0246] Receive pose change sequence data output by the pose change sequence generation subunit, as well as the time interval information between adjacent time markers;

[0247] For each set of pose change data, combined with its corresponding time interval, the rate of pose change of the sampling probe within that time period is determined to form a candidate pose change rate description.

[0248] The descriptions of the candidate pose change rate obtained in different time periods are stored sequentially to form multiple candidate pose change rate data.

[0249] The candidate pose change rate data is output to the rate fluctuation suppression subunit for subsequent stability processing.

[0250] In a preferred embodiment of the present invention, the rate fluctuation suppression subunit is used to perform continuous analysis on the candidate pose change rate data, suppress abnormal change rates, and generate smooth rate data, specifically including:

[0251] Receive candidate pose change rate data output by the rate candidate generation subunit;

[0252] Compare the candidate pose change rate data over multiple consecutive time periods to determine whether the rate change shows a continuous trend.

[0253] When a candidate pose change rate is detected to be significantly different from the rate before and after a time period and not continuous, the candidate rate is determined to be an abnormal change rate.

[0254] The abnormal rate of change is suppressed so that it does not participate in the determination of the subsequent effective rate;

[0255] The candidate pose change rate data with good continuity is retained to form smooth rate data, and the smooth rate data is output to the effective rate determination subunit.

[0256] In a preferred embodiment of the present invention, the effective rate determination subunit is used to determine pose change rate data characterizing the current motion state of the sampling probe from the smoothed rate data, specifically including:

[0257] Receive smoothed rate data output by the rate fluctuation suppression subunit;

[0258] By comprehensively analyzing the smoothing rate data, the overall speed of the sampling probe's movement within the current time period can be determined.

[0259] Select a rate description that can represent the current motion state from the smoothed rate data and use it as the pose change rate data of the sampling probe.

[0260] The determined pose change rate data is output to the delayed displacement range prediction unit for subsequent delayed displacement prediction processing.

[0261] In a preferred embodiment of the present invention, the delayed displacement range prediction unit includes:

[0262] The display delay interval parsing subunit is used to parse the corresponding video display delay interval data based on the display time difference data.

[0263] The delay interval pose trend construction subunit is used to construct the pose change trend data of the sampling probe within the delay interval based on the video display delay interval data and pose change rate data;

[0264] The displacement change cumulative estimation subunit is used to accumulate and calculate the displacement changes that the sampling probe may produce within the delay interval based on the pose change trend data, and generate cumulative displacement data.

[0265] The displacement range constraint correction subunit is used to constrain and correct the cumulative displacement data according to the structural limitations and motion constraints of the sampling probe, and generate spatial displacement range data.

[0266] In this embodiment of the invention, by analyzing the video display delay interval and combining it with the pose change rate of the sampling probe to construct the pose change trend within the delay interval, the delay displacement prediction no longer relies on the pose information at a single time point, but is based on the overall motion trend within the delay interval. Simultaneously, by cumulatively estimating the pose change and combining it with the structural constraints and motion constraints of the sampling probe for range correction, the predicted spatial displacement range reflects both the motion possibility of the sampling probe and conforms to actual executable conditions, thus providing a more reasonable range basis for displacement compensation at the target position.

[0267] In a preferred embodiment of the present invention, the display delay interval parsing subunit is used to parse the corresponding video display delay interval data based on the display time difference data, specifically including:

[0268] Receive display time difference data output by the display time difference acquisition unit, wherein the display time difference data is used to characterize the length of time that the video image takes from the completion of acquisition to the completion of display;

[0269] Based on the displayed time difference data, the actual acquisition time of the video image corresponding to the current display time point is determined, thereby clarifying the delay start position corresponding to the video display.

[0270] Based on the aforementioned delay start position and combined with the duration of the displayed time difference data, the complete time range covered by the video display delay is determined;

[0271] The determined time range is used as the video display delay interval data output, providing a clear time boundary for the subsequent construction of pose change trends.

[0272] In a preferred embodiment of the present invention, the displacement change accumulation estimation subunit is used to accumulate and calculate the displacement changes that the sampling probe may generate within the delay interval based on the pose change trend data, and generate displacement accumulation data, specifically including:

[0273] Receive pose change trend data output by the pose trend construction subunit of the delay interval, wherein the pose change trend data is used to characterize the overall motion direction of the sampling probe within the delay interval;

[0274] Based on the direction of motion and the rate of change reflected by the pose change trend data, the displacement changes in different time periods within the delay interval are analyzed segment by segment.

[0275] Within each time period, based on the change characteristics described by the pose change trend data, determine the possible displacement changes of the sampling probe within that time period.

[0276] The displacement changes corresponding to each time period within the delay interval are sequentially superimposed to form cumulative displacement data that reflects the overall displacement changes that the sampling probe may produce throughout the entire delay interval.

[0277] The accumulated displacement data is output to the displacement range constraint correction subunit for subsequent displacement range correction processing.

[0278] In a preferred embodiment of the present invention, the displacement range constraint correction subunit is used to perform constraint correction on the cumulative displacement data according to the structural limitations and motion constraints of the sampling probe, and generate spatial displacement range data, specifically including:

[0279] Receive cumulative displacement data output by the cumulative displacement estimation subunit;

[0280] Obtain the limiting information related to the structure of the sampling probe and the motion constraints of the sampling probe in the current working state, including the maximum movable range and the allowable motion direction limit;

[0281] The displacement changes reflected by the cumulative displacement data are compared with the structural limitations and motion constraints to determine whether the displacement changes exceed the executable range of the sampling probe.

[0282] When the displacement change corresponding to the cumulative displacement data exceeds the executable range, the excess part is corrected so that the displacement change result falls within the allowable range.

[0283] The displacement change results after constraint correction are output as spatial displacement range data for subsequent target position displacement compensation processing.

[0284] In a preferred embodiment of the present invention, the pose-field-of-view association construction subunit includes:

[0285] The field-of-view partitioning subunit is used to divide the video image into multiple field-of-view regions based on the imaging range and display structure of the video image, and generate field-of-view region data.

[0286] The pose change component analysis subunit is used to perform component analysis on pose change feature data, extract the direction change component and amplitude change component corresponding to the field of view data, and generate pose change component data.

[0287] The region association rule generation subunit is used to establish the correspondence rules between the pose change component data and each field of view region based on the standardized mapping parameter data, and generate region association rule data.

[0288] The multi-region influence overlay subunit is used to overlay the corresponding regional association rule data when pose change components act on multiple field of view regions simultaneously, and generate comprehensive association data.

[0289] The pose-field-of-view correlation data generation subunit is used to determine the manifestation of field-of-view changes caused by pose change feature data in the video frame based on the comprehensive correlation data, and generate pose-field-of-view correlation data.

[0290] In this embodiment of the invention, by dividing the video frame into multiple field-of-view regions and performing component analysis on the pose change features, the pose change of the sampling probe can be reflected in different positions of the video frame in a regionalized manner. Simultaneously, by establishing correspondence rules between pose change components and each field-of-view region, and by superimposing the results when multiple regions are simultaneously affected, the correlation between pose change and field-of-view change becomes more refined and comprehensive. This improves the accuracy of the field-of-view change representation of pose change in the video frame and avoids field-of-view description bias caused by using only a single region or a single mapping relationship.

[0291] In a preferred embodiment of the present invention, a field-of-view partitioning subunit is used to divide the video frame into multiple field-of-view regions based on the imaging range and display structure of the video frame, and to generate field-of-view region data, specifically including:

[0292] Obtain information on the overall imaging range of the real-time video frame, including the effective display boundary of the video frame;

[0293] Based on the display structure of the video frame, the imaging range is divided into multiple mutually distinct field-of-view regions, so that each field-of-view region corresponds to a different position in the video frame.

[0294] Assign corresponding area identification information to each field of view so as to distinguish different field of view areas in subsequent processing;

[0295] The range information of each field of view region and the corresponding region identification information are integrated to form field of view region data for describing the partitioning of the video screen, and the field of view region data is output to the pose change component analysis subunit.

[0296] In a preferred embodiment of the present invention, the pose change component analysis subunit is used to perform component analysis on the pose change feature data, extract the direction change component and amplitude change component corresponding to the field of view region data, and generate pose change component data, specifically including:

[0297] It receives pose change feature data output by the pose change feature extraction unit and field of view region data output by the field of view partitioning construction subunit;

[0298] Based on the regional positional relationships described by the field of view data, directional correlation analysis is performed on the pose change characteristic data to determine the directional change components corresponding to each field of view region.

[0299] The amplitude information reflected in the pose change feature data is split and processed so that different fields of view can correspond to different amplitude change components.

[0300] The direction change components and amplitude change components obtained from the analysis are combined to form pose change component data that characterizes the impact of pose change in each field of view.

[0301] The pose change component data is output to the region association rule generation subunit.

[0302] In a preferred embodiment of the present invention, the region association rule generation subunit is used to establish the correspondence rules between the pose change component data and each field of view region based on the standardized mapping parameter data, and to generate region association rule data, specifically including:

[0303] It receives standardized mapping parameter data output by the mapping parameter standardization subunit and pose change component data output by the pose change component parsing subunit.

[0304] Based on the structural characteristics of the sampling probe and the camera viewing angle characteristics reflected in the standardized mapping parameter data, the influence relationship of different pose change components on each field of view is determined.

[0305] Establish correspondence rules between different directional change components and amplitude change components and the field of view regions respectively, so that pose changes can be mapped to different field of view regions under different conditions;

[0306] The established corresponding rules are summarized to form regional association rule data, and the regional association rule data is output to the multi-region influence superposition sub-unit.

[0307] In a preferred embodiment of the present invention, the multi-region influence superposition subunit is used to superimpose the corresponding region association rule data to generate comprehensive association data when pose change components act on multiple field-of-view regions simultaneously, specifically including:

[0308] Receive the region association rule data output by the region association rule generation subunit;

[0309] Determine whether the current pose change component corresponds to multiple fields of view simultaneously, and identify the multiple fields of view involved;

[0310] When pose change components affect multiple fields of view simultaneously, the influence relationships of each field of view are combined according to the region association rules.

[0311] The influence relationships of multiple fields of view are superimposed to form comprehensive correlation data that can reflect the overall influence of multiple regions;

[0312] The integrated correlation data is output to the pose-field correlation data generation subunit.

[0313] In a preferred embodiment of the present invention, the pose-field-of-view correlation data generation subunit is used to determine the manifestation of field-of-view changes caused by pose change feature data in the video frame based on the comprehensive correlation data, and to generate pose-field-of-view correlation data, specifically including:

[0314] Receive comprehensive correlation data output from multi-region influence superposition sub-units;

[0315] Based on the multi-regional impact described in the comprehensive correlation data, determine the field-of-view changes corresponding to each field-of-view region in the video image;

[0316] The field of view changes corresponding to each field of view region are integrated to form a field of view change description that describes the overall impact of pose change feature data on the video image.

[0317] The field of view change description is output as pose-field of view associated data and passed to the mapping weight adjustment subunit for subsequent processing.

[0318] In a preferred embodiment of the present invention, the delay interval pose trend construction subunit includes:

[0319] The delay interval time segmentation sub-unit is used to divide the delay interval into multiple consecutive time segments based on the video display delay interval data, and generate delay time segment data;

[0320] The rate change interval matching subunit is used to match the rate change within the corresponding time period from the pose change rate data based on the delay time period data, and generate interval rate data.

[0321] The dominant change trend extraction sub-unit is used to compare and analyze interval rate data, extract the dominant pose change trend that persists within the delay interval, and generate dominant trend data.

[0322] The trend continuity verification subunit is used to verify the continuity of the dominant trend data between adjacent time delay periods, eliminate discontinuous trend changes, and generate continuous trend data.

[0323] The pose change trend data generation subunit is used to construct pose change trend data to characterize the overall motion direction of the sampling probe within the delay interval based on continuous trend data.

[0324] In this embodiment of the invention, by dividing the video display delay interval into multiple consecutive time periods and matching the pose change rate with each time period, the motion of the sampling probe within the delay interval can be analyzed segmentally. Simultaneously, by extracting the dominant pose change trend that persists within the delay interval and verifying the continuity of the trend between adjacent time periods, the final constructed pose change trend can reflect the overall motion trajectory of the sampling probe within the delay interval, thereby avoiding interference from short-term fluctuations or local changes in the determination of the delayed pose trend.

[0325] In a preferred embodiment of the present invention, the delay interval time period division subunit is used to divide the delay interval into multiple consecutive time periods based on the video display delay interval data, and generate delay time period data, specifically including:

[0326] Receive video display delay interval data output by the display delay interval parsing subunit, wherein the delay interval data is used to characterize the actual time range corresponding to the current video display;

[0327] Based on the overall length of the video display delay interval, the time period division rules for trend analysis are determined, so that the delay interval is divided into multiple consecutive sub-time periods.

[0328] According to the time period division rules, the video display delay interval is divided into segments to form multiple continuous and non-overlapping delay time periods;

[0329] For each delay period, a corresponding time period identifier is generated to form delay period data that describes the time structure within the delay interval, and the delay period data is output to the rate change interval matching subunit.

[0330] In a preferred embodiment of the present invention, the rate change interval matching subunit is used to match the rate change within a corresponding time period from the pose change rate data based on the delay time period data, and generate interval rate data, specifically including:

[0331] Receive delay time period data output by the delay interval time period division sub-unit, and pose change rate data output by the pose change rate calculation unit;

[0332] Based on the time range corresponding to each delay period, filter out the rate change information that matches the time period from the pose change rate data;

[0333] The selected rate change information is organized to reflect the speed change of the sampling probe within the corresponding delay time period;

[0334] The rate change within each delay period is output as a set of interval rate data and transmitted to the dominant change trend extraction subunit.

[0335] In a preferred embodiment of the present invention, the dominant change trend extraction subunit is used to compare and analyze the interval rate data, extract the dominant pose change trend that persists within the delay interval, and generate dominant trend data, specifically including:

[0336] Receive interval rate data output by the rate change interval matching subunit, wherein the interval rate data includes the pose change rate within multiple delay time periods;

[0337] By comparing the interval rate data within different delay time periods, the trend of pose change rate within the delay interval can be determined.

[0338] Identify rate changes that maintain a consistent direction and characteristics of change over multiple consecutive time delay periods, and determine this rate change as the dominant pose change trend;

[0339] The determined dominant pose change trends are organized into dominant trend data, and the dominant trend data is output to the trend continuity verification subunit.

[0340] In a preferred embodiment of the present invention, the trend continuity verification subunit is used to perform continuity verification on the dominant trend data between adjacent delay time periods, eliminate discontinuous trend changes, and generate continuous trend data, specifically including:

[0341] Receive dominant trend data output by the dominant trend extraction sub-unit;

[0342] Compare the dominant trend data corresponding to adjacent delay time periods to determine the continuity of the dominant trend over time;

[0343] When the dominant trend in a certain time period is found to be inconsistent with the trend direction or change characteristics in the previous time period, the trend change is judged as a discontinuous trend.

[0344] Remove the data corresponding to the discontinuous trends and retain only the dominant trend data that remains consistent within adjacent time periods;

[0345] The dominant trend data, after continuous verification, is output as continuous trend data to the pose change trend data generation sub-unit.

[0346] In a preferred embodiment of the present invention, the pose change trend data generation subunit is used to construct pose change trend data characterizing the overall motion trajectory of the sampling probe within the delay interval based on continuous trend data, specifically including:

[0347] Receive continuous trend data output by the trend continuity verification subunit;

[0348] A comprehensive analysis of continuous trend data is performed to determine the main motion trajectory of the sampling probe throughout the entire delay interval;

[0349] The main motion trends are organized into a trend description that reflects the overall motion changes of the sampling probe;

[0350] The trend description is output as pose change trend data and passed to the displacement change cumulative estimation subunit for subsequent delayed displacement prediction processing.

[0351] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A remote control system for a visual sampling interactive probe, characterized in that, The system includes: The visualization target input module is used to acquire real-time video collected by the front end of the sampling probe and generate an interactive target selection interface in the real-time video to receive the sampling target position specified by the operator in the video screen and form initial target position data. The field-of-view consistency correction module is used to correct the field-of-view changes of the initial target position data based on the current pose state data returned by the sampling probe, and generate field-of-view corrected target position data. The delay displacement compensation module is used to predict the spatial displacement of the sampling probe within the display time difference based on the real-time video display time difference, and to perform displacement compensation processing on the field of view correction target position data to generate time calibration target position data. The target pose analysis module is used to calibrate the target position data according to time, analyze the target pose parameters required for the sampling probe to reach the corresponding sampling target, and generate target pose data. The control command construction module is used to construct probe control commands for controlling the rotation, extension, and sampling posture adjustment of the sampling probe based on the target pose data. The remote execution feedback module is used to send probe control commands to the actuator of the sampling probe and obtain the pose status data of the sampling probe after execution, so that the field of view consistency correction module can continuously update and process it.

2. The remote control system for the visual sampling interactive probe according to claim 1, characterized in that, The field-of-view consistency correction module includes: The pose state sequence construction unit is used to acquire pose state data returned by the sampling probe in multiple consecutive control cycles and construct pose state sequence data in chronological order. The pose change feature extraction unit is used to perform adjacent cycle comparison processing on pose state sequence data, extract the motion direction and motion amplitude features of the sampling probe in continuous control cycle, and generate pose change feature data. The field-of-view change mapping unit is used to map pose change feature data to corresponding field-of-view change feature data based on the structural parameters of the sampling probe and the camera viewing angle parameters. The field of view offset determination unit is used to determine the field of view offset data caused by the movement of the sampling probe in the video frame based on the field of view change characteristic data; The target position correction generation unit is used to perform reverse correction processing on the initial target position data based on the field of view offset data to generate field of view corrected target position data.

3. The remote control system for the visual sampling interactive probe according to claim 1, characterized in that, The time-delay displacement compensation module includes: The display time difference acquisition unit is used to acquire the display time difference data corresponding to the time difference from the completion of real-time video acquisition by the sampling probe to the completion of display on the visualization interface. The historical pose data caching unit is used to cache the pose state data of the sampling probe in multiple control cycles within the time range corresponding to the display time difference, forming a historical pose data set; The pose change rate calculation unit is used to perform time correlation analysis on the historical pose data set to determine the pose change rate data of the sampling probe per unit time. The time-delay displacement range prediction unit is used to predict the spatial displacement range data generated by the sampling probe during the display time difference based on the display time difference data and the pose change rate data. The target position displacement compensation unit is used to perform displacement compensation processing on the field-of-view corrected target position data based on the spatial displacement range data, and generate time-calibrated target position data.

4. The remote control system for the visual sampling interactive probe according to claim 1, characterized in that, The target pose analysis module includes: The spatial relative relationship construction unit is used to construct the spatial relative relationship data between the current position of the sampling probe and the sampling target position based on the time calibration target position data and the current pose state data. The pose adjustment requirement decomposition unit is used to decompose the pose adjustment requirements required for the sampling probe to reach the sampling target into rotation adjustment requirement data, telescopic adjustment requirement data, and sampling posture adjustment requirement data based on spatial relative relationship data. The pose execution sequence generation unit is used to plan the sequence of rotation adjustment requirement data, telescopic adjustment requirement data and sampling posture adjustment requirement data according to the motion constraints of the sampling probe, and generate pose execution sequence data. The target pose data generation unit is used to combine and process the pose adjustment requirement data according to the pose execution sequence data to generate the target pose data.

5. The remote control system for the visual sampling interactive probe according to claim 2, characterized in that, The pose change feature extraction unit includes: The pose difference sequence generation subunit is used to compare the pose state data corresponding to adjacent control cycles in the pose state sequence data one by one to generate pose difference sequence data that characterizes the pose change between each control cycle. The difference stability filtering subunit is used to filter pose changes that recur within multiple consecutive control cycles based on pose difference sequence data, and generate stable pose difference data. The dominant change direction determination subunit is used to determine the dominant change direction of the sampling probe in the current time period based on the stable pose difference data, and generate dominant direction data. The variation amplitude level classification subunit is used to classify the variation amplitude according to the distribution of the magnitude of the variation in the stable pose difference data, and generate variation amplitude level data. The pose change feature construction subunit is used to combine the dominant direction data with the change amplitude level data to generate pose change feature data that characterizes the motion state of the sampling probe.

6. The remote control system for the visual sampling interactive probe according to claim 2, characterized in that, The field-of-view transformation mapping unit includes: The mapping parameter standardization subunit is used to obtain the structural parameters of the sampling probe and the camera viewing angle parameters, and perform uniform scaling to generate standardized mapping parameter data; The pose-field association construction subunit is used to establish a correspondence between pose change feature data and field of view changes in video images based on standardized mapping parameter data, and generate pose-field association data. The mapping weight adjustment subunit is used to adjust the weight of the pose-field association data according to the difference in the degree of influence of the sampling probe on the video field of view under different pose change directions and different change amplitude levels, and generate weighted association data. The field-of-view change feature generation subunit is used to convert pose change feature data into field-of-view change feature data corresponding to the video frame based on weighted correlation data. The field-of-view change continuity verification subunit is used to verify the continuity of the field-of-view change feature data generated within a continuous control cycle, and generate stable field-of-view change feature data.

7. The remote control system for the visual sampling interactive probe according to claim 3, characterized in that, The pose change rate calculation unit includes: The time series marker subunit is used to generate time markers for the pose state data corresponding to each control cycle in the historical pose data set, forming pose time series data. The pose change sequence generation subunit is used to compare and process the pose state data corresponding to adjacent time markers in the pose time series data to generate pose change sequence data. The rate candidate generation subunit is used to generate multiple candidate pose change rate data based on the pose change sequence data and the corresponding time interval data. The rate fluctuation suppression subunit is used to perform continuous analysis on candidate pose change rate data, suppress abnormal change rates, and generate smooth rate data. The effective rate determination subunit is used to determine the pose change rate data characterizing the current motion state of the sampling probe from the smoothed rate data.

8. The remote control system for the visual sampling interactive probe according to claim 3, characterized in that, The time-delay displacement range prediction unit includes: The display delay interval parsing subunit is used to parse the corresponding video display delay interval data based on the display time difference data. The delay interval pose trend construction subunit is used to construct the pose change trend data of the sampling probe within the delay interval based on the video display delay interval data and pose change rate data; The displacement change cumulative estimation subunit is used to accumulate and calculate the displacement change generated by the sampling probe within the delay interval based on the pose change trend data, and generate cumulative displacement data. The displacement range constraint correction subunit is used to constrain and correct the cumulative displacement data according to the structural limitations and motion constraints of the sampling probe, and generate spatial displacement range data.

9. The remote control system for the visual sampling interactive probe according to claim 6, characterized in that, The pose-field-of-view association construction subunit includes: The field-of-view partitioning subunit is used to divide the video image into multiple field-of-view regions based on the imaging range and display structure of the video image, and generate field-of-view region data. The pose change component analysis subunit is used to perform component analysis on pose change feature data, extract the direction change component and amplitude change component corresponding to the field of view data, and generate pose change component data. The region association rule generation subunit is used to establish the correspondence rules between the pose change component data and each field of view region based on the standardized mapping parameter data, and generate region association rule data. The multi-region influence overlay subunit is used to overlay the corresponding regional association rule data when pose change components act on multiple field of view regions simultaneously, and generate comprehensive association data. The pose-field-of-view correlation data generation subunit is used to determine the manifestation of field-of-view changes caused by pose change feature data in the video frame based on the comprehensive correlation data, and generate pose-field-of-view correlation data.

10. The remote control system for the visual sampling interactive probe according to claim 8, characterized in that, The time-delay interval pose trend construction subunit includes: The delay interval time segmentation sub-unit is used to divide the delay interval into multiple consecutive time segments based on the video display delay interval data, and generate delay time segment data; The rate change interval matching subunit is used to match the rate change within the corresponding time period from the pose change rate data based on the delay time period data, and generate interval rate data. The dominant change trend extraction sub-unit is used to compare and analyze interval rate data, extract the dominant pose change trend that persists within the delay interval, and generate dominant trend data. The trend continuity verification subunit is used to verify the continuity of the dominant trend data between adjacent time delay periods, eliminate discontinuous trend changes, and generate continuous trend data. The pose change trend data generation subunit is used to construct pose change trend data to characterize the overall motion direction of the sampling probe within the delay interval based on continuous trend data.

Images