Reinforcement learning-based kvm switch seamless switching control system and method

Abstract

This invention relates to the field of KVM switching control and intelligent video transmission, specifically to a seamless switching control system and method for KVM switches based on reinforcement learning. The system includes: collecting physical layer state data of the current data link and the target data link, including cable impedance attenuation values, bit error rate before adaptive equalizer compensation, historical timing data readout time, estimated environmental signal-to-noise ratio, and terminal configuration mode; inputting the physical layer state data into a reinforcement learning decision engine based on a deep Q-network, outputting link compensation parameters and pre-loaded timing parameters; performing a virtual handshake operation based on the pre-loaded timing parameters to generate a source-end link hold state; completing the reconstruction of underlying physical layer parameters and cross-matrix switching in the vertical blank area of ​​the video stream, and updating the model reward value based on the retrained state of the switched link; this invention achieves adaptive seamless switching control for high-speed video links such as display ports.

Description

Technical Field

[0001] This invention relates to the field of KVM switching control and intelligent video transmission, specifically to a KVM switch seamless switching control system and method based on reinforcement learning. Background Technology

[0002] In air traffic control automation, command and dispatch, and high-reliability display control scenarios, KVM switches are typically used to switch video streams received by display terminals between multiple data sources to meet the requirements of primary / backup host switching and continuous service display. The stability of the switching process is subject to the physical layer status of the data link, the display handshake timing, and the accuracy of the switching timing control.

[0003] In existing technical solutions, KVM switches mostly perform link connection and port switching according to preset fixed parameters. When switching to the target data source, it is often necessary to retrain the display link, read the display capability, and adapt the physical layer parameters. When the link status fluctuates, the cable attenuation exceeds the preset tolerance range, or the terminal compatibility difference does not match the preset parameters, problems such as black screen, resolution degradation, frame loss, or switching latency exceeding the latency threshold are still likely to occur, resulting in low video stream continuity. Summary of the Invention

[0004] The purpose of this invention is to provide a reinforcement learning-based seamless switching control system and method for KVM switches, addressing the following technical problems: Existing traditional KVM switching technologies have significant shortcomings in adapting physical layer parameters and controlling the target link's handshake timing when faced with link state fluctuations, large cable attenuation, or significant differences in terminal compatibility. There is an urgent need to propose a reinforcement learning-based seamless switching control method and system for KVM switches that can adaptively correct link compensation strategies based on multi-dimensional physical layer states, and combine virtual handshakes with dynamic reconstruction of vertical blank areas to ensure continuous and seamless switching of video streams. The purpose of this invention can be achieved through the following technical solutions:

[0005] On one hand, this invention provides a seamless switching control method for KVM switchers based on reinforcement learning. The KVM switcher connects multiple data sources through an internal cross matrix and receives video streams from the current data source, including:

[0006] Collect physical layer status data of the current data link corresponding to the current data source and the target data link corresponding to the target data source; among which, the physical layer status data includes cable impedance attenuation value, bit error rate before adaptive equalizer compensation, historical time sequence reading time, environmental signal-to-noise ratio estimation value, and terminal configuration mode;

[0007] The environmental signal-to-noise ratio (SNR) estimate is obtained by weighting the received level jitter amplitude and the number of bit error spikes per unit time. The physical layer state data is input into a reinforcement learning decision engine based on a deep Q-network for state space mapping, and the output is link compensation parameters and pre-loading timing parameters. The link compensation parameters include signal pre-emphasis parameters and voltage swing parameters.

[0008] Based on preloaded timing parameters, a virtual handshake operation is performed between the target data link and the target data source to generate a source-end link hold-up state.

[0009] Receives a switching trigger command for the target data source. In response to the switching trigger command, reconstructs the underlying physical layer parameters of the KVM switcher based on the link compensation parameters in the vertical blank area of ​​the video stream of the current data source and performs a switching operation of the cross matrix. Obtains the link retraining status after switching by polling the link status register of the underlying physical interface of the KVM switcher.

[0010] The model reward value is calculated based on the retraining state of the link after the switch, and the model reward value is used to update the reinforcement learning decision engine.

[0011] Optionally, based on preloaded timing parameters, a virtual handshake operation is performed between the target data link and the target data source to generate a source-end link hold-up state, including: extracting the target timing dataset corresponding to the preloaded timing parameters from a preset hardware timing cache module; using the target timing dataset, sending a terminal simulated online signal to the target data source through the target data link; receiving a video stream hold-up state signal returned by the target data source in response to the terminal simulated online signal; and confirming the video stream hold-up state signal as the source-end link hold-up state.

[0012] Optionally, in response to a switching trigger command, the underlying physical layer parameters of the KVM switch are reconstructed based on link compensation parameters in the vertical blank area of ​​the video stream of the current data source, and a cross matrix switching operation is performed. This includes: parsing the switching trigger command to determine the target switching clock; writing the signal pre-emphasis parameters and voltage swing parameters into the underlying physical interface control core of the KVM switch in the vertical blank area of ​​the video stream, wherein the underlying physical interface control core is a hardware logic control unit containing a transmitter drive strength configuration register and a timing control register to complete the reconstruction of the underlying physical layer parameters; and triggering the port mapping update of the cross matrix according to the target switching clock to complete the cross matrix switching operation.

[0013] Optionally, the model reward value is calculated based on the retraining state of the link after the switch, including: if the retraining state of the link after the switch is a state where retraining is not triggered and no video frames are lost, the model reward value is set to a first preset reward value; if the retraining state of the link after the switch is a state where the resolution is downgraded and output, the model reward value is set to a second preset reward value; if the retraining state of the link after the switch is a state where the screen is black and the connection is interrupted or other states, the model reward value is set to a third preset reward value; wherein, the first preset reward value is greater than the second preset reward value, and the second preset reward value is greater than the third preset reward value.

[0014] Optionally, the reinforcement learning decision engine is updated using the model reward value, including: collecting physical layer state data of the target data link after the switch as the physical layer state data after the switch; constructing an experience replay tuple from the physical layer state data before the switch, the physical layer state data after the switch, the link compensation parameters, the preloaded timing parameters, and the model reward value; storing the experience replay tuple in a preset experience replay pool; extracting training batch data from the experience replay pool; calculating the temporal difference error using the training batch data; and updating the network weight parameters of the reinforcement learning decision engine based on the temporal difference error.

[0015] Optionally, the current data link and the target data link are display port data links, and the preloaded timing parameters are extended display identifier data timing parameters.

[0016] On the other hand, this invention provides a reinforcement learning-based seamless switching control system for KVM switches, applied to a KVM switch that connects multiple data sources via an internal cross matrix and receives video streams from the current data source, comprising:

[0017] The multi-dimensional state awareness module is used to collect physical layer state data of the current data link corresponding to the current data source and the target data link corresponding to the target data source. The physical layer state data includes cable impedance attenuation value, bit error rate before adaptive equalizer compensation, historical time series reading time, environmental signal-to-noise ratio estimation value, and terminal configuration mode.

[0018] The reinforcement learning decision module is used to input physical layer state data into the deep Q-network-based reinforcement learning decision engine for state space mapping and output link compensation parameters and preload timing parameters; among which, the link compensation parameters include signal pre-emphasis parameters and voltage swing parameters.

[0019] The pre-handshake processing module is used to perform a virtual handshake operation with the target data source through the target data link based on pre-loaded timing parameters, and generate the source link hold-up state.

[0020] The dynamic reconstruction and execution module is used to receive the switching trigger command for the target data source. In response to the switching trigger command, it reconstructs the underlying physical layer parameters of the KVM switcher based on the link compensation parameters in the vertical blank area of ​​the video stream of the current data source and performs the switching operation of the cross matrix. It obtains the link retraining status after the switch by polling the link status register of the underlying physical interface of the KVM switcher.

[0021] The feedback adaptive module is used to calculate the model reward value based on the retraining state of the link after the switch, and to use the model reward value to update the reinforcement learning decision engine.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. This invention collects multi-dimensional physical layer state data, such as cable impedance attenuation values ​​and bit error rate before adaptive equalizer compensation, and uses a reinforcement learning decision engine to perform state space mapping to dynamically output link compensation parameters, including signal pre-emphasis parameters and voltage swing parameters. This mechanism can adaptively correct the compensation strategy in real time according to the actual physical layer state, effectively reducing the risk of signal mismatch and improving the continuity of video streams.

[0024] 2. This invention performs a virtual handshake operation with the target data source based on preloaded timing parameters; by sending a terminal simulated online signal to the target data source and receiving the corresponding video stream maintenance command, the source link hold state can be generated before the actual handover occurs; this enables the target data source to continuously maintain a stable video stream output, avoiding the re-probing and initialization delay at the moment of actual handover;

[0025] 3. This invention reconstructs the underlying physical layer parameters of the KVM switcher within the vertical blank area of ​​the video stream and performs port mapping update and switching operations of the cross matrix. Since the vertical blank area is in the inter-frame gap, completing the underlying parameter reconstruction and matrix flipping at this time will not interrupt the effective display area of ​​the pixels, effectively reducing the probability that the display terminal will misjudge the switching action as an abnormal interruption, and achieving a visually seamless switching effect without obvious interference.

[0026] 4. This invention calculates the corresponding model reward value based on the retraining state of the link after switching; by storing relevant data in the experience replay pool to extract training batches, and calculating the time difference error to update the network weight parameters of the reinforcement learning decision engine, the system can continuously learn from historical experience during long-term operation and gradually optimize the switching strategy under complex environment and aging cable conditions. Attached Figure Description

[0027] The present invention will be further explained below with reference to the accompanying drawings and embodiments:

[0028] Figure 1This is a flowchart of the method of the present invention;

[0029] Figure 2 This is a structural diagram of the system of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0031] Example 1:

[0032] Please see Figure 1 A reinforcement learning-based seamless switching control method for KVM switches. The KVM switch connects multiple data sources through an internal cross matrix and receives video streams from the current data source, including:

[0033] Collect physical layer status data of the current data link corresponding to the current data source and the target data link corresponding to the target data source; among which, the physical layer status data includes cable impedance attenuation value, bit error rate before adaptive equalizer compensation, historical time sequence reading time, environmental signal-to-noise ratio estimation value, and terminal configuration mode;

[0034] The environmental signal-to-noise ratio (SNR) estimate is obtained by weighting the received level jitter amplitude and the number of bit error spikes per unit time. The physical layer state data is input into a reinforcement learning decision engine based on a deep Q-network for state space mapping, and the output is link compensation parameters and pre-loading timing parameters. The link compensation parameters include signal pre-emphasis parameters and voltage swing parameters.

[0035] Based on preloaded timing parameters, a virtual handshake operation is performed between the target data link and the target data source to generate a source-end link hold-up state.

[0036] Receives a switching trigger command for the target data source. In response to the switching trigger command, reconstructs the underlying physical layer parameters of the KVM switcher based on the link compensation parameters in the vertical blank area of ​​the video stream of the current data source and performs a switching operation of the cross matrix. Obtains the link retraining status after switching by polling the link status register of the underlying physical interface of the KVM switcher.

[0037] The model reward value is calculated based on the retraining state of the link after the switch, and the model reward value is used to update the reinforcement learning decision engine.

[0038] This embodiment provides a seamless switching control mechanism for KVM switches based on reinforcement learning. Specifically, this mechanism is deployed in the main / backup host switching scenario of an air traffic control automation system. The input side of the KVM switch is connected to host A, host B, and the backup verification host, which serve as data sources, respectively, and the output side is connected to the tower position display terminal. At the current moment, the display terminal is receiving the radar situation video stream output from the current data source, such as host A. When the system is ready to switch to host B, which serves as the target data source, the switch does not directly disconnect and connect. Instead, it first senses the physical layer status of the current link and the target link, and then the reinforcement learning decision engine provides the link compensation parameters and preloaded timing parameters for this switch. The virtual handshake of the target link is completed in advance, and cross-matrix switching and underlying physical layer reconstruction are performed in the vertical blank area of ​​the video stream. The switching result is then used in reverse for model weight updates.

[0039] Specifically, the multi-dimensional state perception module simultaneously reads the multi-dimensional state of the current data link and the target data link. The cable impedance attenuation value can be estimated from the amplitude attenuation difference before and after equalization training at the receiving end. The bit error rate before adaptive equalizer compensation can be obtained from the bit error statistics within a preset sampling window. The historical time sequence reading time can be formed by the average time taken for several previous EDID or auxiliary channel readings. The environmental signal-to-noise ratio estimate can be calculated based on received level jitter and bit error spikes, i.e., through a preset signal quality assessment model, using the received level jitter amplitude and the number of bit error spikes per unit time as input variables for weighted calculation. The weights in the weighted calculation are allocated based on preset channel attenuation prior experience, and the weight of the received level jitter amplitude is greater than the weight of the number of bit error spikes. The terminal configuration mode reflects which mode the terminal is currently in: pass-through mode, compatible mode, or fixed template mode.

[0040] For ease of explanation, a single sampling can be abstracted as a five-dimensional state vector. This vector sequentially represents an impedance attenuation level of 8, a pre-compensation bit error rate of 3, a historical read time of 12ms, a signal-to-noise ratio estimation level of 18, and a terminal configuration mode of 1, i.e., automatic pass-through mode; another target link state can be abstracted into another five-dimensional state vector. In practice, the above data can be directly concatenated to form a unified state vector, or it can be grouped by link first, then normalized and input into the reinforcement learning decision engine.

[0041] The reinforcement learning decision engine maps the state space. In a specific control scenario example, four sets of candidate actions can be predefined: action A1 corresponds to pre-emphasis level 1, voltage swing level 1, and preloaded timing template T1; action A2 corresponds to pre-emphasis level 2, voltage swing level 1, and template T2; action A3 corresponds to pre-emphasis level 2, voltage swing level 2, and template T2; and action A4 corresponds to pre-emphasis level 3, voltage swing level 2, and template T3. When the input state is... At that time, the deep Q-network outputs four Q-values, for example Then, the action with the highest Q value, A2, is selected, resulting in signal pre-emphasis parameters of level 2, voltage swing parameters of level 1, and pre-loading timing parameters of template T2. Here, template T2 corresponds to a set of timing control fields that can actually be written to the buffer or register, such as auxiliary channel start delay, EDID response interval, and hot-plug hold time.

[0042] A deep Q-network consists of an input layer, at least one fully connected hidden layer, and an output layer connected in sequence. The number of nodes in the input layer is consistent with the dimension of the state vector formed by the physical layer state data, and the number of nodes in the output layer is consistent with the size of the preset candidate action space. The candidate action space is composed of discrete combinations of link compensation parameters and preloaded timing parameters.

[0043] After obtaining the preloaded timing parameters, the system performs a virtual handshake operation with the target data source through the target data link. The control principle is as follows: before the formal switch, the target data source continuously recognizes that the display terminal is online and outputs a stable video stream according to the selected template. In this way, when the cross matrix is ​​actually switched to the target data source port, the target data source is already in a state of continuous video stream output, which can avoid restarting the link training or display mode negotiation. After the virtual handshake is completed, the system forms a source-end link hold state, which indicates that the target data source side has established a stable output ready state.

[0044] The system receives a switching trigger command; this command can come from the keyboard, network management console, or upper-layer automated disaster recovery logic. In response to this command, the system does not switch immediately at any time, but waits for the current video stream to enter the vertical blank area. Within this time window, the signal pre-emphasis parameters and voltage swing parameters obtained from reinforcement learning decisions are first written into the underlying physical interface control core, and then the cross-matrix port mapping is switched so that the output end transitions from host A to the target data source host B. Since the vertical blank area is in the inter-frame gap, the effective display area of ​​the pixels is not interrupted, thus reducing the probability that the display interprets the switching as an abnormal interruption. After the switching is completed, the system reads the retraining status by polling the link status register of the underlying physical interface, such as whether it has re-entered the training sequence, whether symbol lock loss has occurred, and whether resolution regression has occurred.

[0045] The model reward value is calculated based on the state after the switch and used to update the reinforcement learning decision engine. For example, no retraining and no frame loss can be recorded as a high reward, resolution degradation but no black screen can be recorded as a medium reward, and black screen reconnection or link disconnection can be recorded as a low reward. If a switch results in a high reward, the utility value of this state and action combination in the experience pool will increase, and it will be more likely to be selected again when encountering similar states in the future. Conversely, if a certain link compensation combination frequently causes black screens, the decision engine will gradually reduce the Q value of that action in similar states.

[0046] In the anomaly handling mechanism, if a certain physical quantity is temporarily unavailable during the acquisition phase, such as the loss of the environmental signal-to-noise ratio estimation sensing window, the system can fill that dimension with the previous stable sampling value, or fill it with a preset median and mark it as missing, to avoid the entire decision-making process being blocked due to the absence of a single dimension; if the virtual handshake is not successfully established within a predetermined time, the system can maintain the current link without switching and roll back the candidate action to a conservative parameter set, such as a pre-emphasis parameter higher than the current level and a compatible template, to reduce the risk of black screen caused by triggering a switch when a stable state has not been established; if the vertical blank area does not appear within the preset waiting window, it can be delayed by one video frame period for detection again, instead of forcibly entering the active display area for switching;

[0047] During the high-load operation phase of the air traffic control automation system, the station terminal is currently displaying the track fusion image of host A. The system detects that the link impedance attenuation value from the target data source host B to the display terminal is greater than or equal to the first preset threshold, the auxiliary channel historical reading time is greater than or equal to the second preset time threshold, and the environmental interference signal-to-noise ratio is within the preset range. Based on this, the decision engine selects a timing template with medium pre-emphasis, low voltage swing, and higher compatibility. First, the target data source detects that the display terminal is online and maintains video stream output. Then, the switch is completed between the end of the current frame and the beginning of the next frame. After the switch, the register shows that the link training has not been re-entered, the display terminal has no black screen and the track symbols are continuously refreshed. Therefore, the system records this result as a high-reward sample.

[0048] The purpose of this step is to organize the physical layer state perception, parameter decision-making, pre-handshake, vertical blank area switching, and result feedback into a closed-loop control chain, so that the switcher no longer operates solely based on static factory parameters, but continuously corrects the compensation strategy according to the actual state of the link, thereby achieving adaptive and seamless switching control of high-speed video links such as display ports.

[0049] Based on preloaded timing parameters, a virtual handshake operation is performed between the target data link and the target data source to generate a source-end link hold-up state, including:

[0050] Extract the target timing dataset corresponding to the preloaded timing parameters from the preset hardware timing cache module;

[0051] Using the target time-series dataset, simulated online signals from the terminal are sent to the target data source via the target data link.

[0052] Receive the video stream sustain status signal returned by the target data source in response to the terminal's simulated online signal;

[0053] The video stream sustain status signal is confirmed as the source link sustain status.

[0054] This embodiment provides a virtual handshake and source link maintenance mechanism. Specifically, when relying solely on the overall process of the previous embodiment, although the system can select compensation parameters for the target link, if the target data source is not notified in advance that the display terminal is online before the formal switch, its graphics card output may still be in a low-power waiting or re-probing state. Once the actual switch occurs, the target data source still needs to re-sensitize hot-plugging and display timing, which can easily lead to a surge in internal switching time and cause a long black screen. Therefore, this embodiment further introduces a hardware timing cache module and a terminal simulated online signal mechanism to ensure that the target data source maintains stable power delivery before the switch.

[0055] Specifically, the hardware timing cache module pre-stores multiple sets of target timing datasets, each corresponding to a type of display terminal handshake preference. For example, dataset D1 is suitable for standard-compatible terminals, dataset D2 is suitable for high-resolution terminals that are more sensitive to the auxiliary channel response interval, and dataset D3 is suitable for terminals that do not support the target display protocol version. For ease of explanation, it is assumed that the preloaded timing parameter is identified as T2. The system then extracts dataset D2 associated with T2 from the cache module. This dataset may contain fields such as a hot-plug hold pulse width of 5ms, an auxiliary channel first packet response delay of 2ms, a subsequent packet interval of 1ms, and a virtual online maintenance period of 20ms.

[0056] After extraction, the control logic uses the timing field in D2 to send a simulated online terminal signal to the target data source via the target data link. This signal can be composed of the hot-plug detection level, the auxiliary channel response rhythm, and the EDID response content in the buffer. It is not achieved by simply pulling a single pin high, but by continuously presenting the state that the terminal has been connected and can respond normally in the timing sequence. To illustrate with a specific timing interaction example: when the target data source sends the first display capability probe, the buffer module returns the first packet of EDID data after 2ms; when the target data source sends the second read request, it returns the subsequent data after 1ms; at the same time, the hot-plug online state remains unchanged. After the above timing interaction, the target data source will continue to maintain its video stream transmission link state and will not re-initialize completely at the moment of formal switching.

[0057] Next, the system receives the video stream maintenance instruction returned by the target data source. The video stream maintenance instruction can be manifested as the target data source maintaining the TMDS / DP main link activity, continuously sending blank frames or normal image frames, and not canceling the previously negotiated display mode, etc. After the control logic reads these states, it confirms them as the source link maintenance state. After that, even if the output end has not yet switched to the target data source, the target data source side is already in a stable video stream output state that can immediately take over the display.

[0058] In the exception handling mechanism, if there is no dataset in the cache module that perfectly matches the current preloaded timing parameters, the system can execute the adjacent template rollback strategy; for example, if T2 is missing, T1 or T3 with the closest timing parameters will be selected first; if the adjacent template is also unavailable, the system will enter the general template and limit the output resolution range to improve the handshake success rate; if the terminal has sent a simulated online signal but has not received a video stream maintenance instruction within a preset period, the system can repeat the handshake once; if it still fails after repetition, the system will report that the target data source has not entered the hold state and will prohibit entering the formal switching phase; if the received maintenance instruction conflicts with the current terminal configuration mode, for example, the terminal is set to a fixed resolution template, but the target data source returns an incompatible timing negotiation result, the system will prioritize maintaining the current display link to prevent an illegal display mode from occurring after the switch.

[0059] In the same air traffic control position, if the target data source of host B is connected to a graphics card that is sensitive to the timing of EDID reading; after the system selects the T2 template according to the preceding decision, it retrieves the D2 dataset from the cache, continuously presents a hot-plug online status to the target data source, and replies with display capability information according to the response interval set by D2; the target data source then maintains its radar video stream output and no longer waits for the actual physical switch; when the subsequent switch action occurs, the target data source has already been continuously sending streams, so the display terminal can directly receive the complete first frame;

[0060] The purpose of this mechanism is to complete the display link preparation on the target data source side in advance, and move the handshake work that must be completed before the formal switch to the background, thereby suppressing the delay of re-probing and output.

[0061] In response to a switching trigger command, the underlying physical layer parameters of the KVM switcher are reconstructed based on link compensation parameters in the vertical blank area of ​​the video stream from the current data source, and a cross-matrix switching operation is performed, including:

[0062] Parse the switching trigger command to determine the target switching clock;

[0063] Within the vertical blank area of ​​the video stream, the signal pre-emphasis parameters and voltage swing parameters are written into the underlying physical interface control core of the KVM switch. This underlying physical interface control core is a hardware logic control unit that includes a transmitter drive strength configuration register and a timing control register to complete the reconstruction of the underlying physical layer parameters.

[0064] The port mapping update of the cross matrix is ​​triggered based on the target switching clock, thus completing the cross matrix switching operation.

[0065] This embodiment provides a dynamic reconstruction and switching execution mechanism within a vertical blank area. Specifically, based on the existing pre-handshake, if the switching action occurs in the display activity area, even if the target host is already in a hold state, the display terminal may still trigger an abnormal judgment due to the pixel stream in the current frame being interrupted midway, manifesting as display flickering, screen tearing, or momentary no signal output. To solve this problem, this embodiment further limits the switching to occur within the vertical blank area and completes the underlying physical layer parameter reconstruction and cross matrix port mapping update within the same time window.

[0066] Specifically, the system first parses the switching trigger command to determine the target switching clock. This target switching clock is not directly equivalent to the moment the command is received, but rather an executable time point calculated by combining the frame synchronization information of the current video stream. In a specific application scenario example, assuming the current video stream is 60Hz, with each frame period of approximately 16.67ms, and the current frame is at 14ms, the system detects that the starting point of the next vertical blank area is expected to be at 16.50ms, so the target switching clock is set to 16.52ms. If the remaining width of the vertical blank area in this frame is insufficient to complete the register writing and cross matrix flipping, it will automatically be postponed to the vertical blank area of ​​the next frame.

[0067] After the vertical blank area arrives, the system first writes the link compensation parameters into the underlying physical interface control core. For ease of explanation, it is assumed that the selected parameters are pre-emphasis level 2 and voltage swing level 1. The register group related to the driving strength of the transmitting end in the control core is updated first, and then the state machine confirms that the parameter latching is complete. The writing here usually adopts the method of configuring the shadow register to be effective first and then submitting it uniformly to avoid unstable transition states caused by partial parameter updates. After the underlying physical layer reconstruction is completed, the system triggers the port mapping update of the cross matrix according to the aforementioned target switching clock. For example, the output port O1 is switched from the input I1 to the input I2. Since the physical layer transmission characteristics have been pre-calibrated to adapt to the target link state in the previous step, the matrix flipping in the next step will not cause the display terminal to receive a significantly mismatched signal amplitude and pre-emphasis combination.

[0068] A more detailed timeline can be used to illustrate this: Within the blank window of 16.50ms to 16.52ms, the pre-emphasis parameters are written from 16.500ms to 16.505ms, the voltage swing parameters are written from 16.505ms to 16.510ms, the latch confirmation is completed from 16.510ms to 16.515ms, and the port mapping update is triggered at 16.520ms. After the switch is completed, the first valid pixel of the next frame is output by the target host B and sent directly to the display terminal through the reconstructed physical layer.

[0069] In the exception handling mechanism, if the current frame synchronization information is unavailable when the switching trigger command arrives, such as a temporary loss of synchronization extraction, the system can adopt a two-level backoff strategy: try to predict the blank area using the average frame period obtained from the previous frame statistics; if the prediction confidence is insufficient, delay the switching and re-establish frame synchronization instead of forcibly executing within the unknown window; if a latching failure occurs during parameter writing, the system can terminate the current matrix flip, maintain the original host output, and avoid forming an intermediate state where the link parameters have changed but the signal source has not yet switched; if the target switching clock conflicts with the latest switching time required by the upper layer service, such as the disaster recovery requirement that it must be completed within the preset switching time limit, the system can enter emergency mode, prioritize the completion of the switching, and then compensate and optimize in subsequent frames, but at the same time mark this event as a risk sample for subsequent training;

[0070] On the tower console display, host A is currently outputting a complete frame of airspace situation image. After the console issues a command to switch to host B, the system resolves that there is still 2.5ms available for the next vertical blank area, which is sufficient to perform parameter updates and matrix flipping. Therefore, the control logic first writes the transmission parameters adapted to the host B link during this gap, and then completes the port mapping update at the predetermined clock point. The starting pixel of the new frame received by the display has come from host B, and there is no active area truncation in the image.

[0071] The purpose of this step is to unify parameter reconstruction and physical switching within the time window that is best synchronized with the display, thereby achieving the technical effect of reducing interference in the display activity area and reducing the probability of abnormal retraining triggers.

[0072] The model reward value is calculated based on the retraining state of the link after the switch, including:

[0073] If the retraining status of the link after the switch is that retraining has not been triggered and no video frames have been lost, the model reward value will be set to the first preset reward value.

[0074] If the retraining state of the link after switching is a resolution downgraded output state, set the model reward value to the second preset reward value;

[0075] If the retraining status of the link after switching is a black screen reconnection interruption status or other status, the model reward value will be set to the third preset reward value.

[0076] Among them, the first preset reward value is greater than the second preset reward value, and the second preset reward value is greater than the third preset reward value.

[0077] This embodiment provides a reward value grading mechanism for switching results. Specifically, in the aforementioned process, if the switching result is only broadly recorded as success or failure, the reinforcement learning decision engine will find it difficult to measure the quality difference between zero frame loss during weightless training and forced resolution degradation even though there is no black screen, which will weaken the learning process's ability to guide the optimal action. Therefore, this embodiment subdivides the link state after switching into at least three levels and assigns ordered reward values.

[0078] Specifically, after the system completes the switch, it classifies the state after the switch by combining the link status register, video frame counter and display mode monitoring results. The first category is the state where retraining has not been triggered and video frames have not been lost. At this time, the relevant registers do not show the retraining flag, the frame count is continuous, and the resolution and refresh rate remain unchanged. The model reward value can be set to the first preset reward value, such as +100. The second category is the state where the resolution is downgraded.

[0079] At this time, the monitor is not completely black, but the working mode has fallen back from the original high resolution to the downgraded secondary resolution, or the refresh rate has decreased. The reward value can be set to the second preset reward value, such as +20. The third type is the black screen reconnection interruption state or other abnormal state, such as the main link losing lock, no valid frames for a long time, re-enumerating the monitor, etc. The reward value can be set to the third preset reward value, such as -100.

[0080] To facilitate understanding, let's take the following specific application scenario as an example; assuming that in state Sx, the decision engine outputs action A2, and after the switch, the register shows no retraining, the frame count continuously increases from 1001 to 1002 and 1003, and the resolution remains 3840×2160, then a reward of +100 is obtained; if action A3 is output in state Sy, after the switch, although there is still a picture, the resolution drops from 3840×2160 to 1920×1080, then +20 is obtained; if action A1 is output in state Sz and a 2-second black screen occurs, then -100 is obtained; in this way, the three state results of optimal availability, degraded operation, and communication interruption are precisely quantified;

[0081] Furthermore, in addition to the three levels mentioned above, additional penalty weights can be set for the third type of abnormal state in the implementation; for example, if not only is the screen black, but the automatic recovery time exceeds the set threshold, then the additional value is subtracted from the third preset reward value; if it is only a brief loss of synchronization but the business layer does not observe the black screen, then it can be between the second and third levels; however, no matter how the subdivision method changes, its basic order remains that the first level is greater than the second level, and the second level is greater than the third level.

[0082] In one implementation for handling anomalies, if multiple state flags appear simultaneously after a switch, such as detecting both a brief retraining flag and a resolution degradation, the system can assign a lower reward based on the principle of prioritizing the worst-case scenario to avoid positive bias in the utility evaluation of the reinforcement learning decision engine. If the state acquisition is incomplete, such as a frame counter reset making it impossible to confirm whether frames are lost, the system can mark the sample as an uncertain sample and assign a conservative reward value to avoid introducing training noise that interferes with the convergence of the reinforcement learning decision engine. If long-term statistics show that the proportion of second-tier reward samples is too high on a certain type of terminal, the system can trigger a reorganization of the parameter templates, separating that terminal from the general template library for separate modeling.

[0083] During the switch from air traffic control position A to host B, after one switch, the monitor did not go black and the image continued to display, but the monitoring program detected that the output mode dropped from 4K to 1080p. At this time, the system did not classify the switch as completely successful, but instead recorded it as a resolution downgrade output and assigned a medium reward. In another switch, the image was completely continuous and the track refresh was uninterrupted, so a high reward was assigned. In yet another abnormal switch, the monitor briefly went black and re-identified the signal source, so a low reward was assigned. As a result, subsequent training will be more inclined to produce truly seamless action combinations.

[0084] The purpose of this mechanism is to transform the switching quality into learnable numerical feedback, so that the direction of network weight updates is consistent with the business objective of prioritizing visual continuity, thereby achieving quantifiable constraints on the actual display effect of the reinforcement learning process.

[0085] The reinforcement learning decision engine is updated using the model reward value, including:

[0086] Collect the physical layer status data of the target data link after the switch, and use it as the physical layer status data after the switch; construct the physical layer status data before the switch, the physical layer status data after the switch, the link compensation parameters, the preloaded timing parameters, and the model reward value into an experience replay tuple.

[0087] Store the experience replay tuple into the preset experience replay pool;

[0088] Draw training batch data from the experience replay pool;

[0089] The temporal difference error is calculated using training batch data, and the network weight parameters of the reinforcement learning decision engine are updated based on the temporal difference error.

[0090] This embodiment provides a feedback update mechanism based on experience replay. Specifically, if the model weights are adjusted immediately after each switch based solely on the current result, the weight update of the reinforcement learning decision engine is prone to overfitting, especially when the ambient noise in the air traffic control room changes in stages or individual cables experience occasional jitter, which can easily cause strategy oscillations. To reduce the accidental influence of a single sample, this embodiment constructs the switch process into experience replay tuples, stores them in the experience replay pool, and then samples them for training in batches.

[0091] Specifically, after each switch is completed, the system encapsulates the current state, action, and feedback into an experience replay tuple. This tuple contains at least four parts: first, the physical layer state data before the switch; second, the link compensation parameters used during execution; third, the selected preload timing parameters; and fourth, the model reward value obtained after the switch is completed.

[0092] In practical implementation, the new state after the switch and the termination flag can also be added; for ease of explanation, it is assumed that the experience replay pool stores three samples, which contain the state of the first link. Second link compensation action Second preload timing template and reward value First Experience Replay Tuple Includes second link state Third-link compensation action First preload timing template and reward value Second experience replay tuple ; and including third link state First link compensation action Third preload timing template and reward value The third experience replay tuple ;

[0093] These tuples are written to the experience replay pool; the experience replay pool can be a circular cache structure, overwriting the oldest sample when the storage capacity reaches its limit; for example, if the pool capacity is set to 10,000, and there are currently 9,999 samples, after the new E10,000 is written, the next time E10,001 is written, it overwrites the oldest E1; this ensures that the experience pool retains recent environmental characteristics while avoiding storage space overflow; during the training phase, the system draws a training batch from the experience replay pool, for example, randomly drawing 32 samples, instead of using only the most recent result; random sampling helps to break down temporal correlations;

[0094] For the extracted training batches, the system calculates the temporal difference error and updates the network weights accordingly. Specifically, the calculation logic of the temporal difference error is as follows: the model reward value in the experience replay tuple is added to the product of the environment discount factor and the maximum predicted Q value of the next state to construct the target Q value; then the predicted Q value corresponding to the current state and action is subtracted to obtain the temporal difference error.

[0095] Its rigorous formula for calculating time difference error is defined as follows:

[0096]

[0097] in, For time difference error, This is the model reward value. As an environmental discount factor, For the next state, The candidate action for the next state. This is the current state. For the currently executing action, It is a state-action utility function;

[0098] To facilitate understanding by those skilled in the art, the current prediction of a certain sample can be... The value is denoted as 30, and the target is obtained based on the above reward and next state estimation. If the value is recorded as 50, then the temporal difference error is 20; if the current prediction of another sample is 40 and the target is 10, then the error is -30; during network training, the parameters will be adjusted according to these errors, so that the utility value of the action corresponding to the former is increased and the utility value of the action corresponding to the latter is decreased; for the joint action of preloaded temporal parameters and link compensation parameters, a unified action index encoding can be used, for example, pre-emphasis level 2 + swing level 1 + template T2 is encoded as action number 7, which is convenient for indexing in the Q network output;

[0099] Furthermore, to accommodate rare but serious outliers, the experience replay pool can be assigned a higher extraction priority to low-reward events. Specifically, although black screen samples occur less frequently than the preset percentage, their impact on communication interruption is significant. Therefore, their probability of being selected can be increased during sampling to accelerate the convergence of the reinforcement learning decision engine to a better policy space during weight updates. Of course, this priority is not necessary, as long as the basic structure of the experience pool, batch sampling, and time difference error-based updates is maintained.

[0100] In one implementation for handling anomalies, if the number of samples in the experience replay pool is insufficient for a training batch (e.g., when the system is first launched and only 10 samples have been accumulated with a batch size of 32), the system can temporarily reduce the batch size or use warm-up parameters and should not force unstable training. If a batch contains more than a first preset proportion of samples from the same abnormal time period (e.g., electromagnetic interference in the computer room lasting less than a preset duration and exceeding a preset threshold causing multiple consecutive failures), the system can limit the sampling proportion of samples within the same time window to prevent the reinforcement learning decision engine from overfitting to transient anomalies. If the temporal difference error continues to diverge during training, the weight updates can be frozen and the system can revert to the most recent stable network version to ensure that the online switching strategy does not deteriorate due to training anomalies.

[0101] During a week of continuous operation at an air traffic control center, the system accumulated more than one preset percentage of samples switching from host A to host B and from host B to host A. Among these, most achieved zero-repetition training switching after optimization. Samples with a percentage in the second preset range experienced resolution degradation due to a section of aging cable. Samples with a percentage below the third preset percentage experienced black screens when electromagnetic interference increased at night. The system stored these samples in an experience replay pool and sampled them for training during off-peak hours. After multiple rounds of updates, the states related to aging cables and nighttime interference were more stably mapped to more suitable compensation parameters and timing templates.

[0102] The purpose of this mechanism is to continuously adjust the parameters of the decision engine by utilizing historical switching experience, so that the system can gradually adapt to cable aging, terminal differences and environmental changes during long-term operation, thereby achieving a balance between policy stability and adaptive capability.

[0103] The current data link and the target data link are display port data links, and the preloaded timing parameters are extended display identifier data timing parameters.

[0104] This embodiment provides a specific implementation mechanism for the display port data link. Specifically, in the aforementioned general scheme, the data link and preloaded timing parameters can be abstractly understood as the high-speed link between the video source and the terminal and its handshake timing. However, in actual engineering, if it is not further limited to the display port data link and its extended display identifier data timing, the implementation boundary of the technical solution will be too broad, which is not conducive to clarifying the interface object and cached content. Therefore, this embodiment specifically limits the current link and the target link to the display port data link, and specifically implements the preloaded timing parameters as the extended display identifier data timing parameters.

[0105] Specifically, in the display port data link, the main link is responsible for carrying video data, while the auxiliary channel is responsible for transmitting link configuration, display capability reading, and related control information. The extended display identifier data timing parameters describe the response time relationship and data organization method when the target host reads the display capability information. For ease of explanation, two sets of EDID timing templates can be set: template E1 is used for ordinary displays, with a first packet reading wait time of 1ms and a subsequent block interval of 1ms; template E2 is used for high-resolution professional displays, with a first packet reading wait time of 3ms and a subsequent block interval of 2ms, and requires that a hot-plug high level be maintained for no less than 5ms before reading. The system selects between E1 and E2 based on the reinforcement learning results and returns the extended display identifier data according to the corresponding rhythm during the virtual handshake.

[0106] Under these constraints, status acquisition and action execution are more clearly defined; cable impedance attenuation values ​​and bit error rate before adaptive equalizer compensation correspond to the physical layer status of the main link of the display port; historical timing reading time corresponds to the historical time taken for the auxiliary channel of the display port to read extended display identifier data; pre-emphasis and voltage swing in the link compensation parameters act on the main link transmitter of the display port, while pre-load timing parameters act on the auxiliary channel and hot-plug related control logic; the main link and auxiliary channel thus form a synergy: the former ensures signal quality, and the latter ensures the continuity of display capability negotiation;

[0107] The specific interaction process is illustrated below: If the EDID first packet read times for a target display in the past three switching operations were 2ms, 3ms, and 3ms respectively, averaging approximately 2.67ms, and the second read often timed out when using the E1 template, the decision engine will be more inclined to select the E2 template for the terminal; at the same time, if the bit error rate of the main link is greater than the preset level threshold, a higher level of pre-emphasis parameter will be superimposed in the action; ultimately, a joint strategy that integrates the main link compensation parameters and the EDID timing template is generated, rather than optimizing only a single dimension;

[0108] In one implementation as an exception handling method, if the target data link is not a display port link but another video interface link, the extended display identifier data timing template in this embodiment can be disabled, and the system can fall back to the general handshake template adapted to the interface. If the target display port device itself does not support the complete extended display identifier data reading, for example, only returning the basic display mode, the system can use simplified caching and reduce template fields, but still retain timing control logic. If the difference between the read EDID content and the historical cached data exceeds the preset tolerance threshold, for example, if the display is replaced or the firmware is upgraded, the system can mark the terminal as a new terminal, re-collect its timing characteristics, and avoid continuing to apply the old template.

[0109] During the upgrade of the air traffic control booth, a newly connected 4K professional monitor was connected to the KVM switch via a display port link. The system found that the monitor had a long first packet response interval when reading extended display identifier data. If the ordinary template was used, the target host would easily misjudge that the monitor was offline. Therefore, the system selected E2 as the preload timing parameter and returned the extended display identifier data according to the E2 delay and block interval during the virtual handshake. At the same time, the main link compensation parameter was used to complete the switching, which ultimately avoided the host output mode rollback.

[0110] The purpose of this step is to implement the general control method specifically in the display port link and extended display identification data processing, so as to achieve the effect of clear solution boundaries, clear engineering interfaces and direct configuration.

[0111] Example 2:

[0112] Please see Figure 2 A reinforcement learning-based KVM switcher seamless switching control system is applied to KVM switches that connect multiple data sources via an internal cross matrix and receive video streams from the current data source, including:

[0113] The multi-dimensional state awareness module is used to collect physical layer state data of the current data link corresponding to the current data source and the target data link corresponding to the target data source. The physical layer state data includes cable impedance attenuation value, bit error rate before adaptive equalizer compensation, historical time series reading time, environmental signal-to-noise ratio estimation value, and terminal configuration mode.

[0114] The reinforcement learning decision module is used to input physical layer state data into the deep Q-network-based reinforcement learning decision engine for state space mapping and output link compensation parameters and preload timing parameters; among which, the link compensation parameters include signal pre-emphasis parameters and voltage swing parameters.

[0115] The pre-handshake processing module is used to perform a virtual handshake operation with the target data source through the target data link based on pre-loaded timing parameters, and generate the source link hold-up state.

[0116] The dynamic reconstruction and execution module is used to receive the switching trigger command for the target data source. In response to the switching trigger command, it reconstructs the underlying physical layer parameters of the KVM switcher based on the link compensation parameters in the vertical blank area of ​​the video stream of the current data source and performs the switching operation of the cross matrix. It obtains the link retraining status after the switch by polling the link status register of the underlying physical interface of the KVM switcher.

[0117] The feedback adaptive module is used to calculate the model reward value based on the retraining state of the link after the switch, and to use the model reward value to update the reinforcement learning decision engine.

[0118] This embodiment provides a seamless switching control system for KVM switches based on reinforcement learning. Specifically, the system can be integrated inside the KVM switch in the air traffic control automation equipment cabinet and consists of a multi-dimensional state perception module, a reinforcement learning decision module, a pre-handshake processing module, a dynamic reconstruction and execution module, and a feedback adaptive module. Each module can be deployed on the same FPGA and its matching processor platform, or it can be allocated according to function among the FPGA logic area, embedded soft core, and peripheral microcontroller.

[0119] Specifically, the multi-dimensional state awareness module is located near the link input side and the underlying physical interface, and is responsible for collecting the physical layer state from the current link and the target link. This module may include a bit error counting unit, a signal-to-noise estimation unit, an auxiliary channel time recording unit, and a terminal pattern recognition unit. The reinforcement learning decision module receives these state data, completes normalization, state splicing and deep Q network inference, and outputs link compensation parameters and preloaded timing parameters.

[0120] After receiving the pre-loaded timing parameters, the pre-handshake processing module calls the target timing dataset in the hardware timing cache module to present the terminal's simulated online status to the target host and listens to whether the target host maintains video stream output; the dynamic reconstruction and execution module is responsible for parsing the switching instructions, tracking the vertical blank area, writing to the underlying physical interface control core, and driving the cross matrix update; the feedback adaptive module collects the switching results from the link state register, frame counter, and display mode detection unit, calculates the reward value, writes the samples into the experience playback pool, and then trains and updates the network weights in the decision module in batches;

[0121] Regarding the interaction between modules, the entire system operates in a closed-loop manner of perception-decision-preprocessing-execution-feedback. A complete control cycle is as follows: the perception module outputs state S5; the decision module outputs action A2 and timing template T2; the pre-handshake processing module establishes the source-end link hold state based on T2; the dynamic reconstruction and execution module completes parameter reconstruction and cross matrix switching in the vertical blank area; the feedback adaptive module reads the result after the switch as high reward and updates the network weights accordingly; the next time the system detects a state similar to S5, the decision module will tend to select the combination of A2 and T2 again based on the updated network weights.

[0122] In one hardware implementation, the multi-dimensional state perception module and the dynamic reconstruction and execution module are mainly implemented using FPGA logic resources to meet the requirements of high-speed sampling and precise timing; the reinforcement learning decision module and the feedback adaptive module can run on the soft-core processor in the FPGA or the peripheral MCU to perform state reasoning and weight updates; the pre-handshake processing module occupies both logic resources and cache resources to respond to the auxiliary channel access of the display port and output cached EDID data; the above division is not unique, as long as the information flow and control closed loop between the modules can be realized;

[0123] In one implementation for exception handling, if the decision-making module experiences an abnormal shutdown or the network weight file is corrupted, the system can switch to a static conservative parameter mode, where a pre-configured set of compatible parameters continues to maintain basic switching capabilities. If the perception module experiences a partial failure, such as an unavailable bit error counting unit, a degraded state set is allowed to participate in the decision-making process, while simultaneously increasing the conservative threshold for pre-handshake and switching execution. If the feedback adaptive module is unavailable during the training period, the system can still complete the online switching first and temporarily store the samples, updating them in batches after the training module recovers. If the pre-handshake processing module fails to establish a target source hold state, the dynamic reconstruction and execution module can refuse to execute the formal switching, avoiding the risk of a black screen when preconditions are not met.

[0124] In the master / standby switchover drill at the air traffic control operation center, the multi-dimensional state awareness module continuously monitors the attenuation, bit error rate, noise, and EDID read characteristics of the master link A and master link B; the reinforcement learning decision module selects a suitable parameter combination for the aging cable and high-resolution display for the action of switching to master B; the pre-handshake processing module allows master B to stabilize the flow in advance; the dynamic reconstruction and execution module completes the underlying reconstruction and port switching in the vertical blank area; the feedback adaptive module records high rewards after confirming no retraining and no frame loss; after multiple drills, the system forms a more stable switchover strategy for the specific cable and display combination of the position.

[0125] The purpose of this system is to support the aforementioned methods and processes in a modular manner, so that the four capabilities of perception, reasoning, execution and parameter update can be formed into a deployable and scalable engineering architecture, thereby achieving seamless KVM switching control for high-reliability business scenarios such as air traffic control.

[0126] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A seamless switching control method for KVM switchers based on reinforcement learning, wherein the KVM switcher connects multiple data sources through an internal cross matrix and receives video streams from the current data source, characterized in that... include: Collect physical layer status data of the current data link corresponding to the current data source and the target data link corresponding to the target data source; among which, the physical layer status data includes cable impedance attenuation value, bit error rate before adaptive equalizer compensation, historical time sequence reading time, environmental signal-to-noise ratio estimation value, and terminal configuration mode; The environmental signal-to-noise ratio (SNR) estimate is obtained by weighting the received level jitter amplitude and the number of bit error spikes per unit time. The physical layer state data is input into a reinforcement learning decision engine based on a deep Q-network for state space mapping, and the output is link compensation parameters and pre-loading timing parameters. The link compensation parameters include signal pre-emphasis parameters and voltage swing parameters. Based on preloaded timing parameters, a virtual handshake operation is performed between the target data link and the target data source to generate a source-end link hold-up state. Receives a switching trigger command for the target data source. In response to the switching trigger command, reconstructs the underlying physical layer parameters of the KVM switcher based on the link compensation parameters in the vertical blank area of ​​the video stream of the current data source and performs a switching operation of the cross matrix. Obtains the link retraining status after switching by polling the link status register of the underlying physical interface of the KVM switcher. The model reward value is calculated based on the retraining state of the link after the switch, and the model reward value is used to update the reinforcement learning decision engine.

2. The seamless switching control method for KVM switchers based on reinforcement learning according to claim 1, characterized in that, Based on preloaded timing parameters, a virtual handshake operation is performed between the target data link and the target data source to generate a source-end link hold-up state, including: Extract the target timing dataset corresponding to the preloaded timing parameters from the preset hardware timing cache module; Using the target time-series dataset, simulated online signals from the terminal are sent to the target data source via the target data link. Receive the video stream sustain status signal returned by the target data source in response to the terminal's simulated online signal; The video stream sustain status signal is confirmed as the source link sustain status.

3. The seamless switching control method for KVM switchers based on reinforcement learning according to claim 1, characterized in that, In response to a switching trigger command, the underlying physical layer parameters of the KVM switcher are reconstructed based on link compensation parameters in the vertical blank area of ​​the video stream from the current data source, and a cross-matrix switching operation is performed, including: Parse the switching trigger command to determine the target switching clock; Within the vertical blank area of ​​the video stream, the signal pre-emphasis parameters and voltage swing parameters are written into the underlying physical interface control core of the KVM switch. This underlying physical interface control core is a hardware logic control unit that includes a transmitter drive strength configuration register and a timing control register to complete the reconstruction of the underlying physical layer parameters. The port mapping update of the cross matrix is ​​triggered based on the target switching clock, thus completing the cross matrix switching operation.

4. The seamless switching control method for KVM switchers based on reinforcement learning according to claim 1, characterized in that, The model reward value is calculated based on the retraining state of the link after the switch, including: If the retraining status of the link after the switch is that retraining has not been triggered and no video frames have been lost, the model reward value will be set to the first preset reward value. If the retraining state of the link after switching is a resolution downgraded output state, set the model reward value to the second preset reward value; If the retraining status of the link after switching is a black screen reconnection interruption status or other status, the model reward value will be set to the third preset reward value. Among them, the first preset reward value is greater than the second preset reward value, and the second preset reward value is greater than the third preset reward value.

5. The seamless switching control method for KVM switchers based on reinforcement learning according to claim 1, characterized in that, The reinforcement learning decision engine is updated using the model reward value, including: Collect the physical layer status data of the target data link after the switch, and use it as the physical layer status data after the switch; construct the physical layer status data before the switch, the physical layer status data after the switch, the link compensation parameters, the preloaded timing parameters, and the model reward value into an experience replay tuple. Store the experience replay tuples into a preset experience replay pool; extract training batch data from the experience replay pool; calculate the temporal difference error using the training batch data; and update the network weight parameters of the reinforcement learning decision engine based on the temporal difference error.

6. The seamless switching control method for KVM switchers based on reinforcement learning according to claim 1, characterized in that, The current data link and the target data link are display port data links, and the preloaded timing parameters are extended display identifier data timing parameters.

7. A reinforcement learning-based seamless switching control system for KVM switches, applied to KVM switches that connect multiple data sources via an internal cross matrix and receive video streams from the current data source, characterized in that... include: The multi-dimensional state awareness module is used to collect physical layer state data of the current data link corresponding to the current data source and the target data link corresponding to the target data source. The physical layer state data includes cable impedance attenuation value, bit error rate before adaptive equalizer compensation, historical time series reading time, environmental signal-to-noise ratio estimation value, and terminal configuration mode. The reinforcement learning decision module is used to input physical layer state data into the deep Q-network-based reinforcement learning decision engine for state space mapping and output link compensation parameters and preload timing parameters; among which, the link compensation parameters include signal pre-emphasis parameters and voltage swing parameters. The pre-handshake processing module is used to perform a virtual handshake operation with the target data source through the target data link based on pre-loaded timing parameters, and generate the source link hold-up state. The dynamic reconstruction and execution module is used to receive the switching trigger command for the target data source. In response to the switching trigger command, it reconstructs the underlying physical layer parameters of the KVM switcher based on the link compensation parameters in the vertical blank area of ​​the video stream of the current data source and performs the switching operation of the cross matrix. It obtains the link retraining status after the switch by polling the link status register of the underlying physical interface of the KVM switcher. The feedback adaptive module is used to calculate the model reward value based on the retraining state of the link after the switch, and to use the model reward value to update the reinforcement learning decision engine.

Images