Method and system for restoring transmission session flow in IP network traffic, and intelligent terminal

Abstract

The present invention relates to the field of network technology, and relates to a method and system for restoring a transmission session flow in IP network traffic, and an intelligent terminal. The method comprises: acquiring data sending response information of data to be sent; determining whether the data sending response information is consistent with reference response information; on the basis of the data transmission response information being inconsistent with the reference response information, controlling a computer by means of a data resending method, so as to resend the data; on the basis of the data sending response information being consistent with the reference response information, acquiring device network state information; on the basis of the device network state information, performing analysis to determine a network impact type, wherein the network impact type comprises bandwidth occupation and network interruption; on the basis of the bandwidth occupation, processing a bandwidth by means of a bandwidth processing method, so as to send the data; and on the basis of the network interruption, reconnecting a network by means of a network reconnection method, so as to resume the data sending. The present application has an effect of enabling data to be sent to be normally sent to a receiving party even after the data is impacted by a local device.

Description

Methods, systems, and smart terminals for restoring transmission session streams in IP network traffic

[0001] This invention relates to the field of network technology, and in particular to a method, system, and smart terminal for restoring transmission session streams in IP network traffic. Background Technology

[0002] In IP networks, a transport session stream refers to a data transmission channel established between two or more communication endpoints to achieve reliable or unreliable data transmission. This channel can be established based on transport layer protocols such as TCP (Transmission Control Protocol) or UDP (User Datagram Protocol).

[0003] Packet loss is possible during transmission of session streams between communication endpoints. Session streams can also be lost due to local device interference during transmission. In cases of data loss due to local device interference, if the sender does not notice unsuccessful data transmission after sending, it may easily result in the loss or delay of data transmission, which requires further resolution. Summary of the Invention

[0004] In order to ensure that data to be sent can still be sent normally to the receiver even if it is affected by local devices, the present invention provides a method, system and smart terminal for restoring transmission session streams in IP network traffic.

[0005] In a first aspect, the present invention provides a method for restoring transmission session streams in IP network traffic, employing the following technical solution:

[0006] A method for restoring transport session streams in IP network traffic includes:

[0007] Retrieve the data transmission response information for the data to be sent;

[0008] Determine whether the data transmission response information is consistent with the preset baseline response information;

[0009] Based on the inconsistency between the data transmission response information and the baseline response information, the computer is controlled to retransmit the data using a preset data retransmission method.

[0010] Based on the consistency between the data transmission response information and the baseline response information, the device network status information is obtained;

[0011] The network impact type is determined by analyzing the device network status information. The network impact types include bandwidth usage and network outage.

[0012] Based on bandwidth usage, the bandwidth is processed using a preset bandwidth processing method to send the data;

[0013] Based on network interruption, the network is reconnected using a preset reconnection method to continue sending data.

[0014] By adopting the above technical solution, the system determines whether the computer's peripherals can be used normally to send data by determining the data transmission response information of the data to be sent. The system also obtains the computer's device network status information to determine whether the computer's network status will affect the normal transmission of data. By processing the above factors that affect data transmission, the data to be sent can be sent normally.

[0015] Optionally, data retransmission methods include:

[0016] Acquire device input image;

[0017] Determine whether the preset data transmission button has been pressed based on the input image from the device and the preset key button press actions;

[0018] Based on the fact that the data transmission button was not pressed, the characteristics of the pressed button were determined according to the device input image.

[0019] The deviation distance is determined based on the characteristics of the pressed button and the data transmission button.

[0020] Based on the deviation distance not exceeding the preset single-button distance, an incorrect press is defined as an error press. The preset pressing device is controlled to press the preset cancel button to cancel the error press process, and the data send button is pressed to resend the data.

[0021] Optional, also includes:

[0022] Based on the data transmission button being pressed, the button pressing depth and the center position of the button are determined according to the device input image and the data transmission button before and after the button is pressed.

[0023] Match the downward pressure intensity according to the depth of button press;

[0024] The corrective force is determined based on the downward force and the preset force increment;

[0025] The corrective force control pressing device presses the center position of the data transmission button again and obtains the data transmission response information of the data to be transmitted again.

[0026] If the data transmission response information is inconsistent with the baseline response information, it is defined as a button failure. A hotkey combination scheme is determined based on the data transmission button and the preset hotkey database.

[0027] The computer is set to generate combination hotkeys according to the hotkey combination scheme. The function of the combination hotkeys is the same as the function of the data sending key.

[0028] The pressure of the control device is adjusted to press the combination hotkey to resend the data.

[0029] Optionally, if there are foreign objects at the bottom of the data transmission button, it can also cause the button to malfunction. Methods for removing foreign objects include:

[0030] The control pressing device presses the data transmission button with a corrective force, and determines the actual pressing depth based on the device input image before and after the button is pressed.

[0031] Based on the discrepancy between the actual pressing depth and the preset benchmark pressing depth, it is defined as a foreign object obstruction, and image information of the button area is obtained;

[0032] Determine whether the foreign object feature is completely obscured by the data transmission button based on the image information of the button area and the preset reference area image information;

[0033] If the foreign object feature is completely obscured by the data transmission button, the control pressing device presses the center position of the data transmission button, and determines the tilt direction of the button based on the image information of the button area and the data transmission button.

[0034] Determine the blowing position based on the tilt direction of the button;

[0035] Control the preset air blowing device to blow air at the blowing position to blow out the foreign object feature from the bottom of the data transmission button;

[0036] If the foreign object feature is not completely obscured by the data transmission button, the location of the foreign object feature is determined based on the image information of the button area and the foreign object feature.

[0037] Determine the location of the maximum gap between adjacent keys based on the image information of the key area;

[0038] A movement path is formed based on the location of the foreign object and the location of the maximum gap.

[0039] The preset clamping device is controlled to clamp the foreign object at the location of the foreign object feature, and the foreign object feature is removed from the position of maximum gap by moving along the path.

[0040] Optional bandwidth processing methods include:

[0041] Determine the average network speed based on the device's network status information;

[0042] Based on the premise that the average network speed is no greater than the preset benchmark good network speed, obtain the data content of the data to be sent;

[0043] Determine the sending priority level based on the content of the data to be sent;

[0044] Based on a transmission priority level greater than a preset baseline priority, the average network speed of all computer devices on the same local area network is determined according to the device network status information.

[0045] The average network speed of all computer devices is screened to identify devices whose average network speed exceeds the preset abnormal network speed, and the screen image information of these devices is obtained.

[0046] The device user operation type is determined based on the image information and preset streaming video characteristics. The device user operation type includes watching streaming video and downloading data.

[0047] Based on data download, the device network status information of the device user is controlled with a preset network speed limit in order to free up network bandwidth;

[0048] Based on watching streaming video, the device network status information of the user is controlled at a preset critical network speed during the preset network outage period so that the data to be sent can be sent during the network outage period.

[0049] Optional methods for reconnecting to the network include:

[0050] Obtain real-time network latency information;

[0051] Analyze real-time network latency information to determine the types of network changes, including latency increases and latency fluctuations.

[0052] Based on latency fluctuations, the maximum fluctuation is determined according to real-time network latency information;

[0053] If the maximum fluctuation exceeds the preset applicable fluctuation range, the data to be sent is stored in the preset data buffer. When the maximum fluctuation falls into the applicable fluctuation range, the data to be sent in the data buffer is controlled to be sent.

[0054] The operating status of the backup network is obtained based on the increasing delay.

[0055] Based on the fact that the operating status of the backup network is consistent with the preset stable operating status, the local network of the computer equipment is switched to the preset backup network.

[0056] Optional, also includes:

[0057] Based on the inconsistency between the operating status of the backup network and the stable operating status, the transmission ratio of the data to be sent is obtained;

[0058] The proportion of items not sent is determined based on the proportion of items sent.

[0059] The local network of the control computer equipment is reconnected, and the unsent proportion of data to be sent is sent.

[0060] The data to be sent according to the sending ratio and the data to be sent according to the non-sending ratio are merged to obtain merged data, and it is determined whether the merged data is consistent with the data to be sent.

[0061] Based on the inconsistency between the merged data and the data to be sent, the control computer equipment retransmits the complete data to be sent.

[0062] Optionally, if the device loses power during the transmission of data, it will affect data transmission. Methods for handling device power outages include:

[0063] When a data transmission response is triggered, the computer backs up the data to be transmitted to form backup data and stores it in a preset backup cache.

[0064] Control the computer equipment to switch to a preset backup power supply and obtain the sent content information sent to the preset transmission buffer;

[0065] Retrieve the backup data from the backup cache and match and analyze the backup data with the sent content information to obtain the unsent content information;

[0066] Unsent content information is sent to the data buffer, merged with sent content information to form complete data to be sent, and then resent the complete data to be sent.

[0067] Secondly, this application provides a system for restoring transmission session streams in IP network traffic, employing the following technical solution:

[0068] A system for restoring transport session streams in IP network traffic, comprising:

[0069] The acquisition module is used to acquire data transmission response information of the data to be sent, device network status information, device input image, key area image information, data content of the data to be sent, screen image information of the device user, real-time network latency information, operating status of the backup network, and transmission ratio of the data to be sent.

[0070] Memory, used to store programs for reconstructing transmission session streams from IP network traffic;

[0071] The processor and memory programs can be loaded and executed by the processor to implement methods for restoring transmission session streams in IP network traffic.

[0072] Thirdly, this application provides a smart terminal, which adopts the following technical solution:

[0073] A smart terminal includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed to restore a transmission session stream in IP network traffic.

[0074] In summary, this application includes at least one of the following beneficial technical effects:

[0075] The system determines whether the computer's peripherals can be used normally to send data by checking the data transmission response information of the data to be sent. The system also obtains the computer's device network status information to determine whether the computer's network status will affect the normal transmission of data. By processing the above factors that affect data transmission, the data to be sent can be sent normally.

[0076] The system can detect when a user presses the data send button to determine whether the button is being pressed correctly and whether it is functioning properly. If the button is not pressed correctly, the system can undo the incorrect press and automatically control the pressing device to press the button. If the button malfunctions, the system can reset the hotkey combination to mimic its function. Furthermore, if the system detects that the bottom of the button is affected by foreign objects preventing it from being pressed down properly, the system can target and remove the foreign object based on its location to ensure the button can be pressed correctly.

[0077] When the data to be sent is affected by the computer network, the system can monitor the usage status of other device users within the local area network and disconnect the network for device users who are watching videos so that the data to be sent can be sent normally during the downtime. Attached Figure Description

[0078] Figure 1 is a flowchart of the method for restoring transmission session flow in IP network traffic according to an embodiment of the present invention;

[0079] Figure 2 is a flowchart of the data retransmission method according to an embodiment of the present invention.

[0080] Figure 3 is a second flowchart of the data retransmission method according to an embodiment of the present invention;

[0081] Figure 4 is a flowchart of the foreign matter removal method according to an embodiment of the present invention. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0083] This application discloses a method for restoring transmission session streams in IP network traffic.

[0084] Referring to Figure 1, the method for restoring transmission session flows in IP network traffic includes the following steps:

[0085] Step S100: Obtain the data transmission response information of the data to be sent.

[0086] Data transmission response information refers to the computer's acquisition of keyboard or mouse press trigger information when data is sent via keyboard or mouse. The computer detects keyboard or mouse presses in real time, and the system acquires data transmission response information for the data to be sent whenever the keyboard or mouse is pressed, regardless of whether it is a normal press. When the keyboard or mouse is pressed normally, the computer can directly obtain the data transmission response information for the data to be sent from the system.

[0087] Step S101: Determine whether the data transmission response information is consistent with the preset baseline response information.

[0088] The baseline response information refers to the response information triggered when the keyboard or mouse is pressed normally, which will not be elaborated here.

[0089] By comparing the data transmission response information with the baseline response information, it can be determined whether the keyboard or mouse was pressed normally when preparing to send data.

[0090] Step S1011: Based on the inconsistency between the data transmission response information and the baseline response information, control the computer to retransmit the data using a preset data retransmission method.

[0091] If the data transmission response information is inconsistent with the baseline response information, it indicates that the keyboard or mouse was not pressed normally. This could be due to incorrect key presses, key malfunctions, or obstructed key presses, resulting in incomplete data reception. In such cases, the system retransmits the data using a data retransmission method to ensure complete delivery to the receiver. The data retransmission method will not be detailed here but will be described in detail in subsequent embodiments.

[0092] Step S1012: Based on the consistency between the data transmission response information and the baseline response information, obtain the device network status information.

[0093] If the data transmission response information is consistent with the baseline response information, it means that the keyboard or mouse has been pressed normally. In this case, the reason for incomplete data reception may be that the data transmission process is affected by the network.

[0094] Device network status information refers to the network status of a device at the moment data transmission response is triggered. This information can be obtained directly by monitoring the system's network conditions.

[0095] Step S10121: Analyze and determine the network impact type based on the device network status information. The network impact types include bandwidth usage and network interruption.

[0096] Network impact type refers to the type of impact the network has on data transmission, including bandwidth usage and network outages. Network impact type can be directly obtained by analyzing the device's network status information. Bandwidth usage may lead to high network latency, resulting in packet loss and incomplete data. Network outages will directly cause data transmission failure.

[0097] Step S10122: Based on bandwidth occupancy, process the bandwidth using a preset bandwidth processing method to send the data.

[0098] In cases of bandwidth occupancy, the system will manage network bandwidth to ensure that data can be sent normally without being affected by network bandwidth limitations. The bandwidth management method will not be elaborated here, but will be described in detail in subsequent embodiments.

[0099] Step S10123: Based on the network interruption, reconnect the network using a preset network reconnection method to continue sending data.

[0100] In the event of a network interruption, the system will process the network to allow data to continue being sent. The method for reconnecting after a network interruption will not be described in detail here, but will be explained in detail in subsequent embodiments.

[0101] Referring to Figure 2, the data retransmission method includes the following steps:

[0102] Step S200: Acquire the device input image.

[0103] Device input images refer to images taken by operators while using a computer, capturing photos of the keyboard and mouse. Computers typically have built-in cameras that capture these device input images.

[0104] Step S201: Determine whether the preset data transmission button has been pressed based on the device input image and the preset key button pressing action.

[0105] The key button pressing action is the hand movement of the operator pressing the data sending button, which is pre-saved in the database by the technicians. It will not be described in detail here.

[0106] By comparing the device's input image with key button presses, if a key button press is observed in the input image, it indicates that the data send button has been pressed. The data send button refers to the key that needs to be pressed to send data.

[0107] Step S2011: Based on the fact that the data transmission button was not pressed, determine the characteristics of the pressed button according to the device input image.

[0108] When an operator presses the wrong key, the image recognition may show that the data transmission button is not pressed. In this case, by determining the positional relationship between the pressed key and the data transmission button, it can be determined whether the mistake was unintentional. The pressed key characteristic refers to the actual position of the button pressed by the operator.

[0109] Step S20111: Determine the deviation distance based on the characteristics of the pressed button and the data sending button.

[0110] Deviation distance refers to the distance between the pressed button feature and the data transmission button.

[0111] Step S20112: Based on the deviation distance not being greater than the preset single button distance, it is defined as an incorrect press. The preset pressing device is controlled to press the preset cancel button to cancel the incorrect press process, and the data send button is pressed to resend the data.

[0112] The single-key distance refers to the spacing between two adjacent keys set by technicians, which will not be elaborated here.

[0113] If the deviation distance is greater than the distance of a single key press, it means that the pressed key feature and the data sending key are not adjacent keys. In this case, if the pressed key feature is pressed, it may be performing other data operations, which is normal operation.

[0114] If the deviation distance is not greater than the distance of a single key press, it means that the pressed key feature and the data send key are adjacent keys. In this case, it may be an incorrect press. The user intended to press the data send key but instead pressed the pressed key feature, resulting in the data not being sent normally.

[0115] The pressing device is located on the side of the computer and can recognize and assist pressing the keyboard and mouse.

[0116] The undo button is a computer-preset button used to undo the function of a button that has already been pressed.

[0117] When the operator's pressing action is determined to be an incorrect press, the system will control the pressing device to assist in pressing the cancel button. After the press is completed, the system will return to the previous state. At this time, the pressing device will be used to press the data send button again to resend the data.

[0118] Referring to Figure 3, the data retransmission method further includes the following steps:

[0119] Step S2012: Based on the data transmission button being pressed, determine the button pressing depth and the center position of the button according to the device input image before and after the button is pressed and the data transmission button.

[0120] If the data send button is pressed but the data is not sent to the receiver, the data send button may be faulty.

[0121] The button press depth refers to the distance the data send button moves downwards when an operator presses it. The button press depth can be determined by acquiring the device input images before and after the operator presses the button, and by performing image recognition analysis on both images.

[0122] The center position of the button is the top center of the data transmission button. The center position of the button can be determined by image recognition analysis of the input image from the device.

[0123] Step S20121: Match the pressing force according to the button pressing depth.

[0124] Downward pressure refers to the force required to press the data send button downwards. The button's downward depth corresponds to the downward pressure; the greater the downward depth, the greater the downward pressure. Here, downward pressure refers to the force required to press the data send button downwards to a specific depth, representing the pressure applied by the operator to the key.

[0125] Step S20122: Determine the correction force based on the downward force and the preset force increment.

[0126] The force increment is the correction amount set by the technician to correct the downward force, which will not be elaborated here.

[0127] The correction force is the parameter value after downward pressure correction. The correction force here is the pressure applied by the pressing device to press the data transmission button. When the correction force is applied, it is assumed that the button can be fully pressed normally.

[0128] Step S20123: Press the center position of the data transmission button again according to the correction force control pressing device, and obtain the data transmission response information of the data to be transmitted again.

[0129] Once the correction force and button center position are determined, the system uses a pressing device to press the data transmission button again and obtains the data transmission response information of the data to be transmitted, in order to determine whether the data was not successfully transmitted because the operator pressed too lightly.

[0130] Step S20124: If the data transmission response information is inconsistent with the baseline response information, it is defined as a button malfunction. A hotkey combination scheme is determined based on the data transmission button and the preset hotkey database.

[0131] If the data transmission response information is consistent with the baseline response information at this time, it indicates that the key pressing pressure is indeed too light, and the data can be resent during the process of verifying the cause.

[0132] If the data transmission response information is inconsistent with the baseline response information, it means that even if the key is fully pressed, no effect will be produced, that is, the key is malfunctioning.

[0133] The hotkey database is a database set up by technical personnel, which contains data on various existing hotkeys for computers, and will not be elaborated on here.

[0134] Hotkey combination schemes refer to methods that combine keys to achieve the same function as the data transmission key. The system identifies the function of the data transmission key and compares it with a hotkey database. By avoiding hotkeys already existing in the database, the system can determine the hotkey combination scheme.

[0135] Step S20125: Control the computer to set up combination hotkeys according to the hotkey combination scheme. The function of the combination hotkeys is the same as the function of the data sending button.

[0136] Hotkey combinations refer to multiple keys on a keyboard that can be used in combination to achieve the same function as data transmission keys. Once a hotkey combination scheme is determined, the system configures the keyboard's key functions according to the scheme, thus creating the hotkey combination.

[0137] Step S20126: Press the combination hotkey according to the correction force control device to resend the data.

[0138] When the data transmission button malfunctions, the system uses a pressing device to assist in pressing the combination hotkey, thereby enabling the data to be retransmitted.

[0139] Referring to Figure 4, if there are foreign objects at the bottom of the data transmission button, it can also cause the button to malfunction. The method for removing foreign objects includes the following steps:

[0140] Step S300: Control the pressing device to press the data transmission button with a corrective force, and determine the actual pressing depth based on the device input image before and after the button is pressed;

[0141] The system controls the pressing device to press the data transmission button with corrective force, and under normal circumstances the button can be fully pressed down.

[0142] The actual pressure depth is the distance the data transmission button is actually pressed when pressed by the pressing device. Similar to step S2012, the actual pressure depth can be determined by image recognition from two device input images taken before and after the button is pressed.

[0143] Step S301: Based on the inconsistency between the actual pressing depth and the preset reference pressing depth, it is defined as a foreign object obstruction, and the image information of the button area is obtained;

[0144] The reference pressing depth is the distance the key moves downwards when it is fully pressed, as set by the technician; this will not be elaborated upon here.

[0145] When the actual pressing depth is inconsistent with the preset reference pressing depth, it indicates that the key is obstructed when pressed, and there is a foreign object at the bottom of the data transmission key that is blocking its downward movement.

[0146] The image information of the button area refers to the image obtained by taking pictures of the data transmission button and its surrounding area through a camera. By taking pictures of the data transmission button and its surrounding area, the specific location of foreign objects at the data transmission button can be determined, so as to facilitate subsequent removal.

[0147] Step S302: Determine whether the foreign object feature is completely obscured by the data transmission button based on the button area image information and the preset reference area image information;

[0148] The reference area image information refers to the image when there are no foreign objects at the key position, which is an image taken in advance by technicians.

[0149] By comparing and analyzing the image information of the key area with that of the reference area, it can be determined whether the foreign object feature is completely obscured by the data transmission key. If the image information of the key area is the same as that of the reference area, it means that the foreign object feature cannot be identified from the key area image information. Conversely, if the image information of the key area is different from that of the reference area, it means that the foreign object feature can be identified.

[0150] The system employs different processing methods to determine whether the location of foreign objects can be directly identified based on their characteristics.

[0151] Step S3021: If the foreign object feature is completely blocked by the data transmission button, control the pressing device to press the center position of the data transmission button, and determine the tilt direction of the button according to the image information of the button area and the data transmission button.

[0152] If the foreign object is completely obscured by the data transmission button, the system cannot determine the exact location of the foreign object at the bottom of the button. In this case, pressing the center of the data transmission button with a pressing device will cause the button to tilt towards the side without the foreign object when pressed, due to the presence of the foreign object. The direction of this tilt is the button tilt direction. This tilt direction can be determined through image recognition analysis of the data transmission button within the button area image.

[0153] Step S30211: Determine the blowing position according to the tilt direction of the button;

[0154] The blowing position refers to the location where air is blown from the bottom of the data transmission button using a blowing device, causing foreign objects to be blown out. The blowing position is located on the downward-sloping side of the data transmission button, so the blowing position can be determined by the direction of the button's tilt.

[0155] Step S30212: Control the preset air blowing device to blow air at the air blowing position to blow the foreign object feature out from the bottom of the data transmission button;

[0156] The system controls the blowing device to blow air at the blowing position, which can blow foreign objects out from the bottom of the data transmission button, so that the data transmission button cannot completely cover the characteristics of foreign objects.

[0157] Once the foreign object feature emerges from the bottom of the data transmission button, the subsequent handling method for the foreign object feature is the same as that for foreign object features that are not completely obscured by the data transmission button, and will not be elaborated here, but will be described in detail in subsequent embodiments.

[0158] Step S3022: If the foreign object feature is not completely obscured by the data transmission button, determine the location of the foreign object feature based on the button area image information and the foreign object feature;

[0159] When the foreign object feature is not completely obscured by the data transmission button, it can be identified from the image information of the button area. Therefore, the system can determine the location of the foreign object feature by performing image recognition analysis on the button area image information. The location of the foreign object feature refers to its actual position at the data transmission button.

[0160] Step S30221: Determine the position of the maximum gap between adjacent keys based on the key area image information;

[0161] The maximum gap position refers to the location of the largest gap between adjacent keys around the data transmission key. This maximum gap position is determined by analyzing the distance between each adjacent key using image recognition analysis of the key area image information.

[0162] Step S30222: Form a movement path based on the location of the foreign object and the location of the maximum gap;

[0163] The movement path refers to the path taken when the foreign object feature is removed from the bottom of the data transmission button. The foreign object feature needs to be moved from its original location to the maximum gap position, so the system can generate a movement path based on the determined foreign object feature location and the maximum gap position.

[0164] Step S30223: Control the preset clamping device to clamp the foreign object feature at the foreign object feature position, and remove the foreign object feature from the maximum gap position by moving along the path.

[0165] In this embodiment, once the foreign object feature can be identified and its location has been determined, the system can control the clamping device to clamp the foreign object feature at its location and move it along the moving path, thereby removing the foreign object feature from the position of maximum gap.

[0166] The bandwidth processing method includes the following steps:

[0167] Step S400: Determine the average network speed based on the device's network status information.

[0168] Average network speed refers to the current network speed of a computer device when data is to be sent. The average network speed can be directly obtained from the device's network status information. Analyzing the average network speed can determine whether the current network is good or not.

[0169] Step S401: Based on the premise that the average network speed is not greater than the preset benchmark good network speed, obtain the data content of the data to be sent.

[0170] A good benchmark network refers to a network that is in good operating condition and has low latency.

[0171] If the average network speed is greater than the baseline good network speed, it means that the current network status of the computer device is good, and sending data will not affect the data.

[0172] If the average network speed is not greater than the baseline good network speed, it indicates that the current network condition is poor and needs to be addressed so that data can be sent normally.

[0173] The data content to be sent refers to the text content of the data that the current computer device needs to send. The system can read and recognize the content of the data to be sent to obtain the data content.

[0174] Step S402: Determine the sending priority level based on the data content of the data to be sent.

[0175] In this embodiment, the system reads the content of the data to be sent and classifies the importance of the content to be sent, thereby determining the sending priority level. The higher the sending priority level, the more important the data to be sent, and the more important it is to be sent.

[0176] Step S403: Based on the fact that the sending priority level is greater than the preset baseline priority, determine the average network speed of all computer devices on the same local area network according to the device network status information.

[0177] The baseline priority is the level of priority set by technical personnel for data that needs to be prioritized, and will not be elaborated upon here. When the sending priority level is no higher than the baseline priority, it indicates that the data to be sent is of lower importance. In this case, even if the network conditions are poor, the data will be sent only after the network conditions improve. When the sending priority level is higher than the baseline priority, it indicates that the data to be sent is of higher importance and needs to be prioritized for transmission.

[0178] In this embodiment, factors affecting network conditions may include other computer devices on the same local area network (LAN), so it is necessary to determine the average network speed of all computer devices on the same LAN.

[0179] Step S404: Filter the average network speed of all computer devices to identify device users whose average network speed exceeds the preset abnormal network speed, and obtain the screen image information of the device users.

[0180] The term "extraordinary network speed" is a standard set by technicians to measure whether the average network speed of computer equipment is consuming a large amount of bandwidth, and will not be elaborated on here.

[0181] The system monitors devices and users whose average network speed exceeds the normal range and acquires their screen images. These screen images refer to the images displayed on the computer screen, which are obtained through a webcam.

[0182] When the average network speed exceeds the normal network speed, it indicates that the user of the device may be using an application that consumes a lot of bandwidth. At this time, the system will identify the problem and deal with it at the source.

[0183] Step S405: Determine the device user operation type based on the image information and preset streaming video characteristics. The device user operation type includes watching streaming video and downloading data.

[0184] Device user operation type refers to the type of application the device user is currently using. By analyzing the screen image information, it can be determined whether the device user is currently watching a video or downloading data resources. Different processing methods are adopted for different situations.

[0185] Step S4061: Based on data download, control the device network status information of the device user with a preset network speed limit to free up network bandwidth.

[0186] If a device user is downloading data, disconnecting their network might cause the download to fail, and if the data is important, it could have serious consequences. Therefore, the network speed for that specific device user is limited, thus reducing the bandwidth they consume. This speed limit is set by technicians to maintain a speed sufficient for normal data downloads, and will not be elaborated upon further.

[0187] Step S4062: Based on watching streaming video, control the device network status information of the device user within a preset downtime and at a preset downtime threshold network speed so that the data to be sent can be sent within the downtime.

[0188] If a device user is watching streaming video, the system will disconnect the network for a period of time. The network speed increase before the disconnection period is the critical speed for network interruption. During the disconnection period, since the device user is not watching video, pending data can be sent normally. The disconnection period is the length of time the network is disconnected for a device user watching video, as set by technicians, and will not be elaborated here. The critical speed for network interruption is the maximum network speed increase set by technicians for a device user watching video when the network is disconnected, and will not be elaborated here.

[0189] The method for reconnecting to the network after a disconnection includes the following steps:

[0190] Step S500: Obtain real-time network latency information.

[0191] Real-time network latency information refers to the network latency during the transmission of data to be sent. This information can be obtained directly from the computer system.

[0192] Step S501: Analyze the real-time network latency information to determine the network change type, which includes latency increase and latency fluctuation.

[0193] There are two types of network latency information. One is that the network latency is continuously increasing. When the network latency reaches its maximum, it indicates that the network has been disconnected. The second is that the network latency fluctuates, which only indicates that the network status is poor and there is no network disconnection.

[0194] Network change types can be analyzed and determined based on real-time network latency information over a period of time.

[0195] Step S5021: Based on latency fluctuations, determine the maximum fluctuation according to real-time network latency information.

[0196] Maximum fluctuation refers to the maximum amount of latency fluctuation that occurs in a computer network. When the maximum fluctuation exceeds a certain amount, the network condition is poor and it is not suitable for sending data. Maximum fluctuation can help identify and determine real-time network latency information.

[0197] Step S50211: Based on the maximum fluctuation exceeding the preset applicable fluctuation range, the data to be sent is stored in the preset data buffer. When the maximum fluctuation falls into the applicable fluctuation range, the data to be sent in the data buffer is controlled to be sent.

[0198] The applicable fluctuation range refers to the delay fluctuation range set by technicians for the appropriate data transmission, which will not be elaborated here.

[0199] When the maximum fluctuation falls within the applicable fluctuation range, data transmission will not be affected.

[0200] When the maximum fluctuation exceeds the applicable fluctuation range, it will affect the normal transmission of data. In this embodiment, for the above situation, the system first stores the data to be sent in the data buffer area, and then sends the data to be sent in the data buffer area after the maximum fluctuation of the delay fluctuation falls into the applicable fluctuation range.

[0201] Step S5022: Obtain the operating status of the backup network based on the incremental delay.

[0202] In this embodiment, if a network outage occurs due to increasing latency, the system will switch the local network to the backup network. Before doing so, the system needs to determine the operating status of the backup network, which can be obtained directly from the system.

[0203] Step S50221: Based on the fact that the operating status of the backup network is consistent with the preset stable operating status, switch the local network of the computer device to the preset backup network.

[0204] The stable operating status refers to the state in which the backup network is operating normally as set by the technicians, and will not be elaborated here.

[0205] If the operating status of the backup network is inconsistent with the stable operating status, it indicates that there is an anomaly in the backup network and the local network cannot be switched to the backup network. In this case, the system will take other measures, which will not be elaborated here, but will be described in detail in subsequent embodiments.

[0206] If the backup network operates in the same way as the stable network, it means that the backup network is operating well and the local network can switch to the backup network at any time. When switching networks, data transmission will not be affected and network latency will be reduced.

[0207] The method for reconnecting to the network after a disconnection also includes the following steps:

[0208] Step S600: Based on the inconsistency between the operating status of the backup network and the stable operating status, obtain the transmission ratio of the data to be sent.

[0209] If the backup network's operating status differs from the stable operating status, and switching to the backup network is not possible, the computer network will experience a downtime. When a downtime occurs, some data being transmitted may have been successfully sent, while some remains untransmitted. The percentage of data to be transmitted refers to the percentage of successfully transmitted data out of the total data; this percentage can be analyzed and obtained from the data to be transmitted.

[0210] Step S601: Determine the unsent ratio based on the sent ratio.

[0211] The unsent percentage refers to the proportion of data that has not yet been sent. The sent percentage and the unsent percentage are added together to equal 100%, therefore the unsent percentage can be determined based on the sent percentage.

[0212] Step S602: Control the computer device to reconnect to the local network and send the unsent proportion of data to be sent.

[0213] After the local network is restored, the system sends the data to be sent according to the proportion of data that was not sent. At this time, the proportion of data that was not sent may overlap with the proportion of data that was already sent, and the two cannot be combined to form complete data.

[0214] Step S603: Merge the data to be sent (the sending ratio) and the data to be sent (the non-sending ratio) to obtain merged data, and determine whether the merged data is consistent with the data to be sent.

[0215] The merged data is obtained by combining the data sent before the network outage with the proportion of unsent data to be sent after the network is restored. By comparing the merged data with the data to be sent, it can be determined whether the two are consistent, thus determining whether the data was sent completely.

[0216] Step S604: Based on the inconsistency between the merged data and the data to be sent, the control computer device retransmits the complete data to be sent.

[0217] If the merged data matches the data to be sent, it means the data has been sent completely. If the merged data does not match the data to be sent, it means the merged data is not complete data to be sent, and the system needs to resend the data to be sent.

[0218] Sending unsent data proportionally can reduce the amount of data sent, improve sending efficiency, and prevent data loss when sending complete data again.

[0219] Data transmission will be affected if the device loses power during the transmission process. The following steps are included in the method for handling device power failure:

[0220] Step S700: When the data transmission response information is triggered, the computer is controlled to back up the data to be transmitted to form backup data and store it in a preset backup cache area.

[0221] In this embodiment, when the mouse or keyboard presses the data send key, the computer simultaneously backs up the data to be sent and stores it in a backup cache to prevent data loss due to power failure. The backup cache is an area on the computer designated by technicians for storing backup data.

[0222] Step S701: Control the computer device to switch to the preset backup power supply for power supply, and obtain the sent content information sent to the preset transmission buffer.

[0223] In this embodiment, when the computer sends data, it first sends the data to the sending buffer, and then sends it from the sending buffer to the receiver.

[0224] When the computer loses power, some of the data to be sent will be sent to the sending buffer in advance, while the remaining data will be lost due to the power outage.

[0225] When the computer loses power, the system will switch the current power supply to the backup power supply, at which point the computer can be used normally again. Once the computer is back in use, the system will first retrieve the sent content information from the send buffer.

[0226] Step S702: Retrieve the backup data from the backup cache and match and analyze the backup data with the sent content information to obtain the unsent content information.

[0227] Since the unsent content information is lost during the power outage, it is necessary to retrieve the backup data from the backup cache, analyze the backup data and the sent content information, and then recover the unsent content information.

[0228] Step S703: Send the unsent content information to the data buffer, merge the unsent content information with the sent content information to form complete data to be sent, and resend the complete data to be sent.

[0229] Once the unsent content information is retrieved, it will be sent back to the sending buffer. The unsent content information and the sent content information will be merged in the sending buffer to obtain the complete data to be sent. At this time, the complete data to be sent is located in the sending buffer and will not be affected by the network outage, so it can be sent normally.

[0230] The method for determining hotkey combination schemes includes the following steps:

[0231] Step S800: Obtain keyboard image information.

[0232] Keyboard image information refers to images of the keyboard obtained by taking pictures of the keyboard with a camera.

[0233] Step S801: Determine whether the keyboard has a numeric keypad area based on the keyboard image information.

[0234] By analyzing keyboard image information, the keyboard type can be determined. Some keyboards have 108 keys and a numeric keypad area containing keys that function as data transfer keys. Other keyboards have 68 keys and lack a numeric keypad area. Therefore, different hotkeys can be used to replace keys for different keyboard types.

[0235] Step S8021: If the keyboard has a numeric keypad area, output the preset replacement key positions as hotkey combination schemes.

[0236] For keyboards with a numeric keypad, when the data send key malfunctions, the numeric keypad's replacement key can be used to replace the data send key. By pressing the replacement key via the UI, it can achieve the same function as the data send key.

[0237] In this embodiment, if the replacement key also malfunctions, it is necessary to redetermine the hotkey combination scheme from the main keyboard area. This will not be elaborated here, but will be described in detail in subsequent embodiments.

[0238] Step S8022: If there is no numeric keypad area, determine the general function keys and combination keys according to the keyboard image information, control the computer device to set the combination keys and combination keys as combination keys with the same function as data transmission presses, control the pressing device to press the combination keys, and obtain data transmission response information.

[0239] By analyzing keyboard image information, the general function keys are first identified within the function area, including keys such as Ctrl, Alt, Win, and Fn. Then, the corresponding key combinations are determined within the letter and number keys. These key combinations are those used in conjunction with the general function keys and pressed simultaneously. When selecting the above scheme, it needs to be matched against used hotkey schemes in the hotkey database to avoid duplication.

[0240] When selecting general function keys and combination keys, it is necessary to determine whether there are any malfunctioning keys among the currently selected hotkey combinations. Therefore, a check is required. The system first sets the function of the combination key to the same function as the data transmission press, then presses it through the pressing device, and determines whether there is a malfunction by transmitting data response information.

[0241] Step S80221: When the data transmission response information is inconsistent with the preset baseline response information, the matching key or general function key is changed in sequence until the data transmission response information is consistent with the baseline response information, and the hotkey combination scheme is output.

[0242] If the data transmission response information is consistent with the baseline response information, it indicates that the currently selected key combination function is intact and can be used as a key combination scheme.

[0243] If the data transmission response information is inconsistent with the baseline response information, it indicates that at least one of the general function keys and the paired keys is malfunctioning. In this case, the system will reselect the general function key from the alternative keys in the function area or reselect the paired key from the number keys and letter keys, and then check again until the data transmission response information is consistent with the baseline response information.

[0244] Once the data transmission response information matches the baseline response information, the final selected general function keys and combination keys will be used as the final hotkey combination scheme.

[0245] In this embodiment, the pressing device is in the form of a gantry frame, on which multiple selectable pressing heads capable of pressing keyboard keys slide. The pressing device can move horizontally, and after recognizing the keyboard position, it moves from one side of the keyboard to above the keyboard and along the length of the keyboard. During the movement, the pressing heads press the keyboard keys. The pressing method for pressing combination hotkeys includes the following steps:

[0246] Step S900: Determine the short side position and keyboard model of the keyboard based on the input image of the device and the preset keyboard features.

[0247] The short side position of the keyboard refers to the position of the end face of the keyboard along its width. The short side position can be obtained through image recognition analysis of keyboard features from the input image of the device.

[0248] Different keyboard models have different dimensions and specifications. If you need to press the keyboard using a pressing device, you must first determine the keyboard model to ensure compatibility with the pressing device. The keyboard model can be identified from the keyboard features in the input image of the device.

[0249] Step S901: Determine the keyboard's length, width, height, and key press information based on the keyboard model and the preset keyboard database.

[0250] The keyboard database is a database set up by technicians, which includes parameter information for all keyboard models, and will not be elaborated here.

[0251] Keyboard length, width, and height information refers to the length, width, and height values ​​of the keyboard. Key press information refers to the force required to press a key; different keyboard models have different key press information.

[0252] Entering the keyboard model into the keyboard database will retrieve the keyboard's length, width, height, and key press information.

[0253] Step S902: Match the door opening size information of the pressing device according to the length, width and height information of the keyboard.

[0254] The doorway size information refers to the existence of the doorway on the pressing device through which the keyboard passes. It mainly consists of the doorway length and height values. The doorway length needs to be compatible with the keyboard width, and the doorway height needs to be compatible with the keyboard height. Therefore, it can be obtained by matching the keyboard length, width, and height information with the doorway size information.

[0255] Step S903: Match the auxiliary pressing force of the pressing head according to the key pressing information.

[0256] The auxiliary pressing force is the pressure applied by the pressing head of the pressing device to the keyboard key. The auxiliary pressing force is consistent with the key pressing information.

[0257] Step S904: Determine the location of the hotkey combination based on the keyboard model and the hotkey combination.

[0258] Hotkey combination position refers to the actual location of the hotkey combination on the keyboard. The same hotkey combination may have different positions on different keyboard models. Therefore, once the keyboard model is determined, the hotkey combination position can be determined based on the keyboard model.

[0259] Step S905: Adjust the size of the pressing device according to the door opening size information so that the keyboard can pass through the pressing device. Control the pressing device to move to the short side of the keyboard and move along the length of the keyboard. During the movement, control the pressing head of the pressing device to press the combination hotkey according to the position of the combination hotkey and the auxiliary pressing force.

[0260] When the keyboard needs to be pressed using a pressing device, the pressing device can automatically identify the keyboard's position and adjust its size according to the keyboard model. Then, it moves to the top of the keyboard and presses the combination hotkeys during the movement. This allows the pressing device to assist in sending data even when the user is not sending data normally or is not on-site.

[0261] Step S906: After pressing, determine the pre-set opening distance of the accordion cover on the pressing device according to the length, width and height information of the keyboard, and control the accordion cover to open and close the keyboard according to the opening distance.

[0262] In this embodiment, the pressing device is integrated with an accordion cover that can be unfolded. The unfolding distance of the accordion cover can be automatically matched and set according to the length of the keyboard. After the pressing device completes the pressing of the combination hotkey, the pressing device can control the accordion cover to close the keyboard, so that other people are less likely to accidentally touch the keyboard keys.

[0263] Based on the same inventive concept, embodiments of the present invention provide a system for restoring transmission session streams in IP network traffic, comprising:

[0264] The acquisition module is used to acquire data transmission response information of the data to be sent, device network status information, device input image, key area image information, data content of the data to be sent, screen image information of the device user, real-time network latency information, operating status of the backup network, and transmission ratio of the data to be sent.

[0265] Memory is used to store programs for reconstructing transmission session streams from IP network traffic.

[0266] The processor and memory programs can be loaded and executed by the processor to implement methods for restoring transmission session streams in IP network traffic.

[0267] Based on the same inventive concept, embodiments of the present invention provide a smart terminal, including a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed to restore a transmission session stream in IP network traffic.

[0268] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for restoring transmission session streams in IP network traffic, characterized in that, include: Retrieve the data transmission response information for the data to be sent; Determine whether the data transmission response information is consistent with the preset baseline response information; Based on the inconsistency between the data transmission response information and the baseline response information, the computer is controlled to retransmit the data using a preset data retransmission method. Based on the consistency between the data transmission response information and the baseline response information, the device network status information is obtained; The network impact type is determined by analyzing the device network status information. The network impact types include bandwidth usage and network outage. Based on bandwidth usage, the bandwidth is processed using a preset bandwidth processing method to send the data; Based on network interruption, the network is reconnected using a preset reconnection method to continue sending data.

2. The method for restoring transmission session streams in IP network traffic according to claim 1, characterized in that, Data retransmission methods include: Acquire device input image; Determine whether the preset data transmission button has been pressed based on the input image from the device and the preset key button press actions; Based on the fact that the data transmission button was not pressed, the characteristics of the pressed button were determined according to the device input image. The deviation distance is determined based on the characteristics of the pressed button and the data transmission button. Based on the deviation distance not exceeding the preset single-button distance, an incorrect press is defined as an error press. The preset pressing device is controlled to press the preset cancel button to cancel the error press process, and the data send button is pressed to resend the data.

3. The method for restoring transmission session streams in IP network traffic according to claim 2, characterized in that, Also includes: Based on the data transmission button being pressed, the button pressing depth and the center position of the button are determined according to the device input image and the data transmission button before and after the button is pressed. Match the downward pressure intensity according to the depth of button press; The corrective force is determined based on the downward force and the preset force increment; The corrective force control pressing device presses the center position of the data transmission button again and obtains the data transmission response information of the data to be transmitted again. If the data transmission response information is inconsistent with the baseline response information, it is defined as a button failure. A hotkey combination scheme is determined based on the data transmission button and the preset hotkey database. The computer is set to generate combination hotkeys according to the hotkey combination scheme. The function of the combination hotkeys is the same as the function of the data sending key. The pressure of the control device is adjusted to press the combination hotkey to resend the data.

4. The method for restoring transmission session streams in IP network traffic according to claim 3, characterized in that, If there are foreign objects at the bottom of the data transmission button, it can also cause the button to malfunction. Methods for removing foreign objects include: The control pressing device presses the data transmission button with a corrective force, and determines the actual pressing depth based on the device input image before and after the button is pressed. Based on the discrepancy between the actual pressing depth and the preset benchmark pressing depth, it is defined as a foreign object obstruction, and image information of the button area is obtained; Determine whether the foreign object feature is completely obscured by the data transmission button based on the image information of the button area and the preset reference area image information; If the foreign object feature is completely obscured by the data transmission button, the control pressing device presses the center position of the data transmission button, and determines the tilt direction of the button based on the image information of the button area and the data transmission button. Determine the blowing position based on the tilt direction of the button; Control the preset air blowing device to blow air at the blowing position to blow out the foreign object feature from the bottom of the data transmission button; If the foreign object feature is not completely obscured by the data transmission button, the location of the foreign object feature is determined based on the image information of the button area and the foreign object feature. Determine the location of the maximum gap between adjacent keys based on the image information of the key area; A movement path is formed based on the location of the foreign object and the location of the maximum gap. The preset clamping device is controlled to clamp the foreign object at the location of the foreign object feature, and the foreign object feature is removed from the position of maximum gap by moving along the path.

5. The method for restoring transmission session streams in IP network traffic according to claim 1, characterized in that, Bandwidth processing methods include: Determine the average network speed based on the device's network status information; Based on the premise that the average network speed is no greater than the preset benchmark good network speed, obtain the data content of the data to be sent; Determine the sending priority level based on the content of the data to be sent; Based on a transmission priority level greater than a preset baseline priority, the average network speed of all computer devices on the same local area network is determined according to the device network status information. The average network speed of all computer devices is screened to identify devices whose average network speed exceeds the preset abnormal network speed, and the screen image information of these devices is obtained. The device user operation type is determined based on the image information and preset streaming video characteristics. The device user operation type includes watching streaming video and downloading data. Based on data download, the device network status information of the device user is controlled with a preset network speed limit in order to free up network bandwidth; Based on watching streaming video, the device network status information of the user is controlled at a preset critical network speed during the preset network outage period so that the data to be sent can be sent during the network outage period.

6. The method for restoring transmission session streams in IP network traffic according to claim 1, characterized in that, Methods for reconnecting to the network after a disconnection include: Obtain real-time network latency information; Analyze real-time network latency information to determine the types of network changes, including latency increases and latency fluctuations. Based on latency fluctuations, the maximum fluctuation is determined according to real-time network latency information; If the maximum fluctuation exceeds the preset applicable fluctuation range, the data to be sent is stored in the preset data buffer. When the maximum fluctuation falls into the applicable fluctuation range, the data to be sent in the data buffer is controlled to be sent. The operating status of the backup network is obtained based on the increasing delay. Based on the fact that the operating status of the backup network is consistent with the preset stable operating status, the local network of the computer equipment is switched to the preset backup network.

7. The method for restoring transmission session streams in IP network traffic according to claim 6, characterized in that, Also includes: Based on the inconsistency between the operating status of the backup network and the stable operating status, the transmission ratio of the data to be sent is obtained; The proportion of items not sent is determined based on the proportion of items sent. The local network of the control computer equipment is reconnected, and the unsent proportion of data to be sent is sent. The data to be sent according to the sending ratio and the data to be sent according to the non-sending ratio are merged to obtain merged data, and it is determined whether the merged data is consistent with the data to be sent. Based on the inconsistency between the merged data and the data to be sent, the control computer equipment retransmits the complete data to be sent.

8. The method for restoring transmission session streams in IP network traffic according to claim 1, characterized in that, Data transmission will be affected if the device loses power during the transmission process. Methods for handling device power outages include: When a data transmission response is triggered, the computer backs up the data to be transmitted to form backup data and stores it in a preset backup cache. Control the computer equipment to switch to a preset backup power supply and obtain the sent content information sent to the preset transmission buffer; Retrieve the backup data from the backup cache and match and analyze the backup data with the sent content information to obtain the unsent content information; Unsent content information is sent to the data buffer, merged with sent content information to form complete data to be sent, and then resent the complete data to be sent.

9. A system for restoring transmission session streams in IP network traffic, characterized in that, include: The acquisition module is used to acquire data transmission response information of the data to be sent, device network status information, device input image, key area image information, data content of the data to be sent, screen image information of the device user, real-time network latency information, operating status of the backup network, and transmission ratio of the data to be sent. A memory for storing a program for restoring a transmission session flow in IP network traffic as described in any one of claims 1 to 8; The processor and memory programs can be loaded and executed by the processor to implement methods for restoring transmission session streams in IP network traffic.

10. A smart terminal, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the method for restoring transport session flow in IP network traffic as described in any one of claims 1 to 8.

Images