Method and system for realizing picture segmentation based on picture segmentation device

Abstract

The invention discloses a method and a system for realizing picture segmentation based on a picture splitter, and the method for realizing picture segmentation comprises the following steps: a KVM receives DP display signals input by more than one piece of source equipment of a touch screen drive realization principle based on a Type-C input end; the DP display signal is converted into a corresponding HDMI (High Definition Multimedia Interface) signal; a plurality of paths of HDMI signals are input to the HDMI picture segmentation chip; and forming a split-screen display picture based on the HDMI picture segmentation chip. According to the invention, the mobile phone or the tablet computer is directly used as the source equipment to access the picture, so that the mouse can still be used after the touch screen equipment is accessed, and meanwhile, the mouse screen penetration can also be realized.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a method and system for implementing picture segmentation based on a picture segmentor. Background Art

[0002] With the rapid development of the Internet era, computer equipment has been widely used in people's daily life. At present, a single display screen can no longer meet the needs of users. Users usually switch back and forth among multiple display screens, so they usually use a screen splitter KVM to split the screen. In the current screen splitting, the mobile phone or tablet cannot be directly connected to the system as a source device. It needs to be connected through an expansion dock, and each mobile phone needs an expansion dock, which will undoubtedly greatly reduce the convenience of screen splitting. And after the mobile terminal or tablet is connected to the system, when the DP display signal output by the source device of these touch screen driver implementation principles is directly input to the KVM, the KVM cannot recognize and parse the DP display signal, resulting in the KVM being unable to directly perform screen splitting processing on the video source output as the mobile terminal or tablet device, making the mobile terminal or tablet unable to be integrated into the application scenario of screen splitting, affecting the application of these terminal devices in this field, thereby reducing the customer's experience. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides a method and system for implementing screen segmentation based on a screen splitter, which enables a mobile terminal or a tablet to be directly connected to a KVM as a source device, thereby enhancing the application of KVM.

[0004] In order to solve the above technical problems, the present invention provides a method for implementing screen segmentation based on a screen splitter, wherein the method is based on the screen splitter KVM accessing multiple different source devices, wherein the multiple different source devices include more than one source device of the touch screen drive implementation principle, and the method comprises:

[0005] The KVM receives the DP display signal input by the source device of the one or more touch screen driving implementation principles based on the Type-C input terminal;

[0006] Convert the DP display signal into a corresponding HDMI signal;

[0007] Input multiple HDMI signals to the HDMI screen splitting chip;

[0008] A split-screen display image is formed based on the HDMI image splitting chip.

[0009] Optionally, converting the DP display signal into a corresponding HDMI signal includes:

[0010] Splitting the DP display signal into an audio signal and a high-definition digital video signal;

[0011] Converting the high-definition digital video signal into a multimedia video signal;

[0012] The multimedia video signal and audio signal are input into a conversion chip for processing to obtain corresponding HDMI signals.

[0013] Optionally, forming a split-screen display image based on the HDMI image splitting chip includes:

[0014] The HDMI picture splitting chip performs multi-channel picture display on corresponding output channels based on the multi-channel HDMI signals.

[0015] Optionally, the method further includes:

[0016] When the KVM uses the screen segmentation technology to form a split-screen display screen using multiple input sources input by multiple different source devices, the USB keyboard and mouse control system enumerates the connected keyboard and mouse device types and keyboard and mouse information, and the keyboard and mouse information is generated by the first source device among the source devices of the one or more touch screen driving implementation principles;

[0017] Based on the keyboard and mouse information, a USB standard descriptor corresponding to the operating system on the screen channel is obtained;

[0018] Convert the keyboard and mouse device type and the keyboard and mouse information into a key event based on the corresponding USB standard descriptor;

[0019] The operating system on the screen channel responds to the key event.

[0020] Optionally, the USB keyboard and mouse control system enumerates the connected keyboard and mouse device types and keyboard and mouse information, including:

[0021] The USB keyboard and mouse control system enumerates the connected first source device and identifies the keyboard and mouse device type of the first source device;

[0022] Reading the keyboard and mouse information generated by the touch screen on the first source device based on the USB Host.

[0023] Optionally, the acquiring of a USB standard descriptor corresponding to the operating system on the screen channel based on the keyboard and mouse information includes:

[0024] triggering identification of a system type of the operating system;

[0025] A USB standard descriptor of the first source device under the operating system is extracted based on the system type.

[0026] Optionally, converting the keyboard and mouse device type and the keyboard and mouse information into a key event based on the corresponding USB standard descriptor includes:

[0027] When the system type is identified as a Windows / MACOS system, converting the keyboard and mouse information generated by the first source device into a key event under the absolute position mouse; or

[0028] When the system type is identified as an iOS system or an Andorid / HarmonyOS system or a non-mainstream operating system, the keyboard and mouse information generated by the first source device is converted into a key event under a relative position mouse or a touch screen device.

[0029] Optionally, when the system type is identified as an iOS system or an Andorid / HarmonyOS system or a non-mainstream operating system, converting the keyboard and mouse information generated by the first source device into a key event under a relative position mouse or a touch screen device includes:

[0030] When the system type is identified as an Andorid / HarmonyOS system, converting the left button pressing event of the absolute position mouse into a touch screen pressing event;

[0031] After converting into a touch screen press event, the mouse fills the channel coordinate value in the keyboard and mouse information into the touch screen absolute position based on the absolute position, and reports the channel coordinate value and touch press event to the operating system on the screen channel based on the touch screen device using the third interface descriptor.

[0032] Optionally, converting the left button pressing event of the absolute position mouse into a touch screen pressing event includes:

[0033] After detecting that the absolute position mouse triggers the left button press event, the button mark position of the touch screen device is converted to the pressed state.

[0034] In addition, the present invention also provides a system for implementing screen segmentation based on a screen segmenter, wherein the system is based on a screen segmenter KVM to access multiple different source devices, wherein the multiple different source devices include more than one source device of a touch screen drive implementation principle, and the system comprises:

[0035] DP display signal receiving module: used for the KVM to receive the DP display signal input by the source device of the one or more touch screen driving implementation principles based on the Type-C input terminal;

[0036] Signal conversion module: used to convert the DP display signal into a corresponding HDMI signal;

[0037] Signal input module: used to input multiple HDMI signals into the HDMI screen splitting chip;

[0038] Screen splitting module: used to form a split-screen display screen based on the HDMI screen splitting chip.

[0039] In an embodiment of the present invention, based on the Type-C input terminal receiving the DP display signal, the HDMI screen splitting chip uses the output channel to perform multi-screen splitting based on the HDMI signal generated by the display signal, so that multi-channel input can realize multi-screen display on a single display, and realizes direct access to a mobile phone or tablet as a source device, and can be connected to the system without an expansion dock, thereby improving the convenience of realizing screen splitting. Based on the USB keyboard and mouse control system, the keyboard and mouse information connected is enumerated, and the keyboard and mouse information is generated by the source device of the one or more touch screen driver implementation principles. The keyboard and mouse information is converted into a key event based on the corresponding USB standard descriptor. The operating system on the screen channel responds to the key event, and more reliable event information can be obtained. At the same time, there is no need to occupy too many computing resources, so as to avoid causing the obtained event information to deviate too much from the actual situation and slow down the mouse movement process of different screens of the system, so that the mouse can still be used after the touch screen device is connected, and the mouse can also be realized through the screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 is a hardware structure diagram of the screen segmentation system in the first embodiment of the present invention;

[0042] Figure 2 is a flow chart of a method for implementing screen segmentation based on a screen segmenter in a second embodiment of the present invention;

[0043] Figure 3 Schematic diagram of a method for implementing screen segmentation based on a screen segmenter in a third embodiment of the present invention

[0044] Figure 4 is a flow chart of a method for implementing screen segmentation based on a screen segmenter in a fourth embodiment of the present invention;

[0045] Figure 5 is a flowchart of a method for implementing screen segmentation based on a screen segmenter in a fifth embodiment of the present invention;

[0046] Figure 6 It is a schematic diagram of the structural composition of a system for implementing screen segmentation based on a screen segmenter in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Embodiment 1

[0049] The hardware structure diagram of the screen segmentation system provided by the embodiment of the present invention is as follows: Figure 1 As shown, the screen splitting system includes: a microprocessor, a TYPE-C input port, an HDMI input port, a USB interface, a conversion chip, an HDMI screen splitting chip, a USB Host chip and a USB HID Device. The microprocessor is connected to the HDMI screen splitting chip and the USB HID Device respectively, the USB HID Device is connected to the USB Host chip and the USB interface respectively, the HDMI screen splitting chip is connected to the conversion chip, the HDMI output port and the HDMI input port respectively, the conversion chip is connected to the TYPE-C input port, the HDMI input port is connected to a source device implemented by a non-touch screen driving principle, the TYPE-C input port is connected to a source device implemented by a touch screen driving principle, the HDMI output port is connected to a display, and the mouse / keyboard is connected to the USB Host chip.

[0050] It should be noted that the HDMI screen splitting chip can be connected to the TYPE-C input port through the conversion chip, that is, the source device implemented by the touch screen driving principle can be connected to the HDMI screen splitting chip through the TYPE-C input port. In the embodiment of the present invention, multiple TYPE-C input ports can be set through the HDMI screen splitting chip to connect the source device implemented by the touch screen driving principle, that is, it supports that all source devices can be source devices implemented by the touch screen driving principle. Similarly, the HDMI screen splitting chip here can partially access the source device implemented by the touch screen driving principle through the TYPE-C input port, and some source devices can be other types of source devices, that is, connected to the source device implemented by the non-touch screen driving principle through the HDMI input port, which can support the source device implemented by the non-touch screen driving principle to access the HDMI screen splitting chip.

[0051] That is, the HDMI screen splitting chip in the embodiment of the present invention can simultaneously support the access of source devices implemented by the touch screen driving principle and source devices implemented by the non-touch screen driving principle, and also the situation where the HDMI screen splitting chip fully supports the access of source devices implemented by the touch screen driving principle.

[0052] The microprocessor can be a high-performance and low-power 32-bit microcontroller with ultra-high computing power and large-capacity memory, which can meet the efficient processing of multiple systems. The USB Host chip can include two USBHost ports, which can connect to two USB HID keyboard and mouse devices. The USB Host chip can embed a USB controller and dual USB transceivers, support USB-Host host mode and USB-Device device mode, and provide rich interface resources. The chip has the ability to enumerate and control USB keyboards and mice, and the USB Host reads raw data. The USB HID Device can use an enhanced E8051 core microcontroller compatible with the MCS51 instruction set. This chip supports multiple USB input endpoints. The HDMI screen segmentation chip is used for screen segmentation, the conversion chip is used for display signal conversion, and the fast charging interface is used to charge the source device connected to the system.

[0053] DP display signals of different operating systems are received through different TYPE-C input ports. The operating system here is the operating system of the source device implemented by the touch screen driving principle. The source device connected is a mobile terminal of Android / HarmonyOS system or IOS system. After the display signal of the mobile terminal is connected, the display signal needs to be converted through the conversion chip, and then the display signal is input to the HDMI screen splitting chip for processing. The DP display signals of different operating systems are also connected through the HDMI input port. The operating system connected here is Windows / MACOS or a non-mainstream operating system. The display signal connected here can be directly input to the HDMI screen splitting chip for processing without the signal conversion of the conversion chip, so as to realize multi-screen display. Therefore, the display signals of different operating systems are connected in different ways, which can be adjusted according to the actual situation, such as connecting four source devices that are all implemented by the touch screen driving principle, or connecting one source device that is implemented by the touch screen driving principle, and three source devices that are implemented by the non-touch screen driving principle, etc. The specific situation is determined by the actual situation of the user. The USB Host chip reads the keyboard and mouse data transmitted by the USB HID Device, and the USB Host chip parses the acquired keyboard and mouse data and converts them into corresponding keyboard and mouse events and reports them to the operating system of the screen. The microprocessor of the operating system of the screen responds to and processes events to realize screen segmentation and mouse crossing in the segmented screen.

[0054] In the embodiment of the present invention, a mobile phone or tablet is directly connected as a source device, and the system can be connected without a docking station, which improves the convenience of realizing screen segmentation. The mouse crossing between different systems can be realized through the cooperation of the microprocessor, the HDMI screen segmentation chip, the USB Host chip and the USB HID Device, so that the mouse can still be used after the touch screen device is connected, and the mouse can also cross the screen.

[0055] Embodiment 2

[0056] See also Figure 2 , Figure 2 It is a flowchart of a method for implementing screen segmentation based on a screen segmenter in Embodiment 2 of the present invention.

[0057] The method is based on the screen splitter KVM access to multiple different source devices, the multiple different source devices include more than one source device of the touch screen driver implementation principle, the source device of the touch screen driver implementation principle may include a mobile phone or tablet with an IOS system and an Andorid / HarmonyOS system. The method for implementing screen splitting includes:

[0058] S011: The KVM receives, based on a Type-C input terminal, DP display signals input by source devices implementing the one or more touch screen driving principles;

[0059] In the specific implementation process of the present invention, based on the Type-C input end receiving the DP display signal input by the source device of the one or more touch screen driving implementation principles, the Type-C input end has multiple channels, each Type-C input end corresponds to a display signal of a system, and realizes the input of multiple display signals. Currently, most of the access display signals use a high-definition multimedia interface (HDMI) input, and the Type-C interface input end is used here. Its signal transmission speed and durability are faster than the traditional display interface. At the same time, the Type-C interface input end supports the synchronous transmission of multiple information, which is an advantage that other input ports do not have.

[0060] S012: Convert the DP display signal into a corresponding HDMI signal;

[0061] In the specific implementation process of the present invention, converting the DP display signal into the corresponding HDMI signal includes: splitting the DP display signal into an audio signal and a high-definition digital video signal; converting the high-definition digital video signal into a multimedia video signal; inputting the multimedia video signal and the audio signal into a conversion chip for processing to obtain a corresponding HDMI signal.

[0062] Specifically, the DP display signal is split and processed to obtain an audio signal and a high-definition digital video signal, and the high-definition digital video signal is converted into a multimedia video signal through a conversion circuit; the multimedia video signal and the audio signal are input into a conversion chip for processing, and a vertical synchronization signal is generated using the multimedia video signal and the audio signal according to the data packet of the adaptive vertical synchronization technology, and a horizontal synchronization signal is generated according to a blank revelation signal of the display signal, wherein the vertical synchronization signal and the horizontal synchronization signal are generated within a half period of a horizontal line defined by two blank start signals of the multiple display signals after receiving the data packet signal of the adaptive vertical synchronization technology, and an HDMI signal is generated according to the so-called vertical synchronization signal and the horizontal synchronization signal to obtain a corresponding HDMI signal.

[0063] S013: Input multiple HDMI signals to the HDMI screen splitting chip;

[0064] In the specific implementation process of the present invention, the HDMI signal is transmitted to an HDMI screen splitting chip, which can be an HDMI multi-input single-output chip or an HDMI multi-input multi-output chip. The HDMI screen splitting chip has the ability to display multiple input images simultaneously on the output channel.

[0065] S014: forming a split-screen display image based on the HDMI image splitting chip.

[0066] In a specific implementation of the present invention, the forming of a split-screen display image based on the HDMI image splitting chip includes: the HDMI image splitting chip performs multi-channel image display on corresponding output channels based on the multi-channel HDMI signals.

[0067] Specifically, the HDMI screen segmentation chip performs multi-screen segmentation based on the HDMI signal using the output channel, decodes the HDMI signal to obtain the decoded HDMI signal, extracts the effective screen area in the decoded HDMI signal to obtain the effective screen data, performs fade-in and fade-out processing on the extracted effective screen data to obtain the effective screen data after the fade-in and fade-out processing, performs scaling processing on the effective screen data after the fade-in and fade-out processing to obtain the effective screen data after the scaling processing, and uses the output channel to perform multi-channel screen output on the effective screen data after the scaling processing to obtain a number of segmented screens, thereby achieving multi-channel input on a single display to realize dual-screen or quad-screen multi-screen display.

[0068] In the embodiment of the present invention, based on the Type-C input terminal receiving the DP display signal, the HDMI screen splitting chip uses the output channel to perform multi-screen splitting based on the HDMI signal generated by the display signal, and can achieve multi-screen display on a single display with multiple inputs, and realize direct access of a mobile phone or tablet as a source device, and can access the system without an expansion dock, thereby improving the convenience of realizing screen splitting.

[0069] Embodiment 3

[0070] See also Figure 3 , Figure 3 It is a flow chart of a method for implementing screen segmentation based on a screen splitter in Embodiment 3 of the present invention. The method is based on a screen splitter KVM accessing multiple different source devices, the multiple different source devices include more than one source device implementing the touch screen driver principle, the KVM uses screen segmentation technology to display the multiple input sources input by the multiple different source devices on the connected display device, the more than one source device implementing the touch screen driver principle is connected to the KVM based on the Type-C input terminal, and the method includes:

[0071] S111: The KVM receives, based on a Type-C input terminal, DP display signals input by source devices of the one or more touch screen driving implementation principles;

[0072] S112: Convert the DP display signal into a corresponding HDMI signal;

[0073] S113: Input multiple HDMI signals to the HDMI screen splitting chip;

[0074] S114: forming a split-screen display image based on the HDMI image splitting chip

[0075] S115: when the KVM uses the screen segmentation technology to form a split-screen display screen for multiple input sources input by multiple different source devices, enumerate the types of connected keyboard and mouse devices and keyboard and mouse information based on the USB keyboard and mouse control system, and the keyboard and mouse information is generated by the first source device among the source devices of the one or more touch screen driving implementation principles;

[0076] Specifically, a split-screen display screen is formed by using screen splitting technology through multiple input sources, and source devices of different operating systems can be connected. The type of connected keyboard and mouse devices and keyboard and mouse information are enumerated through the USB keyboard and mouse control system to obtain specific operation information for subsequent judgment operations.

[0077] S116: Acquire a USB standard descriptor corresponding to the operating system on the screen channel based on the keyboard and mouse information;

[0078] Specifically, different keyboard and mouse device types can be enumerated through USB standard descriptors corresponding to different operating systems to support different systems.

[0079] S117: Converting the keyboard and mouse device type and the keyboard and mouse information into a key event based on the corresponding USB standard descriptor;

[0080] Specifically, an event reporting strategy corresponding to each operating system can be formed through the corresponding USB standard descriptor, which can be compatible with different operating systems to achieve mouse crossing control between multiple mobile phone screens.

[0081] S118: The operating system on the screen channel responds to the key event.

[0082] In the embodiment of the present invention, the USB keyboard and mouse control system enumerates the connected keyboard and mouse information, which is generated by the source device of the one or more touch screen driver implementation principles, and converts the keyboard and mouse information into key events based on the corresponding USB standard descriptor. The operating system on the screen channel responds to the key events, which can obtain more reliable event information without occupying too many computing resources, avoiding the deviation between the obtained event information and the actual situation and slowing down the mouse movement process of different screens of the system, so that the mouse can still be used after the touch screen device is connected, and the mouse can also be crossed through the screen. In this way, the mouse crossing control of the split screen between the touch screen source devices of different operating systems can be realized, so that the mouse crossing control of different system screens can achieve a more ideal effect.

[0083] Embodiment 4

[0084] See also Figure 4 , Figure 4 : is a flow chart of a method for implementing screen segmentation based on a screen splitter in a fourth embodiment of the present invention, wherein the method is based on a screen splitter KVM accessing multiple different source devices, wherein the multiple different source devices include more than one source device implementing the touch screen driver principle, and the KVM displays multiple input sources input by the multiple different source devices on the connected display device using a screen segmentation technology, wherein the more than one source device implementing the touch screen driver principle is connected to the KVM based on a Type-C input terminal, and the method includes:

[0085] S210: configuring a USB standard descriptor corresponding to the operating system in the screen splitter in combination with the keyboard and mouse device type;

[0086] When the keyboard and mouse device type is an absolute position mouse, a corresponding first interface descriptor is configured for the operating system, the first interface descriptor includes a first HID class descriptor and an endpoint descriptor, and the data format reported to the operating system by the first HID class descriptor is eight bytes, and the eight bytes are Report ID, a mouse button flag, an arbitrary value, a wheel value, the upper 8 bits of the absolute value of the horizontal coordinate, the lower 8 bits of the absolute value of the horizontal coordinate, the lower 8 bits of the absolute value of the vertical coordinate, and the upper 8 bits of the absolute value of the vertical coordinate, as shown in the following table:

[0087]

[0088] When the keyboard and mouse device type is a relative position mouse, a corresponding second interface descriptor is configured for the operating system, the second interface descriptor includes a second HID class descriptor and an endpoint descriptor, and the data format reported by the second HID class descriptor to the operating system is six bytes, and the six bytes are a mouse button flag, an arbitrary value, the upper 8 bits of the horizontal coordinate relative displacement, the lower 8 bits of the horizontal coordinate relative displacement, the lower 8 bits of the vertical coordinate relative displacement, and the upper 8 bits of the vertical coordinate relative displacement, as shown in the following table:

[0089]

[0090] When the keyboard and mouse device type is a touch screen or a button, a corresponding third interface descriptor is configured for the operating system, the third interface descriptor includes a third HID class descriptor and an endpoint descriptor, and the data format reported by the third HID class descriptor to the operating system is six bytes, and the six bytes are Report ID, a touch button state flag, the upper 8 bits of the absolute value of the horizontal coordinate, the lower 8 bits of the absolute value of the horizontal coordinate, the lower 8 bits of the absolute value of the vertical coordinate, and the upper 8 bits of the absolute value of the vertical coordinate, as shown in the following table:

[0091]

[0092] S211: accessing multiple different source devices based on the screen splitter KVM;

[0093] In a specific implementation of the present invention, a plurality of different source devices are connected based on a plurality of input ports in the screen splitter KVM, and the source devices include mobile phones or tablets with different operating systems.

[0094] S212: The KVM uses a screen splitting technology to form a split-screen display screen using multiple input sources input from multiple different source devices;

[0095] In the specific implementation process of the present invention, the KVM receives multiple input sources input by different source devices, and performs screen segmentation of different systems through a screen segmentation chip to realize split-screen display.

[0096] S213: when the KVM uses the screen segmentation technology to form a split-screen display screen for multiple input sources input by multiple different source devices, enumerate the types of connected keyboard and mouse devices and keyboard and mouse information based on the USB keyboard and mouse control system;

[0097] In the specific implementation process of the present invention, when the KVM uses the screen segmentation technology to form a split-screen display screen with multiple input sources input by multiple different source devices, the USB keyboard and mouse control system enumerates the connected keyboard and mouse device types and keyboard and mouse information. The USB keyboard and mouse control system includes a USB Host and multiple USB HID Device devices. The USBHost includes two USB Host ports, which can connect to two USB HID keyboard and mouse devices. It has an embedded USB controller and dual USB transceivers, supports USB-Host host mode and USB-Device device mode, and provides rich interface resources. The chip has the ability to enumerate and control USB keyboards and mice, and the USB Host reads raw data. The USB Device can use an enhanced E8051 core microcontroller compatible with the MCS51 instruction set. This chip supports multiple USB input endpoints. The USB Host sends the channel coordinate value to the USB Device of the input channel where the mouse is located based on the serial peripheral interface (Serial Peripheral Interface, SPI) chip select signal line. SPI is a high-speed full-duplex communication bus. It is output to the corresponding enable pin through the SPI chip select signal line. Data is transmitted according to the level and potential of the preset pulse width modulation signal. After receiving the corresponding data, the USB Device of the input channel where the mouse is located will report the data to the operating system, while the USB Device of other input channels will not receive the data because its SPI chip select signal line is not selected, so it will not respond. The types of keyboard and mouse devices connected are enumerated, including touch screen, relative position mouse, absolute position mouse and keyboard. The keyboard and mouse information includes the current position of the mouse, system and relative displacement, etc.

[0098] The types of keyboard and mouse devices connected are enumerated, including touch screen, relative position mouse, absolute position mouse and keyboard, as shown in the following table:

[0099]

[0100] S214: determining whether the mouse has moved across the screen based on the keyboard and mouse information;

[0101] In the specific implementation process of the present invention, the relative displacement of the mouse is extracted based on the keyboard and mouse information, the screen channel value where the mouse is located is calculated based on the relative displacement, and whether the mouse has moved across the screen is determined based on the screen channel value where the mouse is located.

[0102] S215: when it is determined that the mouse has moved the screen, the system type of the system for which the mouse has moved the screen is determined;

[0103] During the specific implementation of the present invention, the device system will provide a button for the user to mark, that is, the system mark can be obtained according to the button, and the system type to which the current Type-C source is connected can be memorized according to the mark. Different systems correspond to different marks, including IOS system and Andorid / HarmonyOS system.

[0104] S216: If it is determined that the mouse crosses from the screen where the IOS system is located to the screen where the Windows / MACOS system is located, and the keyboard and mouse device types supported by the Windows / MACOS system include absolute position mice, extracting a first interface descriptor;

[0105] During the specific implementation of the present invention, if it is determined that the mouse has crossed from the screen where the IOS system is located to the screen where the Windows / MACOS system is located, since the keyboard and mouse device types supported by the Windows / MACOS system include absolute position mouse, when the keyboard and mouse device type is an absolute position mouse, the corresponding first interface descriptor is configured for the operating system. Therefore, the first interface descriptor is extracted, and event data reporting between different operating systems can be achieved through conversion of USB standard descriptors. When it is identified that the screen crossed by the mouse is of a different system, the corresponding USB standard descriptor can be converted according to the identified system type through the GET_DESCRIPTOR request.

[0106] S2161: Convert the keyboard and mouse device type and the keyboard and mouse information into a key event based on the first interface descriptor;

[0107] In the specific implementation of the present invention, the mouse channel coordinate value, the wheel event and the mouse button event are reported to the operating system in the format configured by the first interface descriptor. Different systems have different event reporting strategies, as shown in the following table:

[0108]

[0109]

[0110] S2162: The Windows / MACOS system on the screen channel where the mouse is traversing responds to a key event;

[0111] During the specific implementation of the present invention, the IOS system on the screen channel where the mouse traverses establishes a socket communication interface with the Windows / MACOS system, obtains the keyboard and mouse data contained in the event from the diplay_ios function, decodes the contained keyboard and mouse data according to the encoding protocol, generates decoded keyboard and mouse data, and sends the decoded keyboard and mouse data to the processor of the Windows / MACOS system. The processor of the system calls the corresponding framework to respond accordingly to the decoded keyboard and mouse event.

[0112] S217: if it is determined that the mouse crosses from the screen where the IOS system is located to the screen where the IOS system is located, and the keyboard and mouse device types supported by the IOS system include absolute position mouse and relative position mouse, extract the first interface descriptor and the second interface descriptor in the USB standard descriptor;

[0113] In the specific implementation process of the present invention, the IOS system can support the pointer display of the absolute position mouse, but does not respond to the button events of the absolute position mouse. Therefore, the relative position mouse is used to report events to achieve better compatibility. Therefore, the keyboard and mouse device types supported by the IOS system include absolute position mouse and relative position mouse. When the keyboard and mouse device type is an absolute position mouse, the corresponding first interface descriptor is configured for the operating system. When the keyboard and mouse device type is a relative position mouse, the corresponding second interface descriptor is configured for the operating system. Therefore, the first interface descriptor and the second interface descriptor in the USB standard descriptor are extracted. The relative position mouse is the pointer relative position control, which can be achieved by converting the USB standard descriptor. The event data reporting between the same operating system realizes the conversion of descriptor information through the GET_DESCRIPTOR request to achieve mouse control between different systems. When it is recognized that the screen crossed by the mouse is a different system, the corresponding USB standard descriptor can be converted according to the recognized system type. The second interface descriptor includes a second HID class descriptor and an endpoint descriptor. The data format reported by the second HID class descriptor to the operating system is six bytes. The six bytes are the mouse button flag, an arbitrary value, the high 8 bits of the horizontal coordinate relative displacement, the low 8 bits of the horizontal coordinate relative displacement, the low 8 bits of the vertical coordinate relative displacement and the high 8 bits of the vertical coordinate relative displacement. For any value, the system will ignore this field. The IOS system can support the pointer display of the absolute position mouse, but does not respond to the button event of the absolute position mouse. Therefore, the relative position mouse wheel and button are used for reporting to achieve better compatibility.

[0114] S2171: Convert the keyboard and mouse device type and the keyboard and mouse information into a key event based on the corresponding USB standard descriptor;

[0115] During the specific implementation of the present invention, the keyboard and mouse device type and keyboard and mouse information are reported as events through the corresponding interface descriptors. When reporting to the operating system, the channel coordinate value is first reported through the absolute position mouse device, and no mouse button is pressed at this time; then, the position offset of the relative position mouse is set to 0, and the key event of the absolute position mouse is filled into the relative position mouse button position, so as to achieve the effect of no displacement control of the relative position mouse device and only reporting of mouse key events.

[0116] S2172: The iOS system on the screen channel where the mouse passes through responds to the key event;

[0117] In the specific implementation of the present invention, the same operating system can perform corresponding keyboard and mouse operations according to the keyboard and mouse data included in the received key events.

[0118] S218: if it is determined that the mouse crosses from the screen where the IOS system is located to the screen where the Andorid / HarmonyOS system is located, and the keyboard and mouse device types supported by the Andorid / HarmonyOS system include relative position mouse and touch screen device, extract the second interface descriptor and the third interface descriptor in the USB standard descriptor;

[0119] During the specific implementation of the present invention, when the mouse passes from the screen where the IOS system is located to the screen where the Andorid / HarmonyOS system is located, since the Andorid / HarmonyOS system does not support the absolute position mouse, the keyboard and mouse device type is changed to a touch screen device to achieve the mouse pointer display, that is, the mouse pointer position movement is achieved by converting to the state of a simulated touch screen device, and the corresponding standard descriptor is converted by calling a request function. The keyboard and mouse device types supported by the Andorid / HarmonyOS system include relative position mouse and touch screen device. When the keyboard and mouse device type is a touch screen or a button, the corresponding third interface descriptor is configured for the operating system. When the keyboard and mouse device type is a relative position mouse, the corresponding second interface descriptor is configured for the operating system, and thus the second interface descriptor and the third interface descriptor are extracted.

[0120] S2181: Convert the keyboard and mouse device type and the keyboard and mouse information into a key event based on the corresponding USB standard descriptor;

[0121] In the specific implementation process of the present invention, the keyboard and mouse device type and keyboard and mouse information are reported as events based on the corresponding USB standard descriptor. When reporting to the operating system, the left button press event of the absolute position mouse is first converted into a touch screen press event, that is, when the left button of the mouse is detected to be pressed, the button mark position of the touch screen device is converted into a pressed state, and then the channel coordinate value of the absolute position mouse is filled into the absolute position position of the touch screen, and the channel coordinate value and the touch press event are reported to the operating system through the touch screen device. If there are wheel and other mouse button events, the displacement of the relative position mouse will be set to 0, and then the wheel and other mouse button events will be filled into the relative position mouse device button position and sent to the operating system. By filling the information into the corresponding mark position, data conversion can be realized between different systems to realize event information reporting between different systems.

[0122] S2182: The Android / HarmonyOS system on the screen channel where the mouse passes through responds to a key event;

[0123] During the specific implementation of the present invention, a data transmission protocol is defined, a port for signaling interaction is declared, and an event transmission channel is established based on the defined data transmission protocol and the port for signaling interaction. The Andorid / HarmonyOS system on the screen channel where the mouse is crossing obtains key events based on the event transmission channel, and makes corresponding response processing based on the key events, thereby realizing mouse crossing control between source devices of different operating systems.

[0124] S219: if it is determined that the mouse has crossed from the screen where the IOS system is located to the screen where the non-mainstream operating system is located, and the keyboard and mouse device types supported by the non-mainstream operating system include absolute position mouse and relative position mouse, extract the first interface descriptor and the second interface descriptor in the USB standard descriptor;

[0125] In the specific implementation process of the present invention, the keyboard and mouse device types supported by the non-mainstream operating system include absolute position mouse and relative position mouse. Therefore, the first interface descriptor and the second interface descriptor are extracted, also through the corresponding conversion function. When it is identified that the non-mainstream operating system has been crossed over, it is converted into the configured USB standard descriptor corresponding to the non-mainstream operating system through the conversion function.

[0126] S2191: Convert the keyboard and mouse device type and the keyboard and mouse information into a key event based on the corresponding USB standard descriptor;

[0127] In the specific implementation process of the present invention, the keyboard and mouse device type and keyboard and mouse information are reported as events based on the corresponding USB standard descriptor, the keyboard and mouse device type and keyboard and mouse information are reported as enumerated data based on the corresponding USB standard descriptor, and the relevant data is converted into corresponding pointer events for reporting. For the reporting of the channel coordinate value of the keyboard and mouse information in the non-mainstream system, the channel coordinate value can also be reported based on the absolute position mouse and the touch screen device at the same time. The reporting strategy of the system can also use the absolute position mouse to report the channel coordinate value once, and then use the touch screen device to report the channel coordinate value.

[0128] S2192: The non-mainstream operating system on the screen channel where the mouse passes through responds to the key event.

[0129] During the specific implementation of the present invention, the non-mainstream operating system on the screen channel where the mouse is located performs regularized preprocessing on the keyboard and mouse information contained in the acquired key events, creates a compilation thread, divides the keyboard and mouse information contained in the key events into concurrent components, and uses the compilation thread to convert the information contained in the key events into a compiled language recognizable by the non-mainstream operating system, so that the non-mainstream system can make corresponding response processing.

[0130] In the embodiment of the present invention, the keyboard and mouse information connected is enumerated based on the USB keyboard and mouse control system. The keyboard and mouse information is generated by the source device of the one or more touch screen driver implementation principles. The keyboard and mouse information is converted into key events based on the corresponding USB standard descriptor. The operating system on the screen channel responds to the key events, which can obtain more reliable event information without occupying too many computing resources, avoiding the deviation between the obtained event information and the actual situation and slowing down the mouse movement process of different screens of the system, so that the mouse can still be used after the touch screen device is connected, and the mouse can also be crossed through the screen. In this way, the mouse crossing control of the split screen between the touch screen source device with the IOS operating system and the other devices with different operating systems can be realized, so that the mouse crossing control of different system screens can achieve a more ideal effect.

[0131] Embodiment 5

[0132] See also Figure 5 , Figure 5 is a flow chart of a method for implementing screen segmentation based on a screen segmenter in a fifth embodiment of the present invention, the method comprising:

[0133] S310: configuring a USB standard descriptor corresponding to the operating system in the screen splitter in combination with the keyboard and mouse device type;

[0134] S311: accessing multiple different source devices based on the screen splitter KVM;

[0135] S312: The KVM uses a screen splitting technology to form a split-screen display screen using multiple input sources input from multiple different source devices;

[0136] S313: when the KVM uses the screen segmentation technology to form a split-screen display screen for multiple input sources input by multiple different source devices, enumerate the types of connected keyboard and mouse devices and keyboard and mouse information based on the USB keyboard and mouse control system;

[0137] S314: determining whether the mouse has moved across the screen based on the keyboard and mouse information;

[0138] S315: when it is determined that the mouse has moved the screen, the system type of the system for which the mouse has moved the screen is determined;

[0139] S316: if it is determined that the mouse crosses from the screen where the Andorid / HarmonyOS system is located to the screen where the Windows / MACOS system is located, and the keyboard and mouse device types supported by the Windows / MACOS system include absolute position mice, extracting a first interface descriptor;

[0140] During the specific implementation of the present invention, if it is determined that the mouse has crossed from the screen where the Andorid / HarmonyOS system is located to the screen where the Windows / MACOS system is located, since the keyboard and mouse device types supported by the Windows / MACOS system include absolute position mouse, when the keyboard and mouse device type is an absolute position mouse, the corresponding first interface descriptor is configured for the operating system. Therefore, the first interface descriptor is extracted, and event data reporting between different operating systems can be achieved through the conversion of USB standard descriptors. When it is identified that the screen crossed by the mouse is of a different system, the corresponding USB standard descriptor can be converted according to the identified system type through the GET_DESCRIPTOR request.

[0141] S3161: Convert the keyboard and mouse device type and the keyboard and mouse information into a key event based on the first interface descriptor;

[0142] S3162: The Windows / MACOS system on the screen channel where the mouse passes through responds to the key event;

[0143] During the specific implementation of the present invention, the Andorid / HarmonyOS system on the screen channel where the mouse traverses establishes a socket communication interface with the Windows / MACOS system, obtains the keyboard and mouse data contained in the event from the diplay_ios function, decodes the contained keyboard and mouse data according to the encoding protocol, generates decoded keyboard and mouse data, and sends the decoded keyboard and mouse data to the processor of the Windows / MACOS system. The processor of the system calls the corresponding framework to respond accordingly to the decoded keyboard and mouse event.

[0144] S317: if it is determined that the mouse crosses from the screen where the Andorid / HarmonyOS system is located to the screen where the IOS system is located, and the keyboard and mouse device types supported by the IOS system include absolute position mouse and relative position mouse, extract the first interface descriptor and the second interface descriptor in the USB standard descriptor;

[0145] During the specific implementation of the present invention, when it is determined that the mouse has crossed from the screen where the Andorid / HarmonyOS system is located to the screen where the IOS system is located, the conversion function and the request function are called to convert the corresponding USB standard descriptor through the keyboard and mouse device types supported by the IOS system. The keyboard and mouse device types supported by the IOS system include absolute position mouse and relative position mouse, and the first interface descriptor and the second interface descriptor in the USB standard descriptor can be extracted.

[0146] S3171: Convert the keyboard and mouse device type and keyboard and mouse information into a key event based on the corresponding USB standard descriptor;

[0147] S3172: The iOS system on the screen channel where the mouse passes through responds to the key event;

[0148] In the specific implementation process of the present invention, a data transmission protocol is defined, a port for signaling interaction is declared, and an event transmission channel is established based on the defined data transmission protocol and the port for signaling interaction. The IOS system on the screen channel where the mouse is crossing obtains key events based on the event transmission channel, and makes corresponding response processing based on the key events, thereby realizing mouse crossing control between source devices of different operating systems.

[0149] S318: if it is determined that the mouse crosses from the screen where the Android / HarmonyOS system is located to the screen where the Android / HarmonyOS system is located, and the keyboard and mouse device types supported by the Android / HarmonyOS system include relative position mouse and touch screen device, extract the second interface descriptor and the third interface descriptor in the USB standard descriptor;

[0150] S3181: Convert the keyboard and mouse device type and keyboard and mouse information into a key event based on the corresponding USB standard descriptor;

[0151] S3182: The Android / HarmonyOS system on the screen channel where the mouse passes through responds to the key event;

[0152] S319: if it is determined that the mouse has crossed from the screen where the Andorid / HarmonyOS system is located to the screen where the non-mainstream operating system is located, and the keyboard and mouse device types supported by the non-mainstream operating system include absolute position mouse and relative position mouse, extract the first interface descriptor and the second interface descriptor in the USB standard descriptor;

[0153] In the specific implementation process of the present invention, the keyboard and mouse device types supported by the non-mainstream operating system include absolute position mouse and relative position mouse. Therefore, the first interface descriptor and the second interface descriptor are extracted, also through the corresponding conversion function. When it is identified that the non-mainstream operating system has been crossed over, it is converted into the configured USB standard descriptor corresponding to the non-mainstream operating system through the conversion function.

[0154] S3191: Convert the keyboard and mouse device type and the keyboard and mouse information into a key event based on the corresponding USB standard descriptor;

[0155] S3192: The non-mainstream operating system on the screen channel that the mouse passes through responds to a key event.

[0156] During the specific implementation of the present invention, the non-mainstream operating system on the screen channel where the mouse is located performs regularized preprocessing on the keyboard and mouse information contained in the acquired key events, creates a compilation thread, divides the keyboard and mouse information contained in the key events into concurrent components, and uses the compilation thread to convert the information contained in the key events into a compiled language recognizable by the non-mainstream operating system, so that the non-mainstream system can make corresponding response processing.

[0157] In the embodiment of the present invention, the connected keyboard and mouse information is enumerated based on the USB keyboard and mouse control system. The keyboard and mouse information is generated by the source device of the one or more touch screen driver implementation principles. The keyboard and mouse information is converted into key events based on the corresponding USB standard descriptor. The operating system on the screen channel responds to the key events, which can obtain more reliable event information without occupying too many computing resources, avoiding the deviation between the obtained event information and the actual situation and slowing down the mouse movement process of different screens of the system, so that the mouse can still be used after the touch screen device is connected, and the mouse can also cross the screen. In this way, the mouse crossing control of the split screen between the touch screen source device with the Andorid / HarmonyOS system and the other devices of different systems can be realized, so that the mouse crossing control of different system screens can achieve a more ideal effect.

[0158] Embodiment 6

[0159] See also Figure 6 , Figure 6 : is a schematic diagram of the structure of a system for implementing screen segmentation based on a screen splitter in an embodiment of the present invention, wherein the system is based on a screen splitter KVM to access multiple different source devices, wherein the multiple different source devices include more than one source device of a touch screen drive implementation principle, and the system includes:

[0160] DP display signal receiving module 61: used for the KVM to receive DP display signals input by the source devices of the one or more touch screen driving implementation principles based on the Type-C input terminal;

[0161] Signal conversion module 62: used for converting the DP display signal into a corresponding HDMI signal;

[0162] Signal input module 63: used for inputting multiple HDMI signals into the HDMI screen splitting chip;

[0163] The screen splitting module 64 is used to form a split-screen display screen based on the HDMI screen splitting chip.

[0164] In the specific implementation process of the present invention, the specific implementation method of the system item can refer to the implementation method of the above-mentioned method item, which will not be repeated here.

[0165] In the embodiment of the present invention, based on the Type-C input terminal receiving the DP display signal, the HDMI screen splitting chip uses the output channel to perform multi-screen splitting based on the HDMI signal generated by the display signal, and can achieve multi-screen display on a single display with multiple inputs, and realize direct access of a mobile phone or tablet as a source device, and can access the system without an expansion dock, thereby improving the convenience of realizing screen splitting.

[0166] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0167] In addition, the above is a detailed introduction to a method and system for realizing screen segmentation based on a screen splitter provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for a person skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for implementing screen segmentation based on a screen segmenter, characterized in that: The method is based on accessing a plurality of different source devices by a screen splitter KVM, wherein the plurality of different source devices include more than one source device of a touch screen driving implementation principle, and the method comprises: The KVM receives the DP display signal input by the source device of the one or more touch screen driving implementation principles based on the Type-C input terminal; Convert the DP display signal into a corresponding HDMI signal; Input multiple HDMI signals to the HDMI screen splitting chip; A split-screen display image is formed based on the HDMI image splitting chip.

2. The method for implementing screen segmentation based on a screen segmenter according to claim 1, characterized in that: The converting the DP display signal into a corresponding HDMI signal comprises: Splitting the DP display signal into an audio signal and a high-definition digital video signal; Converting the high-definition digital video signal into a multimedia video signal; The multimedia video signal and audio signal are input into a conversion chip for processing to obtain corresponding HDMI signals.

3. The method for implementing screen segmentation based on a screen segmenter according to claim 1, characterized in that: The forming of a split-screen display image based on the HDMI image splitting chip comprises: The HDMI picture splitting chip performs multi-channel picture display on corresponding output channels based on the multi-channel HDMI signals.

4. The method for implementing screen segmentation based on a screen segmenter according to any one of claims 1 to 3, characterized in that: The method further comprises: When the KVM uses the screen segmentation technology to form a split-screen display screen using multiple input sources input by multiple different source devices, the USB keyboard and mouse control system enumerates the connected keyboard and mouse device types and keyboard and mouse information, and the keyboard and mouse information is generated by the first source device among the source devices of the one or more touch screen driving implementation principles; Based on the keyboard and mouse information, a USB standard descriptor corresponding to the operating system on the screen channel is obtained; Convert the keyboard and mouse device type and the keyboard and mouse information into a key event based on the corresponding USB standard descriptor; The operating system on the screen channel responds to the key event.

5. The method for implementing screen segmentation based on a screen segmenter according to claim 4, characterized in that: The USB keyboard and mouse control system enumerates the connected keyboard and mouse device types and keyboard and mouse information, including: The USB keyboard and mouse control system enumerates the connected first source device and identifies the keyboard and mouse device type of the first source device; Reading the keyboard and mouse information generated by the touch screen on the first source device based on the USB Host.

6. The method for implementing screen segmentation based on a screen segmenter according to claim 5, characterized in that: The method of acquiring the USB standard descriptor corresponding to the operating system on the screen channel based on the keyboard and mouse information includes: triggering identification of a system type of the operating system; A USB standard descriptor of the first source device under the operating system is extracted based on the system type.

7. The method for implementing screen segmentation based on a screen segmenter according to claim 6, characterized in that: The converting the keyboard and mouse device type and the keyboard and mouse information into a key event based on the corresponding USB standard descriptor includes: When the system type is identified as a Windows / MACOS system, converting the keyboard and mouse information generated by the first source device into a key event under the absolute position mouse; or When the system type is identified as an iOS system or an Andorid / HarmonyOS system or a non-mainstream operating system, the keyboard and mouse information generated by the first source device is converted into a key event under a relative position mouse or a touch screen device.

8. The method for implementing screen segmentation based on a screen segmenter according to claim 7, characterized in that: When the system type is identified as an iOS system or an Andorid / HarmonyOS system or a non-mainstream operating system, converting the keyboard and mouse information generated by the first source device into a key event under a relative position mouse or a touch screen device includes: When the system type is identified as an Andorid / HarmonyOS system, converting the left button pressing event of the absolute position mouse into a touch screen pressing event; After converting into a touch screen press event, the mouse fills the channel coordinate value in the keyboard and mouse information into the touch screen absolute position based on the absolute position, and reports the channel coordinate value and touch press event to the operating system on the screen channel based on the touch screen device using the third interface descriptor.

9. The method for implementing screen segmentation based on a screen segmenter according to claim 8, characterized in that: The step of converting the left button pressing event of the absolute position mouse into a touch screen pressing event comprises: After detecting that the absolute position mouse triggers the left button press event, the button mark position of the touch screen device is converted to the pressed state.

10. A system for implementing screen segmentation based on a screen segmenter, characterized in that: The system is based on a screen splitter KVM to access multiple different source devices, wherein the multiple different source devices include more than one source device of the touch screen drive implementation principle, and the system includes: DP display signal receiving module: used for the KVM to receive the DP display signal input by the source device of the one or more touch screen driving implementation principles based on the Type-C input terminal; Signal conversion module: used to convert the DP display signal into a corresponding HDMI signal; Signal input module: used to input multiple HDMI signals into the HDMI screen splitting chip; Screen splitting module: used to form a split-screen display screen based on the HDMI screen splitting chip.

Images