Display control method and first electronic device

Abstract

The present disclosure provides a display control method and a first electronic device, comprising: in response to a screen projection state update, determining a target display configuration parameter corresponding to the current update, and generating a corresponding parameter version identifier; enabling an output type parameter in the target display configuration parameter to generate a video frame carrying the parameter version identifier, and sending the video frame to a second electronic device; receiving a confirmation signal fed back by the second electronic device, wherein the confirmation signal is used to represent that the video frame carrying the parameter version identifier has been displayed on the display interface of the second electronic device; in response to the confirmation signal matching the parameter version identifier, enabling an interaction type parameter in the target display configuration parameter as a current effective mapping parameter, and mapping an input event coordinate from the second electronic device based on the current effective mapping parameter.

Description

Technical Field

[0001] This disclosure relates to the field of display control technology, and in particular to a display control method and a first electronic device. Background Technology

[0002] In screen mirroring scenarios, when the display configuration is updated, the host may have difficulty confirming the actual display status at the remote end, resulting in a missynchronization between the video output and input coordinate mapping, which affects the accuracy of interaction. Summary of the Invention

[0003] This disclosure provides a display control method and a first electronic device.

[0004] According to one aspect of this disclosure, a display control method is provided, applied to a first electronic device, wherein the first electronic device establishes a screen projection connection with a second electronic device and outputs a video frame display screen to the second electronic device. The method includes: in response to a screen projection status update, determining the target display configuration parameters corresponding to the update and generating a corresponding parameter version identifier; enabling output-type parameters in the target display configuration parameters to generate a video frame carrying the parameter version identifier and sending the video frame to the second electronic device; receiving a confirmation signal from the second electronic device, wherein the confirmation signal is used to indicate that the video frame carrying the parameter version identifier has been displayed on the display interface of the second electronic device; in response to the confirmation signal matching the parameter version identifier, enabling interactive-type parameters in the target display configuration parameters as currently effective mapping parameters, and mapping the coordinates of input events from the second electronic device based on the currently effective mapping parameters.

[0005] According to embodiments of this disclosure, generating the corresponding parameter version identifier includes: generating an incrementing version identifier based on the current display configuration parameter version identifier corresponding to the screen casting session of the screen casting connection; associating and storing the incrementing version identifier with the output-type parameters and interaction-type parameters in the display configuration parameters; wherein the output-type parameters include one or all of the following: display width, display height, rotation direction, and display coordinate position, and the interaction-type parameters include the input coordinate transformation relationship.

[0006] According to embodiments of this disclosure, the method further includes: during the transition period from sending a video frame carrying a parameter version identifier to receiving an acknowledgment signal, outputting a video frame to a second electronic device based on the output class parameter in the target display configuration parameters, and setting the interaction class parameter in the target display configuration parameters as an interaction parameter to be activated.

[0007] According to embodiments of this disclosure, the method further includes: during the transition period from sending a video frame carrying a parameter version identifier to receiving an acknowledgment signal, performing at least one of the following processing on the input event from the second electronic device: pausing processing, buffering processing, discarding processing, or conversion processing; and after the acknowledgment signal matches the parameter version identifier, mapping the coordinates of the input event from the second electronic device based on the interactive parameters in the target display configuration parameters.

[0008] According to embodiments of this disclosure, the screen projection status update occurs during multiple consecutive input sequences where there are input events from a second electronic device; the method further includes: during the transition period from sending a video frame carrying a parameter version identifier to receiving an acknowledgment signal, processing the input points of the consecutive input sequence based on the interactive parameters corresponding to the previous display configuration parameter version to obtain first trajectory data and outputting it to the second electronic device along with the video frame; after the acknowledgment signal matches the parameter version identifier, processing the subsequent input points of the consecutive input sequence based on the interactive parameters in the target display configuration parameters to obtain second trajectory data and outputting it to the second electronic device along with the video frame.

[0009] According to embodiments of this disclosure, the method further includes: in response to a confirmation signal matching a parameter version identifier, acquiring accumulated first trajectory data during the transition period; converting the accumulated first trajectory data into third trajectory data adapted to the target display configuration parameters based on the transformation relationship between the interactive parameters corresponding to the previous display configuration parameter version and the interactive parameters in the target display configuration parameter version; and merging and outputting the third trajectory data and the second trajectory data in chronological order.

[0010] According to embodiments of this disclosure, the method further includes: if no acknowledgment signal is received within a first time after sending a video frame carrying a parameter version identifier, or if the output class parameter in the target display configuration parameters cannot be used for video frame generation, then the continuous input sequence is terminated and the locally stored first trajectory data is cleared.

[0011] According to embodiments of this disclosure, the screen casting status update includes restoring the connection within a second time period after the screen casting connection is disconnected; wherein, within the second time period after the screen casting connection is disconnected, the currently effective display parameters are retained, and the extended display instance, display topology, and display position corresponding to the second electronic device remain unchanged.

[0012] According to embodiments of this disclosure, the confirmation signal includes a session identifier corresponding to the screen projection connection and a parameter version identifier; in response to the confirmation signal matching the parameter version identifier, the method includes: if the parameter version identifier parsed by the second electronic device is consistent with the parameter version identifier generated by the first electronic device, and the session identifier in the confirmation signal matches the screen projection connection, then the confirmation signal is determined to be valid.

[0013] Another aspect of this disclosure provides a first electronic device that establishes a screen projection connection with a second electronic device and outputs a video frame display to the second electronic device. The first electronic device includes a processor and a wireless communication module. The wireless communication module is used to interact with the second electronic device. The processor is used to: determine the target display configuration parameters corresponding to the current update in response to a screen projection status update, and generate a corresponding parameter version identifier; enable the output-type parameters in the target display configuration parameters to generate a video frame carrying the parameter version identifier, and send the video frame to the second electronic device; receive a confirmation signal from the second electronic device, wherein the confirmation signal is used to indicate that the video frame carrying the parameter version identifier has been displayed on the display interface of the second electronic device; and, in response to the confirmation signal matching the parameter version identifier, enable the interaction-type parameters in the target display configuration parameters as the currently effective mapping parameters, and map the input event coordinates from the second electronic device based on the currently effective mapping parameters.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0016] Figure 1 This is a flowchart of a display control method according to an embodiment of the present disclosure;

[0017] Figure 2 This is a schematic diagram of a display control method according to an embodiment of the present disclosure;

[0018] Figure 3 This is a flowchart of a display control method according to another embodiment of the present disclosure;

[0019] Figure 4 This is a flowchart of a display control method according to yet another embodiment of the present disclosure;

[0020] Figure 5 This is a schematic diagram of the structure of a first electronic device according to an embodiment of the present disclosure; and

[0021] Figure 6 This is a schematic block diagram of an example first electronic device used to implement embodiments of the present disclosure. Detailed Implementation

[0022] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0023] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of data (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals.

[0024] Figure 1 This is a flowchart of a display control method according to an embodiment of the present disclosure.

[0025] like Figure 1 As shown, the display control method of this embodiment is applied to a first electronic device, the first electronic device establishes a screen projection connection with a second electronic device, and outputs a video frame display screen to the second electronic device. The method includes operations S110-S140.

[0026] When operating S110, in response to the screen projection status update, the target display configuration parameters corresponding to this update are determined, and the corresponding parameter version identifier is generated.

[0027] In the embodiments of this disclosure, the first electronic device refers to a device that provides screen content and performs display control, such as a personal computer (PC) or laptop computer. The second electronic device refers to a device that receives the screen as an extended display, such as a tablet or mobile phone. Screen mirroring connection refers to a communication connection established between the first electronic device and the second electronic device for transmitting video images and realizing screen extension.

[0028] In the embodiments of this disclosure, a screen mirroring status update refers to a session state change event that triggers the re-determination of display configuration parameters during a screen mirroring connection session between a first electronic device and a second electronic device. This state change may include session-level events such as adjustments to display attributes or changes in connection status during the screen mirroring connection process. Any session state change that may cause a mismatch between the locally effective display configuration and the actual display state at the remote end can constitute a screen mirroring status update in this application.

[0029] In the embodiments of this disclosure, the target display configuration parameters refer to a set of display configurations that need to take effect after the current screen projection status update, including output class parameters for generating video frames and interaction class parameters for mapping remote input events to the local coordinate system.

[0030] In the embodiments of this disclosure, the parameter version identifier refers to a version number that uniquely corresponds to a set of target display configuration parameters, and is used to establish a version association between video frames and confirmation signals.

[0031] In the embodiments of this disclosure, in response to a screen projection status update, the target display configuration parameters corresponding to this update are determined, and a corresponding parameter version identifier is generated. When the first electronic device detects a screen projection status update, it determines the target display configuration parameters corresponding to this update and generates a parameter version identifier corresponding to that set of parameters. This parameter version identifier is used for the association and confirmation of subsequent video frames.

[0032] For example, the PC detects a tablet rotation event, determines the new display width, height, and orientation, and generates a new version number "config2".

[0033] In operation S120, the output class parameter in the target display configuration parameter is enabled to generate a video frame carrying a parameter version identifier, and the video frame is sent to the second electronic device.

[0034] In embodiments of this disclosure, output parameters refer to the parameters used to generate video frames in the target display configuration parameters. For example, a display width of 1080, a height of 1920, and a rotation direction of portrait mode.

[0035] In the embodiments of this disclosure, a video frame refers to a frame containing image data, which carries a parameter version identifier. For example, a PC captures the image of a virtual display, generates a video frame according to the new width and height, and carries the version number "config2" in the frame header or metadata.

[0036] In embodiments of this disclosure, an output class parameter in the target display configuration parameters is enabled to generate a video frame carrying a parameter version identifier, and the video frame is sent to a second electronic device. A first electronic device enables the output class parameter in the target display configuration parameters, generates a video frame, carries the parameter version identifier along with the video frame, and then sends it to the second electronic device. The second electronic device can identify the configuration version corresponding to the video frame.

[0037] For example, the PC generates a frame using the new resolution, labels it "config2", and sends it to the tablet via a wireless network.

[0038] In operation S130, an acknowledgment signal is received from the second electronic device, wherein the acknowledgment signal is used to indicate that the video frame carrying the parameter version identifier has been displayed on the display interface of the second electronic device.

[0039] In the embodiments of this disclosure, the confirmation signal refers to the signal sent by the second electronic device to the first electronic device after the video frame carrying the parameter version identifier is actually displayed on the screen. For example, after the tablet decodes and displays the "config2" frame, it sends a "DisplayReady" message to the PC, which includes a session identifier and a parameter version identifier.

[0040] In embodiments of this disclosure, a confirmation signal is received from a second electronic device. The first electronic device receives the confirmation signal from the second electronic device, which indicates that the second electronic device has displayed the video frame carrying the version identifier of the parameter on its display interface. For example, after the tablet displays the "config2" frame, it replies to the PC: "config2 frame has been displayed," and the PC receives the confirmation signal.

[0041] In operation S140, in response to the confirmation signal matching the parameter version identifier, the interactive class parameter in the target display configuration parameter is enabled as the currently effective mapping parameter, and the input event coordinates from the second electronic device are mapped based on the currently effective mapping parameter.

[0042] In embodiments of this disclosure, the confirmation signal includes a parameter version identifier. Interactive parameters refer to parameters in the target display configuration parameters used for input coordinate mapping. For example, an input coordinate transformation matrix used to convert the tablet's touch coordinates to PC screen coordinates.

[0043] In the embodiments of this disclosure, the currently active mapping parameter refers to the parameter currently used to map the input event coordinates after a match is confirmed, i.e., the enabled interaction class parameter. Input event coordinates refer to the coordinates of an input event from a second electronic device. For example, the coordinates of a touch point on a tablet.

[0044] In embodiments of this disclosure, in response to a confirmation signal matching a parameter version identifier, an interactive parameter in the target display configuration parameters is enabled as the currently effective mapping parameter, and the input event coordinates from the second electronic device are mapped based on this currently effective mapping parameter. When the first electronic device determines that the parameter version identifier in the confirmation signal is consistent with (i.e., matched) a locally generated parameter version identifier, it enables an interactive parameter in the target display configuration parameters as the currently effective mapping parameter, and the input event coordinates from the second electronic device are mapped based on this parameter. The switching of input coordinate mapping is strictly aligned with the moment when the new screen is actually displayed at the remote end, avoiding inconsistencies between the screen and the input.

[0045] For example, if the PC receives a confirmation signal from the tablet and the version number "config2" matches, it will activate the new coordinate transformation matrix, and all subsequent touch coordinates sent by the tablet will be transformed according to the new matrix.

[0046] Figure 2This is a schematic diagram of a display control method according to an embodiment of the present disclosure.

[0047] like Figure 2 As shown, an exemplary process for handling display configuration interaction states is provided. During the user input phase, the user initiates a display configuration change operation to the system. This operation instruction is transmitted to the display configuration module of the PC.

[0048] Furthermore, after receiving the change operation, the display configuration module detects the current rotation or display parameter changes and generates an updated display configuration (new configuration). After generating the new configuration, the display configuration module sends the corresponding new configuration screen data to the video link.

[0049] Furthermore, after receiving the new configuration screen data, the video link needs to perform encoding, transmission, decoding, and compositing. Since these processing steps are time-consuming, there are processing gaps in the video link. During these gaps, the remote display has not yet received the new configuration screen data, so the remote display continues to show the old configuration screen.

[0050] Furthermore, once the video link completes processing and successfully outputs the new configuration image to the remote display, the remote display then renders and displays the first frame of the new configuration. Subsequently, the remote display sends a display ready signal back to the display configuration module, completing the interaction for this display configuration change.

[0051] It should be noted that the above embodiments are only used to illustrate the basic interaction process of the solution in this embodiment.

[0052] Through the embodiments of this disclosure, output parameters and interactive parameters are associated through parameter version identifiers. Output parameters are immediately used to generate video frames, while interactive parameters take effect only after remote confirmation and display. This achieves decoupling and synchronization of video output and input mapping, ensuring that the display configuration, video image, and input coordinates are consistent in version. It avoids problems such as click offset and image mismatch caused by asynchronous switching, effectively improving the accuracy and continuity of screen projection interaction.

[0053] Figure 3 This is a flowchart of a display control method according to another embodiment of the present disclosure.

[0054] like Figure 3 As shown, the corresponding parameter version identifiers are generated, including operations S310-S320.

[0055] When operating S310, an incremental version identifier is generated based on the current display configuration parameter version identifier corresponding to the screen casting session of the screen casting connection.

[0056] In the embodiments of this disclosure, the projection session refers to the logical communication session maintained between the first electronic device and the second electronic device for this extended screen display. During the session, video frame delivery and input event feedback are both performed through this session channel.

[0057] For example, once a PC and a tablet establish a wireless extension connection, the logical context of that connection is a screen mirroring session until the user actively disconnects it.

[0058] In the embodiments of this disclosure, the current display configuration parameter version identifier refers to the version number corresponding to the display configuration parameter that is currently in effect in the screen casting session.

[0059] For example, after the last rotation was completed and remote confirmation was received, the system was using version number configN.

[0060] In the embodiments of this disclosure, the incremental version identifier refers to the version number generated in a predetermined order of the current display configuration parameter version identifiers, used to mark the new configuration after this screen projection status update.

[0061] For example, adding 1 to configN results in configN+1. This incrementing characteristic ensures that the version number is globally monotonic, and later versions will not be overwritten by earlier confirmation signals.

[0062] In the embodiments of this disclosure, an incrementing version identifier is generated based on the current display configuration parameter version identifier corresponding to the screen casting session. After detecting an update in the screen casting status, the first electronic device does not reset the version count, but instead reads the currently active version identifier of the screen casting session and generates a new version identifier according to a predetermined rule (such as +1).

[0063] For example, the screen mirroring session has been running stably in config3 (landscape 1920×1080). When the user physically rotates the tablet to portrait mode, the system reads config3 and generates config4.

[0064] When operating S320, the incremental version identifier is associated with and stored in the output class parameters and interaction class parameters in the display configuration parameters.

[0065] In embodiments of this disclosure, output parameters include one or more of the following: display width, display height, rotation direction, and display coordinate position; interactive parameters include input coordinate transformation relationships.

[0066] In the embodiments of this disclosure, associated storage refers to establishing a mapping relationship between the same version identifier and its corresponding set of output class parameters and set of interaction class parameters, and writing it as a record into the local memory of the first electronic device. Subsequently, as long as the version identifier is obtained, all related parameters under that version can be read at once.

[0067] In the embodiments of this disclosure, output parameters refer to the parameters in the target display configuration parameters that directly determine the content of the video frame. Output parameters may include one or more of the following: display width, display height, rotation direction, and display coordinate position. For example, the rotated pixel width is 1080, the height is 1920, the orientation is vertical, and the virtual desktop is arranged on the right side of the main screen.

[0068] In the embodiments of this disclosure, interaction parameters refer to the parameters in the target display configuration parameters that determine how the input coordinates are transformed. Interaction parameters may include input coordinate transformation relationships, such as an affine transformation matrix containing rotation angle, scaling ratio, and translation amount, used to convert the tablet touch point coordinates into logical coordinates on the PC virtual extended screen.

[0069] In the embodiments of this disclosure, the input coordinate transformation relationship refers to the calculation rule that maps the input position in the screen coordinate system of the second electronic device to the virtual extended screen coordinate system of the first electronic device.

[0070] In the embodiments of this disclosure, the display width and display height in the output parameters jointly define the pixel canvas size of the remote screen; the rotation direction defines the canvas orientation (landscape / portrait / reverse); and the display coordinate position defines the arrangement of the virtual screen in the main desktop topology (e.g., extending to the right of the main screen). The input coordinate transformation relationship of the interaction parameters is calculated in real time based on the above geometric parameters. When the screen rotates, the touch coordinate system must transform accordingly; otherwise, the click will fall on the corresponding position of the old screen.

[0071] In the embodiments of this disclosure, the incrementing version identifier is associated and stored with the output class parameters and interaction class parameters in the display configuration parameters. After generating a new version identifier, the system splits the target display configuration parameters determined in this update into two categories: output class parameters for "outputting the screen" and interaction class parameters for "receiving touch". "config4-{output class parameter set, interaction class parameter set}" is stored as a binding record in the version configuration table. The video encoding module and input mapping module subsequently retrieve their required new parameters by looking up the table in config4, rather than maintaining independent version numbers themselves.

[0072] For example, the system stores the record: config4 corresponds to {width 1080, height 1920, rotation direction 90°, desktop offset (X2,Y2)} and {transformation matrix M4}. Subsequently, the PC acquisition module outputs frames with a width and height of 1080×1920 and adds config4 to the frame metadata; the input module temporarily does not enable M4 and continues to use M3 until a confirmation signal arrives.

[0073] Through the embodiments of this disclosure, the output class and interaction class parameters are bound and stored by incrementing the version identifier, so that the new screen generation and the new coordinate mapping share the same version index. The overall switching is only performed after the first frame is confirmed at the remote end, which can avoid click offset or trajectory jump caused by the split between the screen and input coordinate versions, and effectively improve the accuracy and reliability of the display control method of this embodiment.

[0074] In some embodiments of this disclosure, the method further includes: during the transition period from sending a video frame carrying a parameter version identifier to receiving an acknowledgment signal, outputting a video frame to a second electronic device based on the output class parameter in the target display configuration parameters, and setting the interaction class parameter in the target display configuration parameters as an interaction parameter to be activated.

[0075] In the embodiments of this disclosure, the transition period refers to the time interval from when the first electronic device sends a video frame carrying a parameter version identifier to the second electronic device until the first electronic device receives a confirmation signal from the second electronic device. During this interval, the first electronic device cannot be certain whether the second electronic device has decoded and actually lit up the screen, and the old version of the image may still remain at the remote end.

[0076] In the embodiments of this disclosure, the output video frame refers to a video data unit that is collected and encoded from the output parameters (such as new display width, new display height, new rotation direction, etc.) in the target display configuration parameters determined after the current screen projection status update. The frame carries the parameter version identifier for this time and is continuously pushed to the second electronic device.

[0077] In the embodiments of this disclosure, the interaction parameter to be effective refers to the state of the interaction-type parameters (such as input coordinate transformation relationship, transformation matrix) in the target display configuration parameters during the transition period. This parameter has been bound and stored with the new version identifier, but has not yet been selected as the mapping parameter that actually participates in the input event coordinate conversion.

[0078] In the embodiments of this disclosure, during the transition period from sending a video frame carrying a parameter version identifier to receiving an acknowledgment signal, a video frame is output to a second electronic device based on the output class parameters in the target display configuration parameters, and the interaction class parameters in the target display configuration parameters are set to interactive parameters to be activated. The output class parameters determine the geometric shape of the screen canvas; continuous output means that each subsequent video frame is generated and sent according to the new parameters during the transition period, rather than sending only one new frame and then reverting.

[0079] For example, after the PC generates and sends the first vertical screen image carrying config4, within a few hundred milliseconds before receiving the tablet's DisplayReady, the subsequent dozens of frames are encoded and sent at a vertical screen resolution of 1080×1920 and the new desktop offset position. After the tablet decodes, what it sees is a continuous vertical screen animation without any horizontal screen fragments in between.

[0080] In the embodiments of this disclosure, after entering the transition period, the first electronic device locks the output parameters in the current target display configuration parameters, and the video acquisition and transmission link no longer reverts to the old parameters before the update. The screen refresh of the remote display device is aligned with the local video stream from the first frame, avoiding the visual effect of a half-finished switch where the new configuration command has arrived but the old screen is still playing. For example, during the process of rotating from landscape to portrait mode, the user no longer sees "a brief blackout followed by a sudden portrait switch," but rather a continuous and smooth transition of the new screen.

[0081] Through the embodiments of this disclosure, during the transition period, new images are continuously sent according to new output parameters, and new coordinate mapping is only put on hold and not enabled. This can unify what the remote end sees with the local video stream's previous version, avoiding the situation where the remote end still receives old version video frames after the screen projection status is updated, effectively improving the reliability of the display control method of this embodiment.

[0082] In some embodiments of this disclosure, the method further includes: during the transition period from sending a video frame carrying a parameter version identifier to receiving an acknowledgment signal, performing at least one of the following processing on the input event from the second electronic device: pausing processing, buffering processing, discarding processing, or conversion processing; and after the acknowledgment signal matches the parameter version identifier, mapping the coordinates of the input event from the second electronic device based on the interactive parameters in the target display configuration parameters.

[0083] In the embodiments of this disclosure, the transition period refers to the time interval from when the first electronic device sends a video frame carrying the current parameter version identifier to the second electronic device, until the first electronic device receives a confirmation signal indicating that the version of the video frame has been displayed on the display interface of the second electronic device. During this interval, the remote screen may still retain the old image before the update, and the local device cannot yet confirm that the new image is actually visible.

[0084] In the embodiments of this disclosure, input events refer to event data generated by interactive operations from a second electronic device, including but not limited to touch press, finger / stylus movement, release, mouse movement, or click.

[0085] In the embodiments of this disclosure, the pause processing refers to suspending the input event arriving at the first electronic device side, maintaining the integrity of the event itself (not generating a lift-off command, not sending a cancellation command), but not sending it to the current coordinate mapping link or application side until the signal is confirmed to match before allowing it to pass uniformly.

[0086] In the embodiments of this disclosure, caching refers to temporarily storing the input events arriving during the transition period in a local buffer according to their arrival time, retaining their original screen coordinates, time stamps, and input identity identifiers, and continuing the calculation together with subsequent events after confirmation, without loss or forgery.

[0087] In the embodiments of this disclosure, the discarding process refers to directly discarding some or all of the arriving input events during the transition period, without entering the mapping process or temporarily storing them. This is suitable for events with extremely high density but short-term overlap, or repeated or out-of-order events caused by network jitter, thus avoiding the injection of dirty data.

[0088] In the embodiments of this disclosure, the conversion process refers to first converting the input events that arrive during the transition period according to the interaction parameters (old coordinate transformation relationship) corresponding to the previous version, or converting them into relative displacement relative to a certain anchor point for temporary storage; after the new version takes effect, the anchor point is used as the reference to continue to the new coordinate system to avoid coordinate jumps.

[0089] In embodiments of this disclosure, during the transition period from sending a video frame carrying a parameter version identifier to receiving an acknowledgment signal, at least one of the following processes is performed on input events from the second electronic device: pause processing, buffering processing, discarding processing, or conversion processing. Within the time window during which the local device displays the new parameters but the remote device has not yet displayed the new image, the return input is controlled, and no coordinate mapping output based on the new interaction parameters is generated during the transition period.

[0090] In the embodiments of this disclosure, after confirming that the signal matches the parameter version identifier, the input event coordinates from the second electronic device are mapped based on the interactive parameters in the target display configuration parameters. For example, after the tablet decodes the first frame of config4 and displays it on the screen, it sends back DisplayReady(config4). Once the PC verifies the data, it immediately changes the new input coordinate transformation matrix M4 corresponding to config4 from "pending activation" to "currently activated mapping parameter".

[0091] Through the embodiments of this disclosure, during the transition period, the returned input events are paused, cached, discarded, or anchor point converted as needed, cutting off the cross-use of old and new coordinate mappings. The new interaction parameters are only applied uniformly after the first frame is confirmed at the remote end, avoiding click offset and accidental touch during rotation or resolution switching, and effectively ensuring the continuity of interaction.

[0092] Figure 4 This is a flowchart of a display control method according to yet another embodiment of the present disclosure.

[0093] like Figure 4 As shown, the screen projection state update occurs during multiple consecutive input sequences where there are input events from the second electronic device; the method also includes operations S410-S420.

[0094] During operation S410, after sending a video frame carrying a parameter version identifier, during the transition period before receiving an acknowledgment signal, the input points of the continuous input sequence are processed based on the interactive class parameters corresponding to the previous display configuration parameter version, to obtain the first trajectory data, which is then output to the second electronic device along with the video frame.

[0095] In embodiments of this disclosure, a continuous input sequence refers to a continuous interaction process that has been established and has not yet terminated from a second electronic device, such as a Down→Move→Move…→Up event sequence during the period when a stylus or finger is pressed and not lifted, the sequence carrying the same input identity and pressed state.

[0096] In embodiments of this disclosure, an input point refers to the location data (such as the coordinates of a single move) of a single event that constitutes a continuous input sequence.

[0097] In the embodiments of this disclosure, the first trajectory data refers to a set of coordinate points with temporal relationships obtained by converting the interaction parameters of the previous version point by point during the transition period, representing the output handwriting / gesture path of the continuous input sequence in the old coordinate system.

[0098] In the embodiments of this disclosure, during the transition period from sending a video frame carrying a parameter version identifier to receiving an acknowledgment signal, the input points of the continuous input sequence are processed based on the interactive class parameters corresponding to the previous display configuration parameter version to obtain the first trajectory data, which is then output to the second electronic device along with the video frame.

[0099] In the embodiments of this disclosure, from the time the video frame carrying the new parameter version identifier is sent until the confirmation signal indicating that the version of the screen has been displayed on the remote display interface is received, the system maintains the interactive parameters of the previous version as the currently effective mapping parameters. During this period, input events from the second electronic device continue to arrive. These events are composed of multiple input points arranged in sampling order (i.e., multiple consecutive input sequences, referring to multiple subsets of input points organized chronologically within the same unterminated interactive event). The system applies the input coordinate transformation relationship bound to the previous version point by point, converting each input point into coordinate values ​​in the virtual extended screen coordinate system, forming first trajectory data, and sends this trajectory data to the second electronic device along with the version video frame. This processing ensures that the input trajectory returned from the remote end during the transition period shares the same coordinate semantics as the currently visible screen.

[0100] During operation of S420, after confirming that the signal matches the parameter version identifier, the subsequent input points of the continuous input sequence are processed based on the interactive parameters in the target display configuration parameters to obtain the second trajectory data, which is then output to the second electronic device along with the video frame.

[0101] In embodiments of this disclosure, a subsequent input point refers to a new input point that continues to be reached after the confirmation signal matches but before the continuous input sequence ends due to the user actually lifting the phone.

[0102] In the embodiments of this disclosure, the second trajectory data refers to the set of coordinate points that follow the first trajectory data, calculated according to the interactive parameters in the current target display configuration parameters after the signal matching is confirmed.

[0103] In the embodiments of this disclosure, after confirming that the signal matches the parameter version identifier, the subsequent input points of the continuous input sequence are processed based on the interactive parameters in the target display configuration parameters to obtain second trajectory data, which is then output to the second electronic device along with the video frame. When the parameter version identifier carried by the confirmation signal is consistent with the local pending version identifier, and the confirmation signal belongs to the current screen projection session, the system determines that the second electronic device has actually displayed the first frame corresponding to the current target display configuration, and then sets the interactive parameters in the target display configuration parameters from "pending" to the currently effective mapping parameter. Subsequently, subsequent input points belonging to the same unterminated input event are converted point by point according to the input coordinate transformation relationship of the new version to generate second trajectory data, which continues to be output with subsequent video frames. The first trajectory data and the second trajectory data are connected end-to-end on the time axis, and they share the same input identity identifier, and are still recognized as a complete and uninterrupted interaction process on the application side.

[0104] For example, the first electronic device is a personal computer, and the second electronic device is a tablet terminal. A screen mirroring connection is established between the two, with the tablet acting as a virtual extended screen for the personal computer. The operator uses a stylus to perform a continuous drawing on the tablet's display interface without lifting the stylus. This drawing process generates multiple input points carrying the same stylus identifier and continuously reported at a fixed sampling period. During the drawing process, the system detects a display configuration change (switching from landscape to portrait mode), determines the target display configuration parameters corresponding to this update, and generates a parameter version identifier configN+1. Subsequently, video frames are generated and sent according to the new display width, height, and rotation direction.

[0105] The transition period begins from the moment the first video frame carrying configN+1 is sent until the tablet decodes and submits the frame to the display interface, providing a feedback DisplayReady(configN+1). During this transition period, the tablet continuously reports input points generated by the pen tip's movement. The personal computer still calls the input coordinate transformation matrix corresponding to configN to calculate these input points one by one, obtaining the first trajectory data corresponding to the landscape geometry. This first trajectory data is then sent to the tablet for rendering synchronously along with the configN+1 video frame.

[0106] After the transition period ends and the parameter version identifier and session identifier in the personal computer verification confirmation signal match, the new input coordinate transformation matrix corresponding to configN+1 is activated. Thereafter, input points generated by continued pen tip movement are converted according to the new matrix to form second trajectory data, which continues to be sent with each frame until the operator actually lifts the pen and sends a termination event. The application receives a complete trajectory with the same pen identifier, no inserted cancellation events, and continuous start and end points; the trajectory curve has no inflections or jumps during the screen orientation switch.

[0107] Through the embodiments of this disclosure, the old mapping is used to output the first segment of the trajectory during the transition period. After the first frame is confirmed at the far end, the new mapping is used to continue the second segment of the trajectory. This allows the point sequence and identity to remain continuous when switching coordinate spaces without interruption, avoiding handwriting jumps and trajectory breaks at the moment of switching.

[0108] In some embodiments of this disclosure, the method further includes: in response to a confirmation signal matching a parameter version identifier, acquiring accumulated first trajectory data during the transition period; converting the accumulated first trajectory data into third trajectory data adapted to the target display configuration parameters based on the transformation relationship between the interactive parameters corresponding to the previous display configuration parameter version and the interactive parameters in the target display configuration parameter version; and merging and outputting the third trajectory data and the second trajectory data in chronological order.

[0109] In the embodiments of this disclosure, the confirmation signal matching the parameter version identifier means that the first electronic device parses the received confirmation signal. When the parameter version identifier carried in the signal is consistent with the target version identifier to be switched locally, and the signal belongs to the current screen casting session, it is determined that the second electronic device has actually submitted the video frame of its corresponding version to the display interface and triggers the subsequent trajectory processing process.

[0110] In embodiments of this disclosure, in response to a confirmation signal matching a parameter version identifier, the accumulated first trajectory data during the transition period is acquired. After the confirmation signal verification passes, the first electronic device reads all the first trajectory data corresponding to the unterminated input event from the transition period buffer.

[0111] In the embodiments of this disclosure, based on the transformation relationship between the interaction parameters corresponding to the previous display configuration parameter version and the interaction parameters in the target display configuration parameter version, the accumulated first trajectory data is converted into third trajectory data adapted to the target display configuration parameters. The transformation relationship between the previous and target interaction parameters is invoked, and the same mapping calculation is performed on each coordinate point of the first trajectory data, outputting third trajectory data where all points are under the new screen geometry.

[0112] In the embodiments of this disclosure, the third trajectory data and the second trajectory data are merged and output in chronological order. The third trajectory data (before the switch, converted to the new space) and the second trajectory data (after the switch, native to the new space) are arranged according to the order in which each point arrives at the first electronic device, forming a complete coordinate sequence that extends from the beginning of the transition period to the end of the switch. This sequence is continuously reported using the original input identity and sent with subsequent video frames.

[0113] Through the embodiments of this disclosure, the first frame of the remote display is used as the switching boundary between the old and new coordinate mappings. The old spatial trajectory during the transition period is converted into the new space as a whole and then spliced ​​with the subsequent trajectory according to the sampling time sequence. This ensures that the same unterminated input maintains the continuity of coordinate semantics and point sequence at the moment of display configuration switching, avoids trajectory jumps and human interruption, and improves the continuity of the display control method of this embodiment.

[0114] In some embodiments of this disclosure, the method further includes: if no acknowledgment signal is received within a first time after sending a video frame carrying a parameter version identifier, or if the output class parameter in the target display configuration parameters cannot be used for video frame generation, then the continuous input sequence is terminated and the locally stored first trajectory data is cleared.

[0115] In the embodiments of this disclosure, if no acknowledgment signal is received within the first time period after sending a video frame carrying a parameter version identifier, a timer is started from the date the video frame carrying the parameter version identifier is first sent. If, after the first time period, no acknowledgment signal is received from the second electronic device that matches both the parameter version identifier and the session identifier, it is determined that the remote display ready acknowledgment has timed out.

[0116] For example, after the first electronic device sends the first video frame carrying configN+1 to the tablet, it starts a 500ms timer. Due to severe network link attenuation, the tablet is unable to decode the frame and fails to send back DisplayReady(configN+1) when the timer expires. At this point, the first electronic device determines that the confirmation signal has timed out.

[0117] In the embodiments of this disclosure, output parameters in the target display configuration parameters cannot be used for video frame generation. Output parameters include parameters that directly determine the content of the video frame, such as display width, display height, rotation direction, and display coordinate position. If any of the above parameters have illegal values, exceed the hardware support range, or conflict with each other, causing the video acquisition and encoding module to be unable to generate a valid video frame, then the output parameters are considered unusable.

[0118] For example, if the target display width determined after this screen mirroring status update is 0 or exceeds the maximum texture size supported by the graphics processor, the virtual display driver cannot allocate a frame buffer according to that size, and the video capture module returns an initialization failure. At this time, the first electronic device determines that the target output class parameters cannot be used for video frame generation.

[0119] In the embodiments of this disclosure, the continuous input sequence is terminated, and the locally stored first trajectory data is cleared. The first electronic device performs termination processing on the input event that is still in an unterminated state. Specifically, it injects an end-type event (such as pen lift-off or touch cancellation) corresponding to the identity of the input event into the application layer, so that the application layer recognizes the input event as ended and no longer expects the arrival of subsequent input points, thereby avoiding the application layer from entering abnormal states such as "key sticking" or "touch suspension" due to long-term waiting for the lift-off event.

[0120] In embodiments of this disclosure, the first electronic device releases the buffer storing the first trajectory data, discards all coordinate points that have not yet undergone the transformation relationship between old and new interaction parameters or have not yet been sent with video frames, and marks the trajectory status of the input event as cleared. After the clearing is completed, even if a delayed confirmation signal is received or video output is restored, the system will not resume the trajectory continuation process of the input event, but will treat it as a completely new input event and reprocess it.

[0121] Through the embodiments of this disclosure, when the remote confirmation times out or the new configuration cannot be used for frame output, the continuous input that is suspended is terminated in a timely manner and the residual trajectory cache is cleared, so as to avoid the application layer from the occurrence of contact suspension, key sticking and the wrong injection of expired trajectory, and can ensure the determinism of input state machine and resource recyclability.

[0122] In some embodiments of this disclosure, the screen casting status update includes restoring the connection after the screen casting connection is disconnected within a second time period; wherein, during the second time period after the screen casting connection is disconnected, the currently effective display parameters are retained, and the extended display instance, display topology, and display position corresponding to the second electronic device remain unchanged.

[0123] In the embodiments of this disclosure, a screen mirroring connection disconnection refers to a state in which the screen mirroring connection loses connectivity at the link layer or session layer due to reasons such as wireless link jitter, signal obstruction, brief device hibernation, or network switching, but the first electronic device has not yet executed the virtual display device deregistration process.

[0124] In the embodiments of this disclosure, reconnection means that before the second timer has ended, the screen projection connection is re-established, and the session channel, video downlink, and input backhaul link can all carry data again.

[0125] In embodiments of this disclosure, an extended display instance refers to a virtual display device object (e.g., a virtual display instance enumerated by the operating system) created for this screen projection within the graphics subsystem of the first electronic device. This instance declares to the window manager that "an extended screen is available."

[0126] In the embodiments of this disclosure, the display topology refers to the logical organization of multiple displays maintained by the graphics subsystem, including the enumeration order of extended screens, their relative positions to the main screen (e.g., right-side extension, top extension), and plug-and-play identifiers. The display position refers to the origin and arrangement offset of the extended screen in the desktop virtual coordinate system. For example, when the main screen resolution is 1920×1080, the topology position of the extended screen is (X=1920, Y=0).

[0127] In the embodiments of this disclosure, the screen mirroring status update includes restoring the connection after a screen mirroring connection is disconnected within a second time period. The first electronic device continuously monitors the link status during the duration of the screen mirroring connection. When a screen mirroring connection interruption is detected, it does not immediately trigger the virtual display's deregistration (such as a display departure process at the operating system level), but instead records this status change as a grace period hold state and starts a second timer. If a link layer or session layer response is received from the second electronic device again within the second time period, it is determined that this status update indicates a connection restoration after a disconnection, and the process enters the grace period hold process.

[0128] In the embodiments of this disclosure, during the second time period after the screen mirroring connection is disconnected, the currently effective display parameters are retained, and the extended display instance, display topology, and display position corresponding to the second electronic device remain unchanged. After entering the grace period retention state, the first electronic device locks the display configuration to the version that was effective before the disconnection, and does not reset to the default resolution due to the link interruption, does not clear the current parameter version identifier, and does not trigger a full configuration renegotiation with the second electronic device. The video output module can choose to pause streaming or continuously send the last confirmed frame, but the geometric parameters (width, height, orientation, desktop offset) and input transformation matrix used remain at the values ​​before the disconnection.

[0129] In the embodiments of this disclosure, during the grace period, the first electronic device keeps the virtual display device instance in a "connected" registered state, does not release the frame buffer, and does not unregister the device object; the operating system window manager maintains the original multi-screen topology, application windows already located on the extended screen do not trigger cross-screen migration, the taskbar extended screen area does not disappear, and the display layout diagram in the display settings is not redrawn. If the connection is restored before the second timeout period ends, the above process continues; if the connection is not restored after the second timeout period expires, the normal stop process is executed, the extended display instance is unregistered, the topology occupancy is released, and session resources are cleared.

[0130] Through the embodiments of this disclosure, transient link interruptions and session terminations are handled separately. During the grace period, the currently effective display parameters are locked and the virtual display device instance and desktop topology are preserved. This avoids the operating system re-enumerating the display, which would cause window movement, application layout reset, and input state suspension, thus ensuring the continuity of use of the wireless extended screen under momentary disconnection.

[0131] In some embodiments of this disclosure, the confirmation signal includes a session identifier corresponding to the screen projection connection and a parameter version identifier; in response to the confirmation signal matching the parameter version identifier, the method includes: if the parameter version identifier parsed by the second electronic device is consistent with the parameter version identifier generated by the first electronic device, and the session identifier in the confirmation signal matches the screen projection connection, then the confirmation signal is determined to be valid.

[0132] In the embodiments of this disclosure, the session identifier refers to the logical session number (e.g., the sessionId generated when the session is established) corresponding to the current screen projection connection, used to uniquely identify the current virtual extended display session between the first electronic device and the second electronic device. The session identifiers are different when the same physical device is used at different times or when different devices are connected to the same host.

[0133] In the embodiments of this disclosure, the parameter version identifier refers to an incrementing version number (e.g., configN, configN+1) that is uniquely bound to a set of target display configuration parameters. It is sent with the video frame and embedded in the confirmation signal to identify which version of the screen corresponds to this confirmation.

[0134] In the embodiments of this disclosure, if the parameter version identifier parsed by the second electronic device is consistent with the parameter version identifier generated by the first electronic device, and the session identifier in the acknowledgment signal matches the screen projection connection, then the acknowledgment signal is determined to be valid. During the screen projection connection, the first electronic device continuously listens for the acknowledgment signal returned by the second electronic device. Upon receiving it, the first electronic device parses the signaling payload and separates two fields: the session identifier and the parameter version identifier parsed by the second electronic device.

[0135] In the embodiments of this disclosure, the parsed parameter version identifier is compared with the parameter version identifier to be effective generated locally by the first electronic device for this update. The version condition is determined to be true only if the two are exactly the same (e.g., both are config4); if the parsed value is config3, config5, or empty, or the checksum is abnormal, the version condition is not true.

[0136] In embodiments of this disclosure, the session identifier in the confirmation signal is compared with the session identifier registered by the screen mirroring connection currently maintained by the first electronic device. The session condition is deemed valid only if the two match. This verification can be used to exclude: retransmission packets from the old link after disconnection, echoes generated by another device mistakenly accessing the same port, and delayed historical confirmations after the grace period has expired and deregistration has been performed.

[0137] In the embodiments of this disclosure, the first electronic device outputs a valid confirmation signal only when both of the above conditions are met, and accordingly enables the interaction parameters in the target display configuration parameters as the currently effective mapping parameters to map the coordinates of subsequent input events. If either of the two conditions is not met, the confirmation signal is deemed invalid, the system continues to operate with the mapping parameters of the previous version, and does not perform new version conversion on cached input events.

[0138] Through the embodiments of this disclosure, the confirmation signal must pass both session affiliation and parameter version verification to be effective. This can filter out cross-session residual messages, late old version confirmations, and link packet crosstalk, avoid erroneous coordinate mapping switching, and ensure that the mapping is strictly bound to the current projection session and the current display version, effectively avoiding mismatch and dirty data injection.

[0139] Figure 5 This is a schematic diagram of the structure of a first electronic device according to an embodiment of the present disclosure.

[0140] In this embodiment, a first electronic device 500 establishes a screen projection connection with a second electronic device and outputs a video frame display to the second electronic device. The first electronic device 500 includes a processor 510 and a wireless communication module 520. The wireless communication module 520 is used to interact with the second electronic device. The processor 510 is used to: determine the target display configuration parameters corresponding to the current update in response to a screen projection status update, and generate a corresponding parameter version identifier; enable the output-type parameters in the target display configuration parameters to generate a video frame carrying the parameter version identifier, and send the video frame to the second electronic device; receive a confirmation signal from the second electronic device, wherein the confirmation signal is used to indicate that the video frame carrying the parameter version identifier has been displayed on the display interface of the second electronic device; and in response to the confirmation signal matching the parameter version identifier, enable the interaction-type parameters in the target display configuration parameters as the currently effective mapping parameters, and map the input event coordinates from the second electronic device based on the currently effective mapping parameters.

[0141] According to embodiments of this disclosure, this disclosure also provides a first electronic device, a readable storage medium, and a computer program product.

[0142] Figure 6 This is a schematic block diagram of an example first electronic device used to implement embodiments of the present disclosure.

[0143] like Figure 6 As shown, a first electronic device 600 according to an embodiment of this application includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage portion 608 into a random access memory 603. The processor 601 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0144] Random access memory 603 stores various programs and data required for the operation of the first electronic device 600. Processor 601, read-only memory 602, and random access memory 603 are interconnected via bus 604. Processor 601 executes various steps of the method flow according to embodiments of this application by executing programs stored in read-only memory 602 and / or random access memory 603. It should be noted that programs may also be stored in one or more memories other than read-only memory 602 and random access memory 603. Processor 601 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in one or more memories.

[0145] According to embodiments of this application, the first electronic device 600 may further include an input / output interface 605, which is also connected to a bus 604. The first electronic device 600 may also include one or more of the following components connected to the input / output interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card, such as a local area network card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0146] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0147] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 602 described above, and / or random access memory 603, and / or one or more memories other than read-only memory 602 and random access memory 603.

[0148] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0149] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0150] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by processor 601, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0151] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0153] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A display control method, applied to a first electronic device, wherein the first electronic device establishes a screen projection connection with a second electronic device and outputs video frame display images to the second electronic device, the method comprising: In response to the screen mirroring status update, determine the target display configuration parameters corresponding to this update, and generate the corresponding parameter version identifier; The output class parameter in the target display configuration parameters is enabled to generate a video frame carrying the parameter version identifier, and the video frame is sent to the second electronic device; The system receives a confirmation signal from the second electronic device, wherein the confirmation signal is used to indicate that the video frame carrying the parameter version identifier has been displayed on the display interface of the second electronic device. In response to the confirmation signal matching the parameter version identifier, the interactive parameters in the target display configuration parameters are enabled as the currently effective mapping parameters, and the input event coordinates from the second electronic device are mapped based on the currently effective mapping parameters.

2. The method according to claim 1, wherein generating the corresponding parameter version identifier includes: An incrementing version identifier is generated based on the current display configuration parameter version identifier corresponding to the screen projection session of the screen projection connection; The incrementing version identifier is associated with and stored in the output class parameters and interaction class parameters in the display configuration parameters; The output parameters include one or more of the following: display width, display height, rotation direction, and display coordinate position. The interactive parameters include the input coordinate transformation relationship.

3. The method according to claim 2, further comprising: During the transition period between sending the video frame carrying the parameter version identifier and receiving the confirmation signal, the video frame is output to the second electronic device based on the output class parameter in the target display configuration parameter, and the interaction class parameter in the target display configuration parameter is set as an interaction parameter to be activated.

4. The method according to claim 2, further comprising: During the transition period from sending the video frame carrying the parameter version identifier to receiving the confirmation signal, at least one of the following processes is performed on the input event from the second electronic device: pause processing, buffering processing, discarding processing, or conversion processing. After the confirmation signal matches the parameter version identifier, the input event coordinates from the second electronic device are mapped based on the interactive class parameters in the target display configuration parameters.

5. The method according to claim 2, wherein the screen projection state update occurs during a plurality of consecutive input sequences in which there are input events from the second electronic device; the method further comprises: During the transition period from sending the video frame carrying the parameter version identifier to receiving the confirmation signal, the input points of the continuous input sequence are processed based on the interactive parameters corresponding to the previous display configuration parameter version to obtain the first trajectory data, which is then output to the second electronic device along with the video frame. After the confirmation signal matches the parameter version identifier, the subsequent input points of the continuous input sequence are processed based on the interactive parameters in the target display configuration parameters to obtain the second trajectory data, which is then output to the second electronic device along with the video frame.

6. The method according to claim 5, further comprising: In response to the confirmation signal matching the parameter version identifier, the first trajectory data accumulated during the transition period is obtained; Based on the transformation relationship between the interaction parameters corresponding to the previous display configuration parameter version and the interaction parameters in the target display configuration parameter version, the accumulated first trajectory data is converted into third trajectory data adapted to the target display configuration parameters; The third trajectory data and the second trajectory data are merged and output in chronological order.

7. The method according to claim 5, further comprising: If the acknowledgment signal is not received within the first time after sending a video frame carrying the parameter version identifier, or if the output class parameter in the target display configuration parameter cannot be used for video frame generation, the continuous input sequence is terminated and the locally stored first trajectory data is cleared.

8. The method according to claim 1, wherein the screen mirroring status update includes restoring the connection within a second time period after the screen mirroring connection is disconnected; wherein, During the second time period after the screen mirroring connection is disconnected, the currently effective display parameters are retained, and the extended display instance, display topology, and display position corresponding to the second electronic device remain unchanged.

9. The method according to any one of claims 1 to 8, wherein the confirmation signal includes a session identifier corresponding to the screen mirroring connection and the parameter version identifier; the step of responding to the confirmation signal matching the parameter version identifier includes: If the parameter version identifier parsed by the second electronic device is consistent with the parameter version identifier generated by the first electronic device, and the session identifier in the confirmation signal matches the screen projection connection, then the confirmation signal is determined to be valid.

10. A first electronic device, wherein the first electronic device establishes a screen projection connection with a second electronic device and outputs video frame display images to the second electronic device, the first electronic device comprising: Processor and wireless communication module; wherein, The wireless communication module is used to interact with the second electronic device; The processor is used for: In response to the screen mirroring status update, determine the target display configuration parameters corresponding to this update, and generate the corresponding parameter version identifier; The output class parameter in the target display configuration parameters is enabled to generate a video frame carrying the parameter version identifier, and the video frame is sent to the second electronic device; The system receives a confirmation signal from the second electronic device, wherein the confirmation signal is used to indicate that the video frame carrying the parameter version identifier has been displayed on the display interface of the second electronic device. In response to the confirmation signal matching the parameter version identifier, the interactive parameters in the target display configuration parameters are enabled as the currently effective mapping parameters, and the input event coordinates from the second electronic device are mapped based on the currently effective mapping parameters.

Images