A Remote Virtual Control Method and System for LED Digital Photo Frames Based on Digital Twin
By generating and verifying the page status package and control fingerprint of the LED digital photo frame using digital twin technology, the problem of inconsistency between the remote control interface and the actual status is solved, achieving higher control accuracy and consistency.
Patent Information
- Application Number
- CN202610922494.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-25
- Publication Date
- 2026-11-17
AI Technical Summary
During the remote virtual control of LED digital photo frames, the virtual control interface generated by the remote control terminal is inconsistent with the actual display state, resulting in control recognition errors, device display coordinate positioning errors, and control object offset. Furthermore, the lack of a real-time verification mechanism for the page state affects the accuracy and consistency of control.
Using digital twin technology, the LED digital photo frame outputs page status packets, and the twin control terminal generates a virtual control interface and mapping table. It also generates execution instruction packets through the page status packets, the fingerprint of the current page control, and the fingerprint of the predicted target status. After the LED digital photo frame executes the instruction packets, it performs consistency verification. When a mismatch occurs, the twin control terminal requests page status packets again to update the mapping table.
It improves the accuracy and consistency of remote virtual control, reduces erroneous execution caused by page changes, and ensures the effectiveness and reliability of control commands.
Smart Images

Figure CN123123713A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent display control technology, and in particular to a remote virtual control method and system for LED digital photo frames based on digital twins. Background Technology
[0002] With the development of smart home display terminals and cloud content distribution technologies, LED digital photo frames are no longer limited to local static image display. They are gradually acquiring functions such as multi-page display, image carousel, interface switching, landscape / portrait screen switching, and remote content control. In practical applications, LED digital photo frames typically feature album display pages, function menu pages, parameter setting pages, or other interactive pages. Users can switch pages and operate controls locally on the device, or remotely control the LED digital photo frame via mobile phones, tablets, or other remote terminals. To make remote control more intuitive, the remote control terminal often needs to generate a corresponding visual operation interface based on the currently displayed page of the LED digital photo frame. This allows users to perform clicks, switching, or other operations based on the visual interface and further issue control commands to the LED digital photo frame.
[0003] However, in the specific application of remote virtual control of LED digital photo frames, the actual display page of the LED digital photo frame is not always constant. On the one hand, after the page of the LED digital photo frame is drawn, the current display content may continue to change due to automatic slideshows, page refreshes, or interface switching; on the other hand, when the LED digital photo frame changes its posture, it may trigger a landscape / portrait switching, causing changes in the page layout, control positions, and corresponding response areas. At this time, the virtual control interface, control correspondence, and coordinate mapping relationship previously generated by the remote control terminal may be inconsistent with the current actual display state of the LED digital photo frame. If remote control commands are still generated based on the original page relationships and mapping relationships, problems such as control recognition errors, incorrect device display coordinate positioning, and control object offset are likely to occur.
[0004] Especially after receiving user input at the remote control terminal, it typically needs to determine the target control based on the operation location in the virtual control interface and generate control commands accordingly to send to the LED digital photo frame. If the page information used by the remote control terminal belongs to a previous page, but the LED digital photo frame has actually switched to a new page, or although the page name is the same, the control arrangement order, response area coordinates, or page orientation have changed, then the target control determined by the remote control terminal and the device display coordinates may not correspond to the actual controls on the current page of the LED digital photo frame. This will cause the remote control commands to be received by the device, but their target is not the control that the user expects to operate, resulting in problems such as false triggering, false switching, or execution results inconsistent with the expected page state.
[0005] Furthermore, in remote virtual control scenarios, even if the page state is correct before the control command is issued, the LED digital photo frame may still experience changes in page state due to page refresh, screen orientation adjustment, or changes in displayed content before executing the control command. In this case, without a corresponding verification mechanism for the source page of the command, the current page state, and the page result after execution, the remote control terminal may struggle to promptly identify situations where "the current mapping relationship has become mismatched." The original mapping table may continue to participate in the generation of subsequent commands, causing subsequent remote virtual control to remain based on mismatched page relationships, further reducing the accuracy and consistency of remote control. Summary of the Invention
[0006] Therefore, it is necessary to propose a remote virtual control method and system for LED digital photo frames based on digital twins to address the above problems.
[0007] A remote virtual control method for LED digital photo frames based on digital twins includes the following steps: The LED digital photo frame outputs a page status packet after the page drawing is completed or the screen orientation is switched. The twin control terminal generates a virtual control interface, a mapping table, and a fingerprint of the current page controls based on the page status package; After receiving the operation type and virtual coordinates, the target control identifier and device display coordinates are determined according to the mapping table, and the target page identifier and predicted target state fingerprint are generated according to the preset control state transition rules. An execution instruction package containing the source page identifier, page drawing completion sequence number, current page control fingerprint, target control identifier, device display coordinates, target page identifier and predicted target state fingerprint is sent to the LED digital photo frame. The LED digital photo frame generates a local page control fingerprint based on the currently displayed content. When the current page identifier, page drawing completion number, and local page control fingerprint are consistent with the source page identifier, page drawing completion number, and current page control fingerprint in the execution instruction package, the frame executes the control action corresponding to the target control identifier and outputs the page identifier and the page control fingerprint after execution. When the execution page identifier or execution page control fingerprint is inconsistent with the target page identifier or the predicted target state fingerprint, the twin control terminal re-requests the page state packet and stops using the current mapping table.
[0008] In at least one embodiment of this application, the page state package includes a current page identifier, identifiers of each visible control, coordinates of the response area corresponding to each visible control, display order of each visible control, screen orientation identifier, and page drawing completion sequence number; Both the current page control fingerprint and the local page control fingerprint are generated by hash encoding after concatenating the current page identifier, the identifiers of each visible control, the coordinates of each response area, the display order of each visible control, and the screen orientation identifier according to the same preset field order.
[0009] In at least one embodiment of this application, the preset control state transition rule pre-stores the correspondence between the current page identifier, the target control identifier, and the operation type; For each set of correspondences, the target page identifier, the set of visible controls in the target page, the coordinates of the response area of each target visible control, the display order of each target visible control, and the target screen orientation identifier are also pre-stored. When the twin control terminal generates the predicted target state fingerprint according to the preset control state transition rules, it splices and encodes the target page identifier, the set of visible controls, the response area coordinates of each target visible control, the display order of each target visible control, and the target screen direction identifier according to the same field order and encoding rules as the page control fingerprint.
[0010] In at least one embodiment of this application, the twin control terminal rotates and scales the virtual control area in the virtual control interface according to the screen orientation indicator and the display resolution of the LED digital photo frame, and writes the coordinates of each virtual control area after transformation and each response area in the page status package into the mapping table. The target control identifier is the control identifier corresponding to the virtual control area containing the virtual coordinates.
[0011] In at least one embodiment of this application, when the executed page identifier or executed page control fingerprint is inconsistent with the target page identifier or the predicted target state fingerprint, the twin control terminal marks the corresponding execution instruction package as unexecutable and prohibits the continued use of the current mapping table until it receives the page state package obtained from the re-request.
[0012] In at least one embodiment of this application, the execution instruction package further includes an instruction sequence number; The LED digital photo frame executes the control action corresponding to the target control identifier only when the instruction sequence number is greater than the recorded maximum execution sequence number, and records the instruction sequence number as the new maximum execution sequence number after execution.
[0013] In at least one embodiment of this application, the execution instruction package further includes an instruction generation timestamp; The LED digital photo frame generates a local interactive occupancy state based on local touch press interruption and local touch release interruption, and records the latest local touch press time; The LED digital photo frame executes the control action corresponding to the target control identifier only when the local interaction occupancy state is idle and the latest local touch press time is earlier than the instruction generation timestamp.
[0014] In at least one embodiment of this application, the page state package further includes a current state version number, and the execution instruction package further includes a target state version number and an instruction validity window parameter; The LED digital photo frame executes the control action corresponding to the target control identifier only when the current state version number is consistent with the target state version number and the current judgment time is within the execution time window limited by the instruction valid time window parameter.
[0015] In at least one embodiment of this application, the LED digital photo frame also outputs a page status packet when the local touch is released and there is currently no page drawing task to be performed; Upon receiving the page status packet, the twin control terminal updates the virtual control interface, the current page control fingerprint, and the mapping table.
[0016] A remote virtual control system for LED digital photo frames based on digital twins, comprising: An LED digital photo frame, a twin control terminal, and a remote control terminal are provided. The LED digital photo frame is communicatively connected to the twin control terminal, and the remote control terminal is communicatively connected to the twin control terminal. The LED digital photo frame is used to output a page status packet after the page drawing is completed or the screen orientation is switched, and to generate a local page control fingerprint based on the currently displayed content. When the current page identifier, page drawing completion number and local page control fingerprint are consistent with the source page identifier, page drawing completion number and current page control fingerprint in the execution instruction packet, the control action corresponding to the target control identifier is executed, and the page identifier and page control fingerprint after execution are output. The twin control terminal is used to generate a virtual control interface, a mapping table and a current page control fingerprint according to the page state packet. After receiving the operation type and virtual coordinates, it determines the target control identifier and device display coordinates, and generates a target page identifier and a predicted target state fingerprint according to the preset control state transition rules, forming an execution instruction packet and sending it to the LED digital photo frame. The remote control terminal is used to output the operation type and virtual coordinates; The twin control terminal is also used to request a new page state packet and stop using the current mapping table when the post-execution page identifier or post-execution page control fingerprint is inconsistent with the target page identifier or the predicted target state fingerprint.
[0017] The aforementioned remote virtual control method and system for LED digital photo frames based on digital twins has at least the following beneficial effects: In this application, the LED digital photo frame outputs a page status packet after page drawing is completed or screen orientation is switched. The twin control terminal uses this packet to generate a virtual control interface, a mapping table, and the fingerprint of the current page control, thus establishing remote control based on the current actual page state. When generating the execution instruction packet, the twin control terminal writes the source page identifier, page drawing completion number, current page control fingerprint, target page identifier, and predicted target state fingerprint together, ensuring that the execution instruction includes not only target control information but also the corresponding page source and target page constraints. Before executing control actions, the LED digital photo frame compares the current page identifier, page drawing completion number, and local page control fingerprint with the corresponding content in the execution instruction packet, executing only when they match, thereby reducing erroneous executions caused by page changes or changes in control relationships. Finally, if the page identifier or page control fingerprint after execution is inconsistent with the target page identifier or predicted target state fingerprint, the twin control terminal re-requests the page status packet and stops using the current mapping table, thus preventing mismatched mapping relationships from continuing to participate in subsequent control. Based on the above technical means, this application can improve the accuracy, consistency, and reliability of remote virtual control of the LED digital photo frame. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] in: Figure 1 A schematic diagram of the structure of a remote virtual control system for an LED digital photo frame based on digital twin, provided in an embodiment of this application; Figure 2 A flowchart illustrating a remote virtual control method for an LED digital photo frame based on digital twins, provided in an embodiment of this application; Figure 3 A schematic diagram illustrating the generation of a page state package and page control fingerprints provided in this application embodiment; Figure 4 A schematic diagram illustrating the coordinate mapping relationship between a virtual control interface and a device display, provided in an embodiment of this application; Figure 5 This is a schematic diagram illustrating the generation of an execution instruction package and the consistency verification before execution, as provided in an embodiment of this application.
[0020] Explanation of main component symbols 101. LED digital photo frame; 102. Twin control terminal; 103. Remote control terminal; 111. Page status package; 121. Mapping table; 122. Execution instruction package; 131. Virtual control interface; 132. Virtual control area; 141. Device display page; 142. Response area. Detailed Implementation
[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0022] This application provides a remote virtual control method for an LED digital photo frame based on digital twins. The method is applied to a remote virtual control system consisting of an LED digital photo frame 101, a digital twin control terminal 102, and a remote control terminal 103. Figure 1 As shown, the LED digital photo frame 101 is used to display page content, perform page switching, receive local touch operations and remote control commands, and provide feedback on the execution results; the twin control terminal 102 is used to generate a corresponding virtual control interface 131, mapping table 121, current page control fingerprint, and predicted target state fingerprint based on the current actual display state of the LED digital photo frame 101; the remote control terminal 103 is used to receive the operation type and virtual coordinates input by the user and send them to the twin control terminal 102.
[0023] It should be noted that the application scenarios involved in this application do not refer to any remotely controlled display device in general, but rather to an LED digital photo frame 101 with page switching, image display, menu interaction, setting operation, and landscape / portrait screen switching functions. In a specific embodiment, the LED digital photo frame 101 can be set in a family living room, bedroom, office desk, or public display area to display family photos, electronic calendars, commemorative albums, menu pages, or device setting pages. Users can switch pictures, switch pages, or enter the setting page on the LED digital photo frame 101 through local touch operation, or they can remotely control the LED digital photo frame 101 from a different location through a remote control terminal 103 such as a mobile phone or tablet. For example, family members can use the remote control terminal 103 to perform page switching, album selection, image page turning, or menu setting operations on the LED digital photo frame 101 used by elderly family members; as another example, merchants can use the remote control terminal 103 to switch image content and adjust pages on the LED digital photo frame 101 used for display. In the above scenario, if the page state based on the remote control terminal 103 is inconsistent with the actual page displayed by the LED digital photo frame 101, problems such as target control recognition error, device display coordinate offset, and execution result inconsistency with the expected page may easily occur. Therefore, a remote virtual control solution that can perform consistency verification around the current page state is needed.
[0024] In one specific embodiment, for ease of understanding, this application can be described in conjunction with the following example scenario: Figure 4 As shown and Figure 5 As shown, the LED digital photo frame 101 currently displays a "Family Album Page," showing one photo and three visible controls at the bottom: "Previous," "Next," and "Back Menu." The remote control terminal 103 synchronously displays the corresponding virtual control interface 131. When the user clicks the "Next" area on the remote control terminal 103, the twin control terminal 102 first confirms whether the target control at the clicked location is indeed the "Next" control, then confirms whether the page on which the instruction is based is still the currently displayed "Family Album Page," and then confirms whether the page actually switches to the expected next image display page after the action is executed. If the page changes before or after execution, the original mapping relationship can no longer be used. The following specific steps are based on this scenario.
[0025] S10: The LED digital photo frame 101 outputs a page status packet 111 after the page drawing is completed or the screen orientation is switched.
[0026] It should be noted that "page drawing completed" means that the image content, menu content, control objects, and related display elements on the current page have been drawn and entered a stable display state; "screen orientation switching completed" means that the LED digital photo frame 101 has completed the orientation switch between landscape and portrait modes, and the page layout, control positions, and display content after the switch have been refreshed. By outputting page status packets 111 at the above two points in time, the twin control terminal 102 obtains stable state information corresponding to the current actual page, rather than intermediate state information during the page switching process, thus providing a consistent basis for the subsequent generation of the virtual control interface 131 and the mapping table 121.
[0027] Specifically, in a family photo album display scenario of this application, after the LED digital photo frame 101 finishes displaying the current photo page, or after the user rotates the digital photo frame causing the page to switch from landscape to portrait mode and complete the redraw, the LED digital photo frame 101 outputs a page status packet 111 so that the twin control terminal 102 can determine which visible controls are on the current page, where these controls are located, and which drawing state the current page belongs to. Combined with... Figure 2 and Figure 3 As can be seen, page state package 111 serves as the basic input for subsequent page state recognition and page control fingerprint generation.
[0028] Step S10 further includes the following steps: S11: Page state package 111 contains the current page identifier, the identifiers of each visible control, the coordinates of the response area corresponding to each visible control, the display order of each visible control, the screen orientation identifier, and the page drawing completion sequence number.
[0029] It should be noted that the current page identifier is used to characterize the identity of the page currently displayed by the LED digital photo frame 101, such as a family album page, album directory page, settings page, or main menu page; the visible control identifiers are used to characterize each interactive control object in the current page that is in a display state, such as "previous control," "next control," "return menu control," or "confirm control"; the response area coordinates corresponding to each visible control are used to characterize the triggerable area position of each control in the current page. In a specific embodiment, the upper left corner coordinates and lower right corner coordinates can be used to define the rectangular response area corresponding to a certain control; the display order of each visible control is used to characterize the arrangement order or drawing order of each control in the current page; the screen orientation identifier is used to indicate whether the current page is in landscape or portrait mode; the page drawing completion sequence number is used to characterize the drawing completion state corresponding to the current page. In a specific embodiment, the LED digital photo frame 101 updates the page drawing completion sequence number after each stable page drawing is completed, so that different page states can be distinguished by this sequence number. Figure 3 As shown, the page status packet 111 includes the aforementioned page field information and serves as input for subsequent field concatenation and hash encoding.
[0030] S12: When the local touch is released and there is no page task to be drawn, output page status packet 111 again.
[0031] It should be noted that, in addition to outputting page status packet 111 after page drawing is completed or screen orientation is switched, when a local user completes a touch operation that causes a change in page state, the LED digital photo frame 101 can also output page status packet 111 again when the local touch is released and there is no page drawing task at present. The purpose of this setting is that after the local operation is completed, the twin control terminal 102 can obtain the new page state basis in a timely manner, thereby avoiding the remote control terminal 103 from continuing to perform control recognition and coordinate mapping based on the old page relationship.
[0032] S20: The twin control terminal 102 generates a virtual control interface 131, a mapping table 121, and a current page control fingerprint based on the page status package 111.
[0033] It should be noted that the virtual control interface 131 refers to the visual operation interface displayed on the remote control terminal 103, corresponding to the current page of the LED digital photo frame 101. Users can perform operations such as clicking, long-pressing, swiping, dragging, or page switching on this interface. The mapping table 121 is used to establish the correspondence between virtual coordinates and device display coordinates. The current page control fingerprint is used to characterize the control-level state of the current page of the LED digital photo frame 101, so as to perform consistency verification before and after remote execution.
[0034] In the aforementioned family album page example, after receiving the page status packet 111, the twin control terminal 102 can display a virtual control interface 131 corresponding to the current family album page on the mobile phone. The interface also includes virtual control areas 132 such as "Previous," "Next," and "Back Menu," and generates corresponding mapping relationships and the fingerprint of the current page controls. Combined with... Figure 3 It can be seen that the fingerprint of the current page control is obtained by concatenating and hashing the fields in the page state package 111 in a unified field order; combined with Figure 4 It can be seen that the virtual control area 132 in the virtual control interface 131 and the response area 142 in the device display page 141 establish a corresponding relationship through the mapping table 121.
[0035] Step S20 further includes the following steps: S21: Concatenate the current page identifier, the identifiers of each visible control, the coordinates of each response area, the display order of each visible control, and the screen orientation identifier according to the same preset field order, and then perform hash encoding to generate the fingerprint of the current page control.
[0036] It should be noted that the current page control fingerprint is not generated arbitrarily, but is generated based on key fields in the page state package 111. Specifically, the twin control terminal 102 concatenates the current page identifier, the identifiers of each visible control, the coordinates of each response area, the display order of each visible control, and the screen orientation identifier in a unified field order, and then hashes the concatenation result to generate the current page control fingerprint. By adopting a fixed field order and fixed encoding rules, different page control fingerprints can be generated when the page state changes, while the same page control fingerprint is generated when the page state remains consistent. Figure 3 As shown, the fingerprint of the current page control is generated after the entire input of page status package 111 is processed by "field concatenation + hash encoding".
[0037] S22: Based on the screen orientation indicator and the display resolution of the LED digital photo frame 101, the virtual control area 132 in the virtual control interface 131 is rotated and scaled, and the coordinates of each virtual control area 132 after the transformation are written into the mapping table 121 in a one-to-one correspondence with the coordinates of each response area in the page status package 111.
[0038] It should be noted that, since the LED digital photo frame 101 may be in landscape or portrait mode, and the position and size of the controls may also differ under different display resolutions, the twin control terminal 102 needs to rotate and scale the virtual control area 132 according to the screen orientation and display resolution when generating the mapping table 121. Then, it needs to establish a one-to-one correspondence between the adapted virtual control area 132 and the actual response area coordinates in the page status package 111 and write it into the mapping table 121. Figure 4 It can be seen that the virtual control area 132 in the virtual control interface 131 establishes a correspondence with the response area 142 in the device display page 141 through the mapping table 121.
[0039] S23: After receiving the new page status packet 111 output according to step S12, update the virtual control interface 131, the current page control fingerprint and mapping table 121.
[0040] It should be noted that when the LED digital photo frame 101 outputs a new page status packet 111 due to local operation, the twin control terminal 102 should regenerate the corresponding virtual control interface 131 based on the new page status packet 111, and regenerate the current page control fingerprint and mapping table 121 to ensure that subsequent remote control is always based on the latest page status.
[0041] S30: After receiving the operation type and virtual coordinates, determine the target control identifier and device display coordinates according to the mapping table 121, and generate the target page identifier and predicted target state fingerprint according to the preset control state transition rules, forming an execution instruction package 122 and sending it to the LED digital photo frame 101.
[0042] It should be noted that the operation type can be click, long press, swipe, drag, or page switching, etc.; the virtual coordinates refer to the coordinate position corresponding to the user's action on the remote control terminal 103 when triggering the operation in the virtual control interface 131. After receiving the operation type and virtual coordinates, the twin control terminal 102 first needs to determine, based on the mapping table 121, which specific control the operation applies to, and the display position of that control on the LED digital photo frame 101 device; then, it also needs to determine, theoretically, which target page should be entered after the operation is executed in the current page state, and what control-level state the target page should present. Figure 4 It can be seen that the first half of this step mainly revolves around the virtual control interface 131, the virtual control area 132, the mapping table 121, the device display page 141, and the response area 142; combined with Figure 5 It can be seen that the latter part of this step mainly revolves around the generation of execution instruction package 122.
[0043] Step S30 further includes the following steps: S31: Determine the virtual control area 132 into which the virtual coordinates fall, and determine the control identifier corresponding to the virtual control area 132 and written into the mapping table 121 as the target control identifier, and at the same time determine the corresponding device display coordinates.
[0044] It should be noted that the target control identifier is not directly named based on the virtual coordinates themselves, but is determined through a chain of "virtual coordinates - virtual control area 132 - corresponding control identifier in mapping table 121". In other words, the twin control terminal 102 first needs to determine which virtual control area 132 the virtual coordinates fall into, then determine the control identifier corresponding to that virtual control area 132 as the target control identifier, and determine the corresponding device-side coordinates as the device display coordinates. This process can be combined with... Figure 4 understand.
[0045] S32: Determine the target page identifier based on the preset control state transition rules.
[0046] It should be noted that the preset control state transition rule is used to characterize: in a current page, after performing a certain operation type on a certain target control, what target page state the page will transition to. To this end, the twin control terminal 102 pre-stores the correspondence between "current page identifier - target control identifier - operation type" and determines the theoretically expected target page identifier after performing the target control action based on this correspondence.
[0047] For example, in one specific embodiment, when the current page is the "Family Album Page", the target control is the "Next Image Control", and the operation type is "Click", the twin control terminal 102 can determine that the target page identifier is still "Family Album Page" according to the preset control state transition rules, but the displayed content should be switched to the next image; as another example, when the current page is the "Main Menu Page", the target control is the "Settings Control", and the operation type is "Click", the twin control terminal 102 can determine that the target page identifier is "Settings Page".
[0048] S33: Generate a predicted target state fingerprint based on the set of visible controls in the target page, the coordinates of the response area of each target visible control, the display order of each target visible control, and the target screen orientation identifier, which are pre-stored in the preset control state transition rules.
[0049] It should be noted that after determining the target page identifier, the twin control terminal 102 also needs to further determine the set of controls that should be presented on the target page and their spatial distribution relationship. To this end, the preset control state transition rules also pre-store the set of visible controls on the target page, the response area coordinates of each target visible control, the display order of each target visible control, and the target screen orientation identifier. The twin control terminal 102 concatenates and encodes the above content according to the same field order and encoding rules as the current page control fingerprint, thereby generating a predicted target state fingerprint.
[0050] S34: Write the source page identifier, page drawing completion sequence number, current page control fingerprint, target control identifier, device display coordinates, target page identifier, and predicted target state fingerprint into the execution instruction package 122.
[0051] It should be noted that the execution instruction package 122 not only contains the action information of "which control to click," but also includes "the original page state on which the action is based" and "the page state expected to be achieved after execution." By writing the source page identifier, page drawing completion number, and current page control fingerprint into the execution instruction package 122, the pre-execution page basis corresponding to the instruction can be defined; by writing the target page identifier and predicted target state fingerprint into the execution instruction package 122, the post-execution page result required by the instruction can be defined. This process can be combined with... Figure 5 understand, Figure 5 The generation of the execution instruction package 122 and its role in pre-execution verification are shown.
[0052] S35: In a specific embodiment, the execution instruction package 122 further includes an instruction sequence number.
[0053] It should be noted that the instruction sequence number is used to distinguish the order of execution instruction packets 122, so that the LED digital photo frame 101 can determine whether the currently arriving execution instruction packet 122 is an updated and valid instruction.
[0054] S36: In a specific embodiment, the execution instruction package 122 further includes an instruction generation timestamp.
[0055] It should be noted that the instruction generation timestamp is used to characterize the generation time of the execution instruction package 122, so as to facilitate subsequent comparison with the local interaction time of the LED digital photo frame 101, thereby determining whether a new local operation has occurred after the remote instruction was generated.
[0056] S37: In a specific embodiment, the page state package 111 further includes the current state version number, and the execution instruction package 122 further includes the target state version number and the instruction validity window parameter.
[0057] It should be noted that the current state version number is used to identify the version to which the current page state of the LED digital photo frame 101 belongs, the target state version number is used to identify the page state version that the execution instruction package 122 requires to match, and the instruction validity time window parameter is used to limit the time range within which the execution instruction package 122 is allowed to execute. By introducing the above information into the page state package 111 and the execution instruction package 122, a foundation can be provided for subsequent version verification and time validity verification.
[0058] S40: The LED digital photo frame 101 generates a local page control fingerprint based on the current display content, and executes the control action corresponding to the target control identifier when the current page identifier, page drawing completion number and local page control fingerprint are consistent with the source page identifier, page drawing completion number and current page control fingerprint in the execution instruction package 122, and outputs the page identifier and page control fingerprint after execution.
[0059] It should be noted that after receiving the execution instruction packet 122, the LED digital photo frame 101 does not immediately execute the target control action. Instead, it first generates a local page control fingerprint based on the currently displayed content. Specifically, the LED digital photo frame 101 concatenates and hashes the page identifier, the identifiers of each currently visible control, the coordinates of the response area corresponding to each currently visible control, the display order of each currently visible control, and the current screen orientation identifier of the currently displayed page according to the same field order and encoding rules as the current page control fingerprint, thereby generating the local page control fingerprint.
[0060] Next, the LED digital photo frame 101 compares the current page identifier with the source page identifier in the execution instruction package 122, the current page's page drawing completion number with the page drawing completion number in the execution instruction package 122, and the local page control fingerprint with the current page control fingerprint in the execution instruction package 122. The target control action is executed only if all three items match. This method eliminates the possibility that the page has been switched, redrawn, or the control composition relationship has changed before the action is executed. This process can be combined with... Figure 5 understand, Figure 5 The document also illustrates the overall logic for pre-execution consistency checks.
[0061] Step S40 further includes the following steps: S41: Execute the control action corresponding to the target control identifier only when the instruction sequence number is greater than the recorded maximum executed sequence number, and record the instruction sequence number as the new maximum executed sequence number after execution.
[0062] It should be noted that by comparing the instruction sequence number in the currently executed instruction package 122 with the maximum executed sequence number recorded locally, older instructions that were generated earlier but arrived late can be prevented from being applied to the current page again, thus ensuring the correctness of the remote operation sequence.
[0063] S42: Generate a local interaction occupancy state based on local touch press interruption and local touch release interruption, and record the latest local touch press time; execute the control action corresponding to the target control identifier only when the local interaction occupancy state is idle and the latest local touch press time is earlier than the timestamp generated by the instruction.
[0064] It should be noted that by comparing the local touch occurrence time and the command generation timestamp, it can be determined whether a new local operation has occurred after the remote command was generated. If a new local operation has occurred, it means that the current page state may have been changed by local interaction, and in this case, it is not advisable to continue executing the old remote command.
[0065] S43: The control action corresponding to the target control identifier is executed only when the current state version number is the same as the target state version number and the current judgment time is within the execution time window limited by the instruction valid time window parameter.
[0066] It should be noted that version number matching is used to further determine whether the current page state still corresponds to the execution instruction package 122; the instruction validity window is used to determine whether the execution instruction package 122 is still within the allowed execution time range. Through version verification and time window verification, the risk of outdated or mismatched instructions being executed can be further reduced.
[0067] S50: When the execution page identifier or execution page control fingerprint is inconsistent with the target page identifier or the predicted target state fingerprint, the twin control terminal 102 re-requests the page state packet 111 and stops using the current mapping table 121.
[0068] It should be noted that after receiving the executed page identifier and executed page control fingerprint from the LED digital photo frame 101, the twin control terminal 102 compares the executed page identifier with the target page identifier and compares the executed page control fingerprint with the predicted target state fingerprint. If both match, it means the execution result is consistent with the expected page state, and the current mapping table 121 can still serve as the basis for subsequent remote control; if either does not match, it means the execution result is inconsistent with the expected target page state of the twin control terminal 102, and the current mapping table 121 has mismatched with the actual page state of the LED digital photo frame 101. This process can be combined with the overall workflow. Figure 2 understand.
[0069] Step S50 further includes the following steps: S51: Mark the corresponding execution instruction packet 122 as not to be executed again, and prohibit the continued use of the current mapping table 121 until the page status packet 111 obtained from the re-request is received.
[0070] It should be noted that "marking the corresponding execution instruction package 122 as unexecutable" means setting a prevent-repeated trigger flag for the execution instruction package 122, so that even if the same execution instruction package 122 is received again later, it will no longer be regarded as a valid instruction that can be reused. "Prohibiting the continued use of the current mapping table 121" means that before receiving the page status package 111 obtained from the re-request, the twin control terminal 102 will no longer perform new virtual coordinate parsing and target control identification based on the current mapping table 121, so as to avoid mismatched mapping relationships from continuing to participate in subsequent control.
[0071] S60: In a specific embodiment, the LED digital photo frame 101 also outputs a page status packet 111 when the local touch release is completed and there is no page drawing task at present; after receiving the page status packet 111, the twin control terminal 102 updates the virtual control interface 131, the current page control fingerprint and the mapping table 121.
[0072] It should be noted that this step corresponds to the supplementary update process described in steps S12 and S23 above. When a local user completes a local touch operation and the page has returned to a stable display state, the LED digital photo frame 101 outputs the page status packet 111 again. The twin control terminal 102 updates the virtual control interface 131, the current page control fingerprint, and the mapping table 121 accordingly, so that subsequent remote virtual control is always based on the latest page status.
[0073] In another specific embodiment, this application also provides a remote virtual control system for LED digital photo frames based on digital twins. For example... Figure 1 As shown, the system includes an LED digital photo frame 101, a twin control terminal 102, and a remote control terminal 103. The LED digital photo frame 101 is communicatively connected to the twin control terminal 102, and the remote control terminal 103 is communicatively connected to the twin control terminal 102.
[0074] The LED digital photo frame 101 is used to output a page status package 111 after the page drawing is completed or the screen orientation is switched, and to generate a local page control fingerprint according to the currently displayed content. When the current page identifier, page drawing completion number and local page control fingerprint are consistent with the source page identifier, page drawing completion number and current page control fingerprint in the execution instruction package 122, the control action corresponding to the target control identifier is executed, and the page identifier and page control fingerprint after execution are output.
[0075] The twin control terminal 102 is used to generate a virtual control interface 131, a mapping table 121, and a current page control fingerprint based on the page state packet 111. After receiving the operation type and virtual coordinates, it determines the target control identifier and device display coordinates, and generates a target page identifier and a predicted target state fingerprint according to a preset control state transition rule, forming an execution instruction packet 122 and sending it to the LED digital photo frame 101. The twin control terminal 102 is also used to re-request the page state packet 111 and stop using the current mapping table 121 when the page identifier or the page control fingerprint after execution is inconsistent with the target page identifier or the predicted target state fingerprint.
[0076] The remote control terminal 103 is used to output the operation type and virtual coordinates.
[0077] Based on the above technical solution, this application establishes a pre-execution verification and post-execution verification mechanism for remote virtual control by establishing a correspondence between the page state package 111, the current page control fingerprint, the target page identifier, the predicted target state fingerprint, the local page control fingerprint, and the page identifier and the page control fingerprint after execution. This ensures that remote virtual control is based on the current actual page state of the LED digital photo frame 101, and promptly re-acquires the page state basis and stops using the mismatch mapping table 121 when page state mismatch occurs, thereby improving the accuracy, consistency, and reliability of remote virtual control of the LED digital photo frame 101.
[0078] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0079] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0080] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A remote virtual control method for an LED digital photo frame based on digital twins, characterized in that, Includes the following steps: The LED digital photo frame outputs a page status packet after the page drawing is completed or the screen orientation is switched. The twin control terminal generates a virtual control interface, a mapping table, and a fingerprint of the current page controls based on the page status package; After receiving the operation type and virtual coordinates, the target control identifier and device display coordinates are determined according to the mapping table, and the target page identifier and predicted target state fingerprint are generated according to the preset control state transition rules. An execution instruction package containing the source page identifier, page drawing completion sequence number, current page control fingerprint, target control identifier, device display coordinates, target page identifier and predicted target state fingerprint is sent to the LED digital photo frame. The LED digital photo frame generates a local page control fingerprint based on the currently displayed content. When the current page identifier, page drawing completion number, and local page control fingerprint are consistent with the source page identifier, page drawing completion number, and current page control fingerprint in the execution instruction package, the frame executes the control action corresponding to the target control identifier and outputs the page identifier and the page control fingerprint after execution. When the execution page identifier or execution page control fingerprint is inconsistent with the target page identifier or the predicted target state fingerprint, the twin control terminal re-requests the page state packet and stops using the current mapping table.
2. The remote virtual control method for LED digital photo frames based on digital twins according to claim 1, characterized in that, The page status package includes the current page identifier, the identifiers of each visible control, the coordinates of the response area corresponding to each visible control, the display order of each visible control, the screen orientation identifier, and the page drawing completion sequence number; Both the current page control fingerprint and the local page control fingerprint are generated by hash encoding after concatenating the current page identifier, the identifiers of each visible control, the coordinates of each response area, the display order of each visible control, and the screen orientation identifier according to the same preset field order.
3. The remote virtual control method for LED digital photo frames based on digital twins according to claim 2, characterized in that, The preset control state transition rules pre-store the correspondence between the current page identifier, the target control identifier, and the operation type; For each set of correspondences, the target page identifier, the set of visible controls in the target page, the coordinates of the response area of each target visible control, the display order of each target visible control, and the target screen orientation identifier are also pre-stored. When the twin control terminal generates the predicted target state fingerprint according to the preset control state transition rules, it splices and encodes the target page identifier, the set of visible controls, the response area coordinates of each target visible control, the display order of each target visible control, and the target screen direction identifier according to the same field order and encoding rules as the page control fingerprint.
4. The remote virtual control method for LED digital photo frames based on digital twins according to claim 1, characterized in that, The twin control terminal rotates and scales the virtual control area in the virtual control interface according to the screen orientation indicator and the display resolution of the LED digital photo frame, and writes the coordinates of each virtual control area after transformation and each response area in the page status package into the mapping table. The target control identifier is the control identifier corresponding to the virtual control area containing the virtual coordinates.
5. The remote virtual control method for LED digital photo frames based on digital twins according to claim 4, characterized in that, When the executed page identifier or executed page control fingerprint is inconsistent with the target page identifier or the predicted target state fingerprint, the twin control terminal marks the corresponding execution instruction packet as unexecutable and prohibits the continued use of the current mapping table until it receives the page state packet obtained from the re-request.
6. The remote virtual control method for LED digital photo frames based on digital twins according to claim 1, characterized in that, The execution instruction package also includes an instruction sequence number; The LED digital photo frame executes the control action corresponding to the target control identifier only when the instruction sequence number is greater than the recorded maximum execution sequence number, and records the instruction sequence number as the new maximum execution sequence number after execution.
7. The remote virtual control method for LED digital photo frames based on digital twins according to claim 6, characterized in that, The execution instruction package also includes an instruction generation timestamp; The LED digital photo frame generates a local interactive occupancy state based on local touch press interruption and local touch release interruption, and records the latest local touch press time; The LED digital photo frame executes the control action corresponding to the target control identifier only when the local interaction occupancy state is idle and the latest local touch press time is earlier than the instruction generation timestamp.
8. The remote virtual control method for LED digital photo frames based on digital twins according to claim 6, characterized in that, The page status package also includes the current status version number, and the execution instruction package also includes the target status version number and instruction validity window parameters; The LED digital photo frame executes the control action corresponding to the target control identifier only when the current state version number is consistent with the target state version number and the current judgment time is within the execution time window limited by the instruction valid time window parameter.
9. The remote virtual control method for LED digital photo frames based on digital twins according to claim 1, characterized in that, The LED digital photo frame also outputs a page status packet when the local touch release is completed and there is no page drawing task at present. Upon receiving the page status packet, the twin control terminal updates the virtual control interface, the current page control fingerprint, and the mapping table.
10. A remote virtual control system for LED digital photo frames based on digital twins, characterized in that, include: An LED digital photo frame, a twin control terminal, and a remote control terminal are provided. The LED digital photo frame is communicatively connected to the twin control terminal, and the remote control terminal is communicatively connected to the twin control terminal. The LED digital photo frame is used to output a page status packet after the page drawing is completed or the screen orientation is switched, and to generate a local page control fingerprint based on the currently displayed content. When the current page identifier, page drawing completion number and local page control fingerprint are consistent with the source page identifier, page drawing completion number and current page control fingerprint in the execution instruction packet, the control action corresponding to the target control identifier is executed, and the page identifier and page control fingerprint after execution are output. The twin control terminal is used to generate a virtual control interface, a mapping table and a current page control fingerprint according to the page state packet. After receiving the operation type and virtual coordinates, it determines the target control identifier and device display coordinates, and generates a target page identifier and a predicted target state fingerprint according to the preset control state transition rules, forming an execution instruction packet and sending it to the LED digital photo frame. The remote control terminal is used to output the operation type and virtual coordinates; The twin control terminal is also used to request a new page state packet and stop using the current mapping table when the post-execution page identifier or post-execution page control fingerprint is inconsistent with the target page identifier or the predicted target state fingerprint.
