Mobile slide screen control method, device and storage medium for exhibition display

Abstract

The present disclosure provides a mobile sliding screen control method, device and storage medium for exhibition display, wherein the mobile sliding screen comprises at least one fixed main screen and at least one mobile sub-screen, the mobile sub-screen is located on the content presentation side of the fixed main screen and moves on the content presentation side according to a fixed track; the control method comprises generating the playing content corresponding to the mobile sub-screen and the fixed main screen respectively and the moving strategy of the mobile sub-screen according to the multimedia content to be presented and the environmental perception information obtained by the mobile sliding screen, and performing related multimedia playing and moving operations. Through the technical scheme provided by the present disclosure, the playing content and the moving strategy of the fixed main screen and the mobile sub-screen can be adjusted according to the relative position relationship of the mobile sub-screen relative to the fixed main screen, at the same time, the interaction experience between the mobile sliding screen and the visitors is enhanced through environmental perception, the visit experience of the user is greatly improved, and the present disclosure has promotional value.

Description

Technical Field

[0001] This disclosure relates to the field of multi-display screen collaboration, specifically, it discloses a mobile sliding screen control method, device, and storage medium for exhibition and display. Background Technology

[0002] With the rise of modern exhibition halls, traditional exhibition methods are no longer sufficient to meet the demands of visitors, leading to the emergence of diverse display approaches. One such approach is the mobile sliding screen. Specifically, a mobile sliding screen consists of at least one fixed main screen and at least one mobile secondary screen. The mobile secondary screen is located on the content presentation side of the fixed main screen and moves along a fixed track. The fixed main screen has a wide coverage area, used to present broad exhibition content, while the mobile secondary screen moves according to a preset rhythm, providing multi-angle auxiliary effects to the content displayed on the main screen during its movement.

[0003] However, traditional mobile sliding screens have relatively limited multimedia presentation effects. The content presented is often pre-configured by the operator and cannot be adjusted and controlled according to new exhibition scenarios. If the movement of the secondary screen is obstructed or affected, the overall presentation effect of the mobile sliding screen will be greatly reduced, and it will also be impossible to have any interaction with visitors. Summary of the Invention

[0004] This disclosure provides a method, system, device, and computer-readable storage medium for controlling a mobile sliding screen for exhibitions. Specifically, the first aspect of this disclosure provides a method for controlling a mobile sliding screen for exhibitions, wherein the mobile sliding screen includes at least one fixed main screen and at least one mobile secondary screen, the mobile secondary screen being located on the content presentation side of the fixed main screen and moving on the content presentation side according to a fixed track;

[0005] This method includes the following steps:

[0006] Acquire the multimedia content to be presented on the mobile swipe screen and the environmental awareness information of the mobile swipe screen's location;

[0007] Based on the multimedia content to be presented and the environmental awareness information, obtain the playback content corresponding to the mobile secondary screen and the fixed main screen, as well as the movement strategy of the mobile secondary screen.

[0008] Control the playback content on the fixed main screen and the mobile secondary screen, and control the movement of the mobile secondary screen according to the movement strategy.

[0009] In one possible implementation of the first aspect mentioned above, the display resolution of the mobile secondary screen is higher than that of the fixed primary screen;

[0010] The process of obtaining the playback content corresponding to the mobile secondary screen and the fixed primary screen, as well as the movement strategy of the mobile secondary screen, based on the multimedia content to be presented and the environmental awareness information, includes the following steps:

[0011] The main screen is fixed to play the multimedia content to be presented;

[0012] Continuously acquire the real-time relative position of the mobile secondary screen with respect to the fixed primary screen;

[0013] Based on the real-time relative position, the multimedia content projected onto the fixed main screen from the mobile secondary screen is synchronized to the mobile secondary screen for playback.

[0014] In one possible implementation of the first aspect above, the multimedia content to be presented includes multiple highlight content areas;

[0015] The process of obtaining the playback content corresponding to the mobile secondary screen and the fixed primary screen, as well as the movement strategy of the mobile secondary screen, based on the multimedia content to be presented and the environmental awareness information, includes the following steps:

[0016] Continuously acquire the real-time relative position of the mobile secondary screen with respect to the fixed primary screen;

[0017] Based on the real-time relative position, when there is an overlap between the projection of the mobile secondary screen onto the fixed main screen and the highlight content area, the playback content corresponding to the highlight content area is displayed on the mobile secondary screen, and the comparison table between the swipe highlight and the main screen background content is consulted.

[0018] surface

[0019]

[0020] When the screen is swiped horizontally, b1 = b2 = b3 = bn

[0021] Based on the real-time an value, the corresponding background content is played in the sliding screen and the corresponding background content is looked up in the table Gn. At the same time, the replacement background content is played in the main screen to realize the corresponding position of the highlight content area to play the replacement background content. O(0,0): fixed screen origin; O'(a,b): the coordinates of the sliding screen origin in the fixed screen coordinate system.

[0022] In one possible implementation of the first aspect above, when the playback content corresponding to the highlight content area is displayed on the mobile secondary screen, replacement background content is played at the corresponding position of the highlight content area on the fixed main screen.

[0023] In one possible implementation of the first aspect above, the multimedia content to be presented includes content displayed on the main screen and narration content on the secondary screen;

[0024] Fix the home screen display content to play on the home screen;

[0025] The mobile secondary screen plays corresponding narration content based on its relative position to the fixed main screen.

[0026] In one possible implementation of the first aspect above, the mobile sub-screen is equipped with an environmental sensing component, and the environmental sensing information includes the movement of visitors.

[0027] The process of obtaining the playback content corresponding to the mobile secondary screen and the fixed primary screen, as well as the movement strategy of the mobile secondary screen, based on the multimedia content to be presented and the environmental awareness information, includes the following steps:

[0028] The movement of visitors is obtained based on environmental sensing components;

[0029] Develop a movement strategy for the mobile secondary screen based on the movement of visitors, so that the mobile secondary screen moves with the flow of visitors; or

[0030] Develop a movement strategy for the mobile secondary screen based on the movement of visitors, so that the mobile secondary screen can guide the movement of visitors.

[0031] In one possible implementation of the first aspect above, the environmental perception information includes the visitor's voice information;

[0032] The process of obtaining the playback content corresponding to the mobile secondary screen and the fixed primary screen, as well as the movement strategy of the mobile secondary screen, based on the multimedia content to be presented and the environmental awareness information, includes the following steps:

[0033] Voice information is obtained from environmental perception sensing components;

[0034] The system responds to voice messages in real time based on a preset template.

[0035] The second aspect of this disclosure provides a mobile sliding screen control system for exhibitions, applied to the mobile sliding screen control method for exhibitions provided in the first aspect above; wherein the mobile sliding screen includes at least one fixed main screen and at least one mobile secondary screen, the mobile secondary screen is located on the content presentation side of the fixed main screen, and moves on the content presentation side according to a fixed track;

[0036] This type of mobile sliding screen control system for exhibition displays includes:

[0037] The sensing unit is used to acquire the multimedia content to be presented on the mobile swipe screen and the environmental sensing information of the location of the mobile swipe screen.

[0038] The strategy formulation unit is used to obtain the playback content corresponding to the mobile secondary screen and the fixed main screen, as well as the movement strategy of the mobile secondary screen, based on the multimedia content to be presented and the environmental awareness information.

[0039] The control execution unit is used to control the fixed main screen and the mobile secondary screen to play the corresponding playback content, and to control the mobile secondary screen to move according to the movement strategy.

[0040] A third aspect of this disclosure discloses an electronic device comprising:

[0041] Memory, used to store computer programs;

[0042] A processor for executing computer programs to implement the mobile sliding screen control method for exhibitions provided in the first aspect above.

[0043] The fourth aspect of this disclosure discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the mobile sliding screen control method for exhibition and display provided in the first aspect above.

[0044] Compared with the prior art, this disclosure has the following beneficial effects:

[0045] This disclosure provides a method, system, device, and computer-readable storage medium for controlling a mobile sliding screen in exhibitions. The technical solution proposed in this disclosure can adjust the playback content and movement strategy of both the fixed main screen and the mobile sub-screen based on their relative positions to the fixed main screen. Simultaneously, it enhances the interactive experience between the mobile sliding screen and visitors through environmental awareness, significantly improving the user's visitor experience and possessing widespread applicability. Attached Figure Description

[0046] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0047] Figure 1 According to an embodiment of this disclosure, a flowchart illustrating a mobile sliding screen control method for exhibition displays is shown;

[0048] Figure 2 According to an embodiment of this disclosure, a flowchart illustrating a process for obtaining playback content corresponding to a mobile secondary screen and a fixed primary screen, as well as a movement strategy for the mobile secondary screen, based on the multimedia content to be presented and environmental awareness information.

[0049] Figure 3 According to an embodiment of this disclosure, a flowchart illustrating another method for obtaining playback content corresponding to the mobile secondary screen and the fixed main screen, as well as a mobile strategy for the mobile secondary screen, based on the multimedia content to be presented and environmental awareness information.

[0050] Figure 4According to an embodiment of this disclosure, a flowchart illustrating another method for obtaining playback content corresponding to the mobile secondary screen and the fixed main screen, as well as a mobile strategy for the mobile secondary screen, based on the multimedia content to be presented and environmental awareness information.

[0051] Figure 5 According to an embodiment of this disclosure, a schematic diagram of a mobile sliding screen control method system for exhibition displays is shown.

[0052] Figure 6 According to an embodiment of this disclosure, a schematic diagram of the structure of an electronic device is shown.

[0053] Figure 7 According to an embodiment of this disclosure, a schematic diagram of the structure of a computer-readable storage medium is shown;

[0054] Figure 8 A schematic diagram illustrating the correspondence between image coordinates on a fixed screen and image coordinates on a moving screen.

[0055] Figure 9 Example schematic diagram showing the correspondence between O(0,0): the fixed screen origin and O'(a,b): the coordinates of the moving screen origin in the fixed screen coordinate system. Specific implementation methods

[0056] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed herein. This disclosure can also be implemented or applied to systems through other different specific embodiments, and various details in this disclosure can also be modified or changed according to different viewpoints and application systems without departing from the spirit of this disclosure. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this disclosure can be combined with each other.

[0057] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, so that those skilled in the art to which this disclosure pertains can readily implement it. This disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

[0058] In this disclosure, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic represented in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics represented may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples represented in this disclosure, as well as the features of those different embodiments or examples.

[0059] Furthermore, the terms "first" and "second" are used for illustrative purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the representation of this disclosure, "a plurality of" means two or more, unless otherwise expressly and specifically defined.

[0060] For the purpose of clarity, devices unrelated to the description are omitted, and the same or similar components throughout the specification are given the same reference numerals.

[0061] Throughout this specification, when it is said that a device is "connected" to another device, this includes not only "direct connection" but also "indirect connection" by placing other components in between. Furthermore, when it is said that a device "comprises" a certain constituent element, unless otherwise stated otherwise, this does not exclude other constituent elements, but rather implies that other constituent elements may be included.

[0062] When we say that a device is "above" another device, this can mean that it is directly above the other device, or it can mean that other devices are present in between. Conversely, when we say that a device is "directly" "above" another device, there are no other devices present in between.

[0063] Although the terms first, second, etc., are used in some instances herein to refer to various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, first interface and second interface, etc., are used. Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of features, steps, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms “or” and “and / or” as used herein are interpreted as inclusive, or mean any one or any combination thereof. Thus, “A, B, or C” or “A, B, and / or C” means “any one of: A; B; C; A and B; A and C; B and C; A, B, and C.” Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0064] The technical terms used herein are for reference only to specific embodiments and are not intended to limit the scope of this disclosure. The singular form used herein includes the plural form unless the statement explicitly indicates otherwise. The word "comprising" as used in this specification means to specify a particular characteristic, region, integer, step, operation, element, and / or component, and does not exclude the presence or addition of other characteristics, regions, integers, steps, operations, elements, and / or components.

[0065] Although not explicitly defined, all terms, including technical and scientific terms used herein, shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. Terms defined in commonly used dictionaries shall be further interpreted as having a meaning consistent with the relevant technical literature and the content of this present disclosure, and shall not be over-interpreted as having an ideal or overly formulaic meaning unless otherwise defined.

[0066] To address the shortcomings of existing mobile sliding screens, such as their limited multimedia presentation and lack of interactivity with visitors, this disclosure provides several embodiments. Figure 1 A flowchart illustrating a mobile sliding screen control method for exhibition displays is shown. Figure 1 As shown, this type of mobile sliding screen control method for exhibition displays may include:

[0067] Step 101: Obtain the multimedia content to be presented on the mobile swipe screen and the environmental awareness information of the mobile swipe screen's location. The mobile swipe screen includes at least one fixed main screen and at least one mobile secondary screen. The mobile secondary screen is located on the content presentation side of the fixed main screen and moves along a fixed track on the content presentation side. The multimedia content to be presented and the environmental awareness information will be explained in detail later.

[0068] Step 102: Based on the multimedia content to be presented and the environmental awareness information, obtain the playback content corresponding to the mobile secondary screen and the fixed primary screen, as well as the movement strategy of the mobile secondary screen. The specific acquisition of the playback content and movement strategy will be explained in detail later.

[0069] Step 103: Control the fixed main screen and the mobile secondary screen to play the corresponding playback content, and control the mobile secondary screen to move according to the movement strategy.

[0070] Control the fixed main screen and the mobile secondary screen to play the corresponding playback content, and control the mobile secondary screen to move according to the movement strategy.

[0071] The first movement strategy is: constant speed movement + fixed point stop;

[0072] The mobile secondary screen moves at a fixed speed. When it reaches a certain fixed point, it stops and begins to explain in detail. It uses video images of the fixed position to identify whether there are audience members. If an audience member is identified, it continues to explain; if no audience member is identified, it completes one explanation and returns to constant speed cruise.

[0073] The audience dwell time recognition algorithm uses Faster R-CNN:

[0074] Faster R-CNN is a deep learning algorithm primarily used for object detection tasks.

[0075] The main approach used is to use a Region Proposal Network (RPN) to generate candidate regions, and then use a Fast R-CNN detector to classify these candidate regions and perform bounding box regression.

[0076] The algorithm has the following main characteristics:

[0077] 1. A two-stage detection method was adopted, which generates candidate regions through the RPN network and then uses the Fast R-CNN detector for classification and bounding box regression, thereby improving the accuracy of object detection.

[0078] 2. Region Proposal Network (RPN): The RPN is a fully convolutional neural network that generates a set of candidate regions on the input image. These candidate regions are fed into the Fast R-CNN detector for classification and bounding box regression. The RPN uses an anchor point mechanism to generate candidate boxes, thereby improving detection speed.

[0079] 3. Use a shared convolutional feature extractor: The RPN network and the Fast R-CNN detector share a convolutional feature extractor, which avoids redundant computation and improves the efficiency of the algorithm.

[0080] 4. Multi-scale feature maps: In order to detect targets at different scales, the algorithm uses multi-scale feature maps. By performing different scaling and sliding window operations on the feature maps, candidate regions at different scales are generated.

[0081] 5. End-to-End Training Method: The Faster R-CNN algorithm employs an end-to-end training method, training the RPN network and the Fast R-CNN detector together, thereby improving the algorithm's efficiency and accuracy. In summary, the Faster R-CNN algorithm improves the accuracy and efficiency of object detection by optimizing the RPN network and the Fast R-CNN detector, using techniques such as multi-scale feature maps and shared convolutional feature extractors.

[0082] Specific algorithm workflow:

[0083] 1) Input image;

[0084] (2) Extract features from the image by using convolutional layers in deep networks (such as those in VGG, Alexnet, and Resnet) to obtain the image feature map;

[0085] (3) Obtain the region of interest in the image through a selective search algorithm;

[0086] (4) Perform ROI pooling on the obtained region of interest: that is, by using coordinate projection, obtain the feature region corresponding to the region of interest in the input image on the feature map, and perform max pooling on the region, thus obtaining the features of the region of interest and unifying the feature size;

[0087] (5) The output of the ROI pooling layer (and the features after maximum pooling of the feature map corresponding to the region of interest) is used as the feature vector of each region of interest;

[0088] (6) Connect the feature vector of the region of interest to the fully connected layer and define a multi-task loss function, which is connected to the softmax classifier and the boxbounding regressor respectively, to obtain the category and coordinate bounding box of the current region of interest;

[0089] (7) Perform non-maximum suppression (NMS) on all the obtained bounding boxes to obtain the final detection results.

[0090] Movement Strategy 2: Follow the pace of visitors;

[0091] The system identifies the viewer's movement trajectory through video images, calculates the movement speed, moves the screen according to that speed, and displays the playback content at the current location.

[0092] The tracking algorithm works as follows:

[0093] (1) Create the corresponding Tracks for the results detected in the first frame. Initialize the motion variables of the Kalman filter and predict the corresponding bounding boxes using the Kalman filter. At this time, the Tracks must be unconfirmed.

[0094] (2) Perform IOU matching between the target detection box in this frame and the box predicted by Tracks in the previous frame, and then calculate the cost matrix (calculated as 1-IOU) based on the IOU matching result.

[0095] (3) Use all the cost matrices obtained in (2) as input to the Hungarian algorithm to obtain the linear matching result. At this time, we get three kinds of results. The first is Tracks mismatch (Unmatched Tracks). We directly delete the mismatched Tracks (because this Track is in an uncertain state. If it is in a deterministic state, it must be deleted after a certain number of consecutive times). The second is Detections mismatch (Unmatched Detections). We initialize such Detections as new Tracks. The third is that the detection box and the predicted box are successfully matched. This means that we have successfully tracked the previous frame and the next frame. We update the corresponding Tracks variable of the corresponding Detections through Kalman filtering.

[0096] (4) Repeat steps (2)-(3) until a confirmed Tracks or video frame appears.

[0097] (5) Predict the bounding boxes corresponding to the confirmed Tracks and the unconfirmed Tracks using Kalman filtering. Concatenate the bounding boxes of the confirmed Tracks and the Detections (previously, whenever a Track matched, the appearance features and motion information of the Detections were saved, with the first 100 frames saved by default. The appearance features and motion information are used to perform concatenated matching with the Detections because the confirmed Tracks are more likely to match the Detections).

[0098] (6) There are three possible results after cascaded matching. The first is Tracks matching, where Tracks are updated with their corresponding Tracks variables through Kalman filtering. The second and third are Detections and Tracks mismatch. In this case, the previously unconfirmed Tracks and the mismatched Tracks are matched with the Unmatched Detections one by one using IOU, and then the cost matrix (calculated as 1-IOU) is calculated based on the IOU matching results.

[0099] (7) Use all the cost matrices obtained in (6) as input to the Hungarian algorithm to obtain the linear matching result. At this time, we get three kinds of results. The first is Tracks mismatch (Unmatched Tracks). We directly delete the mismatched Tracks (because this Track is in an uncertain state. If it is in a deterministic state, it must be deleted after a certain number of consecutive times). The second is Detections mismatch (Unmatched Detections). We initialize such Detections as new Tracks. The third is that the detection box and the predicted box are successfully matched. This means that we have successfully tracked the previous frame and the next frame. We update the corresponding Tracks variable of the corresponding Detections through Kalman filtering.

[0100] (8) Repeat steps (5)-(7) repeatedly.

[0101] The system identifies the viewer's movement trajectory through video images, calculates the movement speed, and moves the screen to the desired position to play content according to that speed.

[0102] The following will further explain the specific implementation of steps 101 to 103 above through several practical application scenarios:

[0103] In some embodiments of this disclosure, the display resolution of the mobile secondary screen in a mobile swipe screen can be configured to be higher than that of the fixed primary screen. This is because the fixed primary screen has a larger display area, and a higher display resolution is more costly. Accordingly, in step 102 above, Figure 2 This diagram illustrates a flowchart illustrating a strategy for acquiring playback content corresponding to the mobile secondary screen and the fixed primary screen, as well as a movement strategy for the mobile secondary screen, based on the multimedia content to be presented and environmental awareness information. Figure 2 As shown, it includes the following steps:

[0104] Step 201: Fix the main screen to play the multimedia content to be presented.

[0105] Step 202: Continuously acquire the real-time relative position of the mobile secondary screen with respect to the fixed primary screen.

[0106] Step 203: Based on the real-time relative position, synchronize the multimedia content projected onto the fixed main screen from the mobile secondary screen to the mobile secondary screen for playback. It is understood that the display resolution of the mobile secondary screen is higher than that of the fixed main screen, and using the mobile secondary screen allows for better display of the multimedia content in the projection area.

[0107] For example, with Figure 8 For example

[0108] O(0,0): Fixed screen origin;

[0109] O'(a,b): The coordinates of the moving origin in the fixed screen coordinate system;

[0110] P(xo,yo): Fixed image coordinates on the screen;

[0111] P'(x',y'): Image coordinates in the moving scrolling screen;

[0112] n: Width of the swipe;

[0113] m: the height of the swipe screen;

[0114] G x'y': The set of image pixels captured from a fixed screen;

[0115] The formula for extracting data from a fixed screen to a swiping screen is as follows:

[0116] x' = xo - a (xo ∈ (0, n));

[0117] y' = yo - b (yo ∈ (0, m));

[0118] Interpolate G x'y' (increase the pixel count) and play the set K x'y' with higher pixel count on the swipe screen.

[0119] In other embodiments of this disclosure, the playback content of the mobile secondary screen can be configured to be different from the playback content of the fixed main screen, but closely related to the playback content of the fixed main screen. For example, in some practical application scenarios, the fixed main screen can play a side view of the exterior of a passenger vehicle, while the mobile secondary screen plays an interior view of the corresponding position of the passenger vehicle or a schematic image of the internal mechanical structure such as the engine and shock absorbers, depending on its position relative to the main screen. The combination of the two can achieve better playback and presentation effects.

[0120] In some embodiments of this disclosure, the multimedia content to be presented includes multiple highlight areas, which are distinct from the background areas and represent the most important content to be presented to the viewer. Accordingly, in step 102 described above, Figure 3 This diagram illustrates another flowchart illustrating a strategy for acquiring playback content for the mobile secondary screen and the fixed primary screen, based on the multimedia content to be presented and environmental awareness information, as well as the movement strategy for the mobile secondary screen. Figure 3 As shown, it includes the following steps:

[0121] Step 301: Continuously acquire the real-time relative position of the mobile secondary screen with respect to the fixed primary screen.

[0122] Step 302: Based on the real-time relative position, when there is an overlap between the projection of the mobile secondary screen onto the fixed main screen and the highlight content area, the playback content corresponding to the highlight content area is displayed through the mobile secondary screen. It is understandable that, due to the importance of the highlight content area, when the projection area of ​​the mobile secondary screen overlaps and obscures the highlight content area, the highlight content area can be directly presented through the mobile secondary screen.

[0123] Step 303: While displaying the playback content corresponding to the highlighted content area via the mobile secondary screen, play replacement background content at the corresponding position of the highlighted content area on the fixed main screen. Correspondingly, the relevant area on the fixed main screen can be replaced with the background content.

[0124] The playback of background replacement content at the corresponding position of the highlight content area further includes: (e.g.) Figure 9 (As shown)

[0125] O(0,0): Fixed screen origin;

[0126] O'(a,b): The coordinates of the moving swipe origin in the fixed screen coordinate system; refer to the swipe highlight and main screen background content comparison table as follows:

[0127] O’ Slide highlight content Home screen background content (a1, b1) G1 B1 (a2, b2) G2 B2 (a3, b3) G3 B3 (a4, b4) G4 B4 …… …… …… (an, bn) Gn Bn

[0128] When the screen is swiped horizontally, b1 = b2 = b3 = bn

[0129] Based on the real-time an value, the corresponding lookup table Gn is played in the swipe screen, while the replacement background Bn is played in the main screen, so as to realize the replacement background content playing at the corresponding position of the highlight content area.

[0130] In some embodiments of this disclosure, a virtual narrator can be implemented using a mobile secondary screen. In the above embodiments, the multimedia content to be presented can be configured to include content displayed on the main screen and narration content on the secondary screen, wherein the main screen displays the content by fixing the main screen; and the mobile secondary screen plays the corresponding narration content according to the real-time relative position of the mobile secondary screen with respect to the fixed main screen.

[0131] The mobile secondary screen plays corresponding narration content based on its relative position to the fixed main screen. Please refer to [link / reference]. Figure 9

[0132] O(0,0): Fixed screen origin;

[0133] O'(a,b): The coordinates of the moving origin in the fixed screen coordinate system;

[0134] The narration content is as follows:

[0135] O’ Explanation content (a1, b1) J1 (a2, b2) J2 (a3, b3) J3 (a4, b4) J4 …… …… (an, bn) Jn

[0136] When the screen is swiped horizontally, b1 = b2 = b3 = bn

[0137] Based on the real-time an value, the corresponding explanation Jn obtained from the lookup table is played in the sliding screen.

[0138] In the above embodiments, the mobile secondary screen can be further equipped with an environmental sensing component to sense the movement of visitors and their voice information. Correspondingly, in step 102 mentioned above, Figure 4 This diagram illustrates another flowchart illustrating a strategy for acquiring playback content for the mobile secondary screen and the fixed primary screen, based on the multimedia content to be presented and environmental awareness information, as well as the movement strategy for the mobile secondary screen. Figure 4 As shown, the process includes the following steps: obtaining visitor movement information based on environmental perception sensing components;

[0139] Step 401: Develop a movement strategy for the mobile sub-screen based on the movement of visitors, so that the mobile sub-screen follows or guides the movement of visitors.

[0140] Step 402: Based on the visitor's voice information, respond to the voice information in real time using a preset template. By setting up voice reception and real-time response, the interactivity between the mobile sliding screen structure and the visitor can be greatly improved, which helps to enhance the visitor experience.

[0141] In some embodiments of this disclosure, Figure 5 A mobile sliding screen control system for exhibitions is provided, which is applied to the mobile sliding screen control method for exhibitions provided in the aforementioned embodiments. The mobile sliding screen includes at least one fixed main screen and at least one mobile secondary screen. The mobile secondary screen is located on the content display side of the fixed main screen and moves on the content display side according to a fixed track.

[0142] like Figure 5 As shown, this type of mobile sliding screen control system for exhibition displays may include:

[0143] The perception unit 001 is used to acquire the multimedia content to be presented on the mobile swipe screen and the environmental perception information of the location of the mobile swipe screen.

[0144] The strategy formulation unit 002 is used to obtain the playback content corresponding to the mobile secondary screen and the fixed main screen, as well as the mobile secondary screen's movement strategy, based on the multimedia content to be presented and the environmental awareness information.

[0145] The control execution unit 003 is used to control the fixed main screen and the mobile secondary screen to play the corresponding playback content, and to control the mobile secondary screen to move according to the movement strategy.

[0146] It is understood that the functions implemented by the aforementioned sensing unit 001 to the control execution unit 003 correspond one-to-one with the operations performed in the aforementioned steps 101 to 103, and will not be elaborated here.

[0147] It is understood that various aspects of the technical solutions disclosed herein can be implemented as systems, methods, or program products. Therefore, various aspects of the technical solutions disclosed herein can be specifically implemented in the following forms: a completely hardware implementation method, a completely software implementation method (including firmware, microcode, etc.), or an implementation method combining hardware and software aspects, collectively referred to herein as a "circuit," "module," or "platform." Those skilled in the art should understand that the units, modules, or steps of the above-described disclosure can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using device-executable program code, thereby allowing them to be stored in a storage medium for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module.

[0148] Figure 6 A schematic diagram of the structure of an electronic device is shown according to some embodiments of this disclosure. Referring below... Figure 6 The electronic device 600 implemented according to the method described in this embodiment will be described in detail below. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of any embodiment of the technical solution disclosed herein.

[0149] like Figure 6 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0150] The storage unit stores program code, which can be executed by the processing unit 610, enabling the processing unit 610 to implement the various functional modules in the mobile sliding screen control system for exhibition and display described above in this embodiment.

[0151] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access unit (RAM) 6201 and / or cache storage unit 6202, and may further include read-only storage unit (ROM) 6203.

[0152] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each of these examples or some combination thereof may include an implementation of a network environment.

[0153] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, an image acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0154] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable user interaction with electronic device 600, and / or with any device that enables electronic device to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although... Figure 6 As not shown in the diagram, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0155] In some embodiments of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it can realize the various functional modules in the mobile sliding screen control system for exhibition and display disclosed above.

[0156] Although other specific implementation methods are not listed in detail in this embodiment, in some possible implementation methods, the various aspects described in this disclosure can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to cause the terminal device to perform specific steps of the implementation method according to the various embodiments of this disclosure.

[0157] Figure 7 A schematic diagram of the structure of a computer-readable storage medium is shown according to some embodiments of this disclosure. For example... Figure 7As shown, a program product 800 for implementing the above method according to the present disclosure is described. This product can be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. Of course, the program product generated according to this embodiment is not limited to this. In the present disclosure, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0158] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0159] Computer-readable storage media may include data signals propagated in baseband or as a carrier wave region, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0160] Program code for performing the operations of the disclosed technical solutions can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, locally on the user's device, as a standalone software package, locally on the user's computing device, locally on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0161] In summary, this disclosure provides a method, system, device, and computer-readable storage medium for controlling a mobile sliding screen in exhibitions. The technical solution proposed in this disclosure can adjust the playback content and movement strategy of both the fixed main screen and the mobile sub-screen based on their relative positions to the fixed main screen. Simultaneously, environmental awareness enhances the interactive experience between the mobile sliding screen and visitors, significantly improving the user experience and demonstrating its potential for widespread adoption.

[0162] The above description is only a description of the preferred embodiment of the technical solution disclosed herein, and is not intended to limit the scope of the technical solution disclosed herein. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A method for controlling a mobile sliding screen for exhibition and display, characterized in that, The mobile sliding screen includes at least one fixed main screen and at least one mobile secondary screen. The mobile secondary screen is located on the content display side of the fixed main screen and moves on the content display side according to a fixed track. The control method includes the following steps: Acquire the multimedia content to be presented on the mobile swipe screen and the environmental perception information of the location of the mobile swipe screen; Based on the multimedia content to be presented and the environmental awareness information, obtain the playback content corresponding to the mobile secondary screen and the fixed main screen, as well as the movement strategy of the mobile secondary screen. Control the fixed main screen and the mobile secondary screen to play the corresponding playback content, and control the mobile secondary screen to move according to the movement strategy; The multimedia content to be presented includes multiple highlight content areas; The process of obtaining the playback content corresponding to the mobile secondary screen and the fixed main screen, and the movement strategy of the mobile secondary screen, respectively, based on the multimedia content to be presented and the environmental awareness information, includes the following steps: Continuously acquire the real-time relative position of the mobile secondary screen with respect to the fixed main screen; Based on the real-time relative position, when the projection of the mobile sub-screen onto the fixed main screen overlaps with the highlight content area, the playback content corresponding to the highlight content area is displayed through the mobile sub-screen.

2. The mobile sliding screen control method for exhibition display as described in claim 1, characterized in that, The display resolution of the mobile secondary screen is higher than that of the fixed main screen; The process of obtaining the playback content corresponding to the mobile secondary screen and the fixed main screen, and the movement strategy of the mobile secondary screen, respectively, based on the multimedia content to be presented and the environmental awareness information, includes the following steps: The fixed main screen plays the multimedia content to be presented; Continuously acquire the real-time relative position of the mobile secondary screen with respect to the fixed main screen; Based on the real-time relative position, the multimedia content of the projection area of ​​the mobile secondary screen on the fixed main screen is synchronized to the mobile secondary screen for playback; Furthermore, the formula for setting data from a fixed screen capture to a swiping screen is as follows: x' = xo - a(xo ∈ (0, n)); y' = yo - b (yo ∈ (0, m)); Interpolation calculations are performed on the set of image pixels Gx'y' captured from the fixed screen to enable the playback of the higher pixel set Kx'y' on the sliding screen. O(0,0): Fixed screen origin; O'(a,b): The coordinates of the moving origin in the fixed screen coordinate system; P(xo,yo): Image coordinates on the fixed screen; P' (x',y'): Image coordinates in the moving scrolling screen; n: Width of the swipe; m: the height of the swipe screen; G x' y': The set of image pixels captured from a fixed screen.

3. The mobile sliding screen control method for exhibition display as described in claim 1, characterized in that, When the playback content corresponding to the highlighted content area is displayed on the mobile secondary screen, playing background replacement content at the corresponding position of the highlighted content area on the fixed main screen further includes: When swiping horizontally, set b1=b2=b3=bn; Based on the real-time an value, the corresponding highlight content in the swipe screen is played according to the lookup table Gn, which compares the swipe screen highlight with the main screen background content. At the same time, the replacement background content Bn is played in the main screen, thus realizing the playback of the replacement background content at the corresponding position of the highlight content area. Where O(0,0): fixed screen origin; O'(a,b): coordinates of the moving screen origin in the fixed screen coordinate system.

4. The mobile sliding screen control method for exhibition display as described in any one of claims 1-3, characterized in that, The multimedia content to be presented includes the content displayed on the main screen and the explanatory content on the secondary screen; The fixed main screen plays the content displayed on the main screen; The mobile secondary screen plays corresponding narration content based on its relative position to the fixed main screen. When swiping horizontally, set b1=b2=b3=bn; The narration Jn is obtained by playing the corresponding narration content table in the sliding screen based on the real-time an value; Where O(0,0): fixed screen origin; O'(a,b): coordinates of the moving screen origin in the fixed screen coordinate system.

5. The mobile sliding screen control method for exhibition display as described in claim 4, characterized in that, The mobile sub-screen is equipped with an environmental sensing component, and the environmental sensing information includes the movement of visitors. The process of obtaining the playback content corresponding to the mobile secondary screen and the fixed main screen, and the movement strategy of the mobile secondary screen, respectively, based on the multimedia content to be presented and the environmental awareness information, includes the following steps: The movement of visitors is obtained based on the environmental perception sensing component; Based on the movement of the visitor flow, a movement strategy for the mobile secondary screen is formulated, causing the mobile secondary screen to move in sync with the visitor flow; or The movement strategy of the mobile sub-screen is formulated based on the movement of the visitor flow, so that the mobile sub-screen guides the movement of the visitor flow.

6. The mobile sliding screen control method for exhibition display as described in claim 5, characterized in that, The environmental perception information includes the voice information of visitors; The process of obtaining the playback content corresponding to the mobile secondary screen and the fixed main screen, and the movement strategy of the mobile secondary screen, respectively, based on the multimedia content to be presented and the environmental awareness information, includes the following steps: The voice information is obtained based on the environmental perception sensing component; The system responds to the voice information in real time according to a preset template.

7. A method for controlling a mobile sliding screen for exhibition and display, characterized in that: The tracking algorithm identifies the viewer's movement trajectory from video images, calculates the movement speed, moves the screen according to that speed, and displays the playback content at the current location. Its workflow is as follows: (1) Create the corresponding Tracks for the results detected in the first frame, initialize the motion variables of the Kalman filter, and predict the corresponding bounding boxes through the Kalman filter. At this time, the Tracks must be unconfirmed. (2) Perform IOU matching between the target detection bounding box in this frame and the bounding box predicted by Tracks in the previous frame, and then calculate the cost matrix based on the IOU matching results. The calculation method is 1-IOU. (3) Use all the cost matrices obtained in (2) as input to the Hungarian algorithm to obtain the linear matching result. There are three possible results: the first is Tracks mismatch, in which case the mismatched Tracks are deleted directly; the second is Detections mismatch, in which case the Detections are initialized to a new Tracks; the third is that the detection box and the predicted box are successfully matched, which means that the tracking of the previous frame and the next frame is successful. Update the corresponding Tracks variable of the corresponding Detections through Kalman filtering. (4) Repeat steps (2)-(3) until a confirmed Tracks appears or the video frame ends. (5) Predict the bounding boxes corresponding to the confirmed and unconfirmed tracks using Kalman filtering, and perform concatenated matching of the bounding boxes of the confirmed tracks and the detections. (6) After cascaded matching, there are three possible results. The first is Tracks matching, in which case the Tracks are updated with their corresponding Tracks variables through Kalman filtering. The second and third are Detections and Tracks mismatch. In this case, the previously uncertain Tracks and the mismatched Tracks are matched with the Unmatched Detections one by one using IOU matching. The cost matrix ost matrix is ​​then calculated based on the IOU matching result. The calculation method is 1-IOU. (7) Use all the cost matrices obtained in (6) as input to the Hungarian algorithm to obtain the linear matching result. There are three types of results at this time. The first type is Tracks mismatch. The mismatched Tracks are directly deleted. If it is a deterministic state, it must be continuously reached a certain number of times before it is deleted. The second type is Detections mismatch. Such Detections are initialized as new Tracks. The third type is that the detection box and the predicted box are successfully matched. This means that the tracking of the previous frame and the next frame is successful. The corresponding Detections are updated with their corresponding Tracks variables through Kalman filtering. (8) Repeat steps (5)-(7) repeatedly.

8. A mobile sliding screen control device for exhibition and display, characterized in that, include: Memory, used to store computer programs; A processor is configured to implement the mobile sliding screen control method for exhibition display as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the mobile sliding screen control method for exhibition display as described in any one of claims 1 to 6.

Images