Information display method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to an information display method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring a real-time picture comprising a to-be-displayed real object and mark information of the to-be-displayed real object; marking the real object to be displayed in the real-time picture based on the marking information and generating a marked picture; and displaying the marked picture online in real time. Through the method, the online display function is enriched, and the display experience is improved.

Description

Technical Field

[0001] This disclosure relates to the field of image display technology, and in particular to an information display method and apparatus, electronic device, storage medium, and program product. Background Technology

[0002] With the widespread adoption of applications such as distance education and online conferencing, the demand for online presentations is increasing. However, current traditional online presentation methods only allow for annotation or drawing on projected electronic documents, limiting the functionality of online presentations and resulting in a restricted user experience. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides an information display method and apparatus, electronic device, storage medium, and program product.

[0004] According to a first aspect of the present disclosure, an information display method is provided, the method comprising:

[0005] Acquire real-time images of the object to be displayed, as well as the marking information of the object to be displayed;

[0006] The objects to be displayed in the real-time image are marked based on the marking information, and a marked image is generated.

[0007] The marked screen is displayed online in real time.

[0008] In some embodiments, acquiring the real-time image of the object to be displayed and the marking information of the object to be displayed includes:

[0009] Real-time acquisition of images of the object to be displayed, and display of a real-time screen including the object to be displayed based on the images;

[0010] In response to the detection of a touch operation on the real-time screen, the marking information of the object to be displayed is obtained.

[0011] In some embodiments, the step of obtaining the marking information of the object to be displayed in response to detecting a touch operation on the real-time screen includes:

[0012] In response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image, acquire marker information associated with the first touch operation; and / or,

[0013] The real-time screen also includes a control for indicating the marking method, and in response to detecting a second touch operation on the control in the real-time screen, the marking information associated with the second touch operation is obtained.

[0014] In some embodiments, the step of obtaining tag information associated with the first touch operation and the second touch operation in response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image, and in response to detecting a second touch operation on a control in the real-time image, includes:

[0015] In response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image, the marked position indicated by the first touch operation is obtained;

[0016] In response to detecting a second touch operation on the control in the real-time screen, the mark style indicated by the control operated by the second touch operation is obtained;

[0017] The marker information is obtained based on the marker position and the marker style.

[0018] In some embodiments, the step of obtaining the marker position indicated by the first touch operation in response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image includes:

[0019] In response to detecting a sliding operation on the area where the object to be displayed is located in the real-time screen, and the existence of a closed sliding trajectory, the area indicated by the closed sliding trajectory in the real-time screen is determined as the marked position indicated by the first touch operation.

[0020] In some embodiments, marking the object to be displayed in the real-time image based on the marking information and generating a marked image includes:

[0021] If the real-time screen also includes a control for indicating the marking method, the object to be displayed in the real-time screen is marked based on the marking information indicated by the control operated by the second touch operation, and a marked screen without the control is generated.

[0022] In some embodiments, the controls in the live feed include at least one of the following:

[0023] The first control used to indicate the highlighted content;

[0024] A second control used to indicate where to add comments;

[0025] A third control used to indicate the line drawing;

[0026] A fourth control used to indicate scaling.

[0027] In some embodiments, the method further includes:

[0028] In response to the failure to capture an image of the object to be displayed during the real-time online display of the marked screen, the online display screen is updated to include the object to be displayed and the markings on the object to be displayed.

[0029] In response to updating the online display screen, an image of the physical object to be displayed is captured, and the display of the marked screen is restored; wherein, the restored real-time screen is generated based on the currently captured image of the physical object to be displayed and the marking information corresponding to the marked screen before the update.

[0030] In some embodiments, the marking information includes a marking style and a marking position. Marking the object to be displayed in the real-time image based on the marking information and generating a marked image includes:

[0031] In response to detecting a size change of the object to be displayed in the real-time image, the position of the marker information is adjusted based on the changed size of the object to be displayed;

[0032] Based on the marking style indicated by the marking information, the object to be displayed in the real-time image after adjustment of size is marked according to the adjusted marking position, and the marked image is generated.

[0033] In some embodiments, the real-time online display of the marker screen includes:

[0034] The marked image is sent to the receiving device; wherein the marked image is used to be displayed in real time on the receiving device.

[0035] According to a second aspect of the present disclosure, an information display device is provided, the device comprising:

[0036] The acquisition module is configured to acquire a real-time image of the object to be displayed and the marking information of the object to be displayed;

[0037] The marking module is configured to mark the physical object to be displayed in the real-time image based on the marking information and generate a marked image;

[0038] The display module is configured to display the marked screen online in real time.

[0039] In some embodiments, the acquisition module is further configured to acquire an image of the object to be displayed, and display a real-time screen including the object to be displayed based on the image; and acquire the marking information of the object to be displayed in response to detecting a touch operation on the real-time screen.

[0040] In some embodiments, the acquisition module is further configured to, in response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image, acquire marker information associated with the first touch operation; and / or,

[0041] The real-time screen also includes a control for indicating the marking method, and in response to detecting a second touch operation on the control in the real-time screen, the marking information associated with the second touch operation is obtained.

[0042] In some embodiments, the acquisition module is further configured to, in response to detecting a first touch operation on the area where the object to be displayed is located in the real-time screen, acquire the mark position indicated by the first touch operation; in response to detecting a second touch operation on the control in the real-time screen, acquire the mark style indicated by the control operated by the second touch operation; and acquire the mark information based on the mark position and the mark style.

[0043] In some embodiments, the acquisition module is further configured to, in response to detecting a sliding operation on the area where the object to be displayed is located in the real-time image and a closed sliding trajectory exists, determine the area indicated by the closed sliding trajectory in the real-time image as the marked position indicated by the first touch operation.

[0044] In some embodiments, the marking module is further configured to, when the real-time screen also includes a control for indicating the marking method, mark the object to be displayed in the real-time screen based on the marking information indicated by the control operated by the second touch operation and generate a marked screen that does not include the control.

[0045] In some embodiments, the controls in the live feed include at least one of the following:

[0046] The first control used to indicate the highlighted content;

[0047] A second control used to indicate where to add comments;

[0048] A third control used to indicate the line drawing;

[0049] A fourth control used to indicate scaling.

[0050] In some embodiments, the apparatus further includes:

[0051] The update module is configured to update the online display screen to a screen that does not include the object to be displayed and the markings on the object to be displayed if no image of the object to be displayed is captured during the real-time online display of the marked screen.

[0052] The recovery module is configured to recover the display of the marked screen after acquiring an image of the physical object to be displayed in response to an update of the online display screen; wherein the recovered marked screen is generated based on the currently acquired image of the physical object to be displayed and the marking information corresponding to the marked screen before the update.

[0053] In some embodiments, the tagging information includes tagging style and tagging position.

[0054] The marking module is further configured to, in response to detecting a size change of the object to be displayed in the real-time image, adjust the marking position of the marking information based on the changed size of the object to be displayed; mark the object to be displayed in the real-time image after size adjustment according to the marking style indicated by the marking information and the adjusted marking position, and generate the marked image.

[0055] In some embodiments, the display module is further configured to send the marked image to a receiving device; wherein the marked image is used to be displayed in real time on the receiving device.

[0056] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0057] processor;

[0058] Memory used to store computer programs or instructions;

[0059] The processor executes the computer program or instructions to implement the steps of the method described in any one of the first aspects of the embodiments of this disclosure.

[0060] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, the storage medium storing a computer program or instructions that, when executed by a processor, implement the steps of the method described in any one of the first aspects of the present disclosure.

[0061] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implement the steps of the method described in any one of the first aspects of the present disclosure.

[0062] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0063] In this embodiment of the disclosure, by acquiring a real-time image containing the object to be displayed and its marking information, and by marking the object in the real-time image based on the marking information, a marked image is generated for online display. Compared to the limited function of only being able to annotate or draw on electronic files, the solution in this embodiment of the disclosure also supports marking and displaying the object during the physical demonstration, enriching the online display function of the electronic device; and by marking the real-time image of the object to be displayed in real time, the electronic device can instantly convey the demonstration content including the marked object, improving the demonstration effect and enriching the display experience.

[0064] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0065] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0066] Figure 1 This is a flowchart illustrating an information display method according to an exemplary embodiment. Figure 1 .

[0067] Figure 2 This is a schematic diagram of a real-time screen including controls, according to an exemplary embodiment.

[0068] Figure 3 This is a flowchart illustrating an information display method according to an exemplary embodiment. Figure 2 .

[0069] Figure 4 This is a block diagram illustrating an information display device according to an exemplary embodiment.

[0070] Figure 5 This is a structural block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0071] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0072] With the continuous advancement of digital technology, real-time image annotation has become relatively easy, but these technologies typically require the images to be annotated to be digitized. However, in everyday online video presentations, users often encounter situations where they need to demonstrate real-world objects, such as physical models or printed materials. Because these objects are not digitized, users cannot directly annotate or draw on them in real time to aid in the presentation. Online presentation functions are limited in their ability to annotate real-world objects. Specifically, when users need to emphasize or explain a detail of an object during a presentation, the lack of immediate annotation tools may force them to rely on verbal descriptions, severely impacting the user's presentation experience.

[0073] In response, this disclosure provides an information display method. Figure 1 This is a flowchart illustrating an information display method according to an exemplary embodiment. For example... Figure 1 As shown, the method mainly includes the following steps:

[0074] In step S11, a real-time image of the object to be displayed and the marking information of the object to be displayed are acquired;

[0075] In step S12, the object to be displayed in the real-time image is marked based on the marking information, and a marked image is generated;

[0076] In step S13, the marked screen is displayed online in real time.

[0077] This disclosure provides an information display method applied to an electronic device, which can be a terminal device, or a cloud device, relay device, etc., connected to the terminal device. The terminal device includes, but is not limited to, smart devices such as smartphones, tablets, personal digital assistants (PDAs), handheld devices, computing devices, and wearable devices.

[0078] In step S11, the electronic device acquires a real-time image including the object to be displayed. The object to be displayed is the target object that the user wishes to mark and display during the online demonstration. It can be diverse, including not only the aforementioned physical models and paper-based content, but also any physical object such as biological samples or product samples. The real-time image can be a dynamic video frame, showing either the full view of the object to be displayed or a close-up of a specific part. The number of objects to be displayed in the real-time image can be one or more. This embodiment of the disclosure does not limit the form or content of the real-time image.

[0079] In step S11, the electronic device also acquires marking information of the object to be displayed. This marking information is used to indicate key parts of the object or to explain relevant information about the object. It can include color markings, text annotations, underlines, arrow annotations, size scaling, etc. The acquisition of marking information can be based on touch operations on the real-time screen displaying the object, or it can be determined according to user voice commands. The marking information is used to indicate the marking position and style.

[0080] In some embodiments, a user can use a touch-enabled input device or their finger to interact with a real-time object displayed on a screen, and the electronic device will then acquire marking information based on the touch operation. In other embodiments, the electronic device can capture the user's voice commands via a microphone assembly and identify the content of the marking information associated with the voice commands.

[0081] It should be noted that, in this embodiment of the disclosure, the electronic device can obtain all the marking information based on the user's touch operation or voice command, and can also obtain part of the marking information based on the aforementioned touch operation or voice command. If the electronic device obtains only part of the marking information based on the touch operation or voice command, the electronic device can supplement the marking information based on a built-in algorithm.

[0082] In some embodiments, the electronic device detects a touch operation indicating the marking style and position. For example, a touch operation involving two fingers simultaneously swiping / contracting diagonally on a real-time screen displaying an object can be used to determine the marking position. The zoom in / out indicated by the swiping / contracting action can be determined as the marking style. Based on this, the electronic device can obtain all the marking information based on the user's touch operation. In some embodiments, the electronic device receives a voice command indicating both the marking style and position, such as "highlight the center point of the object in red." Based on the recognition of this command, the marking position can be determined as the center point of the object, and the marking style is highlighted in red, thereby obtaining all the marking information. In other embodiments, the electronic device... The sub-device detects a touch operation, such as a click, on a certain area of ​​the real-time screen. The click operation only indicates the marked location. The electronic device can also mark the marked location indicated by the click operation based on default settings, such as a default yellow highlighting style. In this case, even if the touch operation is only marking the location, the electronic device can obtain the marking style associated with the marked location. Alternatively, the electronic device only receives a voice command indicating the marking style, such as "zoom in on the object". Based on the recognition of this command, the marking style can be obtained as "zoom in on the object to be displayed". The electronic device can also determine the marking location based on default settings, such as a default zoom of 1x. In this case, even if the voice command only indicates the marking style, the electronic device can obtain the marking location associated with the marking style.

[0083] It should be noted that if the executing entity is a cloud device or a relay device, the cloud device or relay device can receive real-time images of the object to be displayed and the marking information of the object to be displayed sent by the terminal device.

[0084] In step S12, the electronic device marks the object to be displayed in the real-time screen based on the marking information and generates a marked screen. As mentioned above, the marking information can be used to indicate the marking position and marking style. The user can mark the object to be displayed in the real-time screen based on the marking information. Specifically, the user can render the style of the object to be displayed in the real-time screen indicated by the marking position based on the marking style. The generated marked screen can display the marked object to be displayed.

[0085] In step S13, the electronic device displays the marked image online in real time. As mentioned above, the marked image is an image including the object to be displayed and the marking information. To present the marked object, the marked image can be displayed online in real time. In some embodiments, the electronic device directly displays the generated marked image in real time, and the user can view the marked object in the marked image on the electronic device's screen.

[0086] In other embodiments, the real-time online display of the marked screen includes:

[0087] The marked image is sent to the receiving device; wherein the marked image is used to be displayed in real time on the receiving device.

[0088] In this embodiment, after the electronic device sends the marked image to the receiving device, the receiving device receives the marked image and displays it in real time. Thus, the receiving user can view the marked object to be displayed, as intended by the presenter, within the marked image. This effectively conveys information from the presenter to the receiving user and emphasizes the key features of the object to be displayed. It should be noted that the electronic device in this embodiment can display the marked image online to the receiving device via screen projection or live streaming.

[0089] It is understood that, in this embodiment of the present disclosure, by acquiring a real-time image containing the object to be displayed and the marking information of the object to be displayed, and by marking the object to be displayed in the real-time image based on the marking information, a marked image is generated for online display. Compared with the limited function of only being able to annotate or draw on electronic files, the solution in this embodiment of the present disclosure supports marking and displaying the object during the physical demonstration, which enriches the online display function of the electronic device; and by marking the real-time image of the object to be displayed in real time, the electronic device can immediately convey the demonstration content including the marked object to be displayed, improve the demonstration effect, and enrich the display experience.

[0090] In some embodiments, acquiring the real-time image of the object to be displayed and the marking information of the object to be displayed includes:

[0091] Real-time acquisition of images of the object to be displayed, and display of a real-time screen including the object to be displayed based on the images;

[0092] In response to the detection of a touch operation on the real-time screen, the marking information of the object to be displayed is obtained.

[0093] In this embodiment, the electronic device acquires images of the object to be displayed in real time. The acquired images are processed and displayed on the screen of the electronic device, forming a real-time image including the object to be displayed. The electronic device acquires images of the object to be displayed and combines artificial intelligence (AI) recognition technology, such as recognizing the object in the image based on captured feature information such as object shape and color distribution, thereby obtaining a real-time image including the object to be displayed. The object to be displayed is displayed on the screen of the electronic device through the real-time image for the demonstration user to view.

[0094] In this embodiment of the disclosure, the electronic device continuously detects user touch operations on the live screen. In response to detecting a touch operation on the live screen, it obtains the marking information of the object to be displayed based on the user's touch operation. Here, the touch operation on the live screen refers to the interactive operation performed by the user on the live screen displayed on the electronic device screen.

[0095] In some embodiments, the live feed may include controls for indicating marker information, and the electronic device may detect operations on the controls to obtain the marker information.

[0096] In other embodiments, where controls are not present in the live view, the electronic device can obtain marking information based on the user's touch operation on a region of the live view. For example, after detecting a selection operation on the area of ​​the object to be displayed in the live view, the electronic device pops up a color selection box and determines the marking information based on the color selected by the user: marking the selected area with the selected color.

[0097] It is understood that, in this embodiment of the disclosure, by real-time acquisition and display of a real-time image including the object to be displayed, and by receiving touch operations on the real-time image to obtain marking information, users can intuitively and instantly mark the object to be displayed based on the real-time image, which helps to improve the accuracy and timeliness of marking and enhance the user's interactive experience.

[0098] In some embodiments, the step of obtaining the marking information of the object to be displayed in response to detecting a touch operation on the real-time screen includes:

[0099] In response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image, acquire marker information associated with the first touch operation; and / or,

[0100] The real-time screen also includes a control for indicating the marking method, and in response to detecting a second touch operation on the control in the real-time screen, the marking information associated with the second touch operation is obtained.

[0101] In this embodiment of the disclosure, the touch operation on the real-time screen includes three touch methods: touch operation performed directly by the user on the real-time screen, touch operation performed by the user on the real-time screen based on the selected control, and a combination of the above two methods.

[0102] In some embodiments, the electronic device responds to detecting a first touch operation on the area where an object to be displayed is located in a live screen. In this case, the first touch operation may indicate a marking position or marking style, and the electronic device may determine the marking style or marking position associated with the first touch operation based on a built-in algorithm. For example, the first touch operation may be a line-drawing operation on the area where the object to be displayed is located in the live screen. In this case, the first touch operation indicates a marking position, and the electronic device may highlight the line position (i.e., the marking position) in yellow based on the default highlighting function (marking style) associated with the line-drawing operation. Based on this, the electronic device may obtain the marking style associated with the first touch operation. Alternatively, the first touch operation may be, for example, a color selection operation. In this case, the first touch operation indicates a marking style, and the electronic device may mark the center point position (marking position) of the object to be displayed based on the selected color (marking style). Based on this, the electronic device may obtain the marking position associated with the first touch operation.

[0103] In other embodiments, the live feed also includes controls for indicating the marking method, such as... Figure 2 This is a schematic diagram of a real-time screen including controls, according to an exemplary embodiment. The real-time screen includes controls for indicating different marking methods. When an electronic device detects a second touch operation on any of these controls, it can obtain marking information associated with the second touch operation. For example, Figure 2 The control identified by L1 is highlighted. When the electronic device detects a click on the L1 control, it receives the highlighted marker information. It should be noted that this marker information indicates the marker style; the electronic device can also determine the marker position based on default settings, voice commands, or selection commands on non-control parts of the real-time screen. For example... Figure 2 The control identified by L2 indicates "draw a line at the bottom". Therefore, when the electronic device detects a click on the L2 control, it can obtain the marking information for the bottom line of the object to be displayed; for example... Figure 2 The control identified by L3 indicates "magnify the object". When the electronic device detects a click on the L3 control, it can obtain the marking information that the surface area of ​​the displayed object is magnified. It should be noted that the marking information also indicates the marking style. The electronic device can also determine the marking position based on default settings (such as the magnification ratio).

[0104] In other embodiments, the marking information associated with the object to be displayed can be obtained by combining the first touch operation and the second touch operation on the control. There can be two combinations. For example, in one method, the electronic device, in response to detecting a first touch operation on the area of ​​the object to be displayed in the real-time screen, obtains the marking style indicated by the first touch operation; and in response to detecting a second touch operation on the control in the real-time screen, obtains the marking style indicated by the control operated by the second touch operation. For instance, if the electronic device detects a first touch operation on the area of ​​the object to be displayed in the real-time screen (e.g., a click operation that brings up a color selection box for color selection), and selects a control with a highlighting function on the real-time screen based on the second touch operation, then the electronic device can determine the color indicated by the first touch operation as the marking style, and determine the outline area of ​​the object to be displayed on the real-time screen by the second touch operation as the marking position.

[0105] For example, in Method 2, the step of obtaining tag information associated with the first touch operation and the second touch operation in response to detecting a first touch operation on the area where the object to be displayed is located in the real-time screen, and in response to detecting a second touch operation on a control in the real-time screen, includes:

[0106] In response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image, the marked position indicated by the first touch operation is obtained;

[0107] In response to detecting a second touch operation on the control in the real-time screen, the mark style indicated by the control operated by the second touch operation is obtained;

[0108] The marker information is obtained based on the marker position and the marker style.

[0109] In this embodiment of the disclosure, the real-time screen includes a control for indicating the marking method, and the touch operation performed on the real-time screen includes a first touch operation and a second touch operation. A first touch operation is detected that operates on the area of ​​the object to be displayed in the real-time screen. Since the first touch operation acts on a specific location, the marking location can be determined based on the first touch operation. For example, the electronic device can select a local area of ​​the object to be displayed through a click operation, and then that local area is determined as the marking location; or a closed shape can be drawn on the area of ​​the object to be displayed through a swipe operation, and the closed shape is determined as the marking area.

[0110] In this embodiment of the disclosure, the electronic device can also select a control in the real-time screen based on a second touch operation to select a marking style, such as highlighting, adding annotations, or drawing lines. When a control is selected, the electronic device can capture this operation and obtain the marking style corresponding to the control from the control.

[0111] In the process of determining the marking information of the object to be displayed, there is no limitation on the order of determining the marking position based on the first touch operation and determining the marking style based on the second touch operation. That is to say, when the user marks the object to be displayed based on touch operation, if the marking position is determined first based on the first touch operation, the electronic device can determine the marking style at that marking position on the real-time screen based on the second touch operation; similarly, if the marking style is determined first based on the second touch operation, the electronic device can determine the marking position of that marking style on the real-time screen based on the first touch operation.

[0112] In this embodiment, after determining the marking position and marking style, these two pieces of information are combined to determine the complete marking information. Although marking information can be obtained solely based on direct association with touch operations on the object and control to be displayed, and is simpler in some scenarios, it may limit the user's operational scope and flexibility. Especially when precise marking is required, direct association with touch operations on the object and control may not be intuitive or accurate enough. In this embodiment, since the marking position and marking style are obtained separately, the marking position and marking style to be marked can be determined more flexibly.

[0113] It is understood that, through simple touch operations and / or control selection, the present disclosure embodiments enable flexible marking of the physical objects to be displayed in the real-time screen, thereby improving user experience and demonstration efficiency.

[0114] In some embodiments, the step of obtaining the marker position indicated by the first touch operation in response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image includes:

[0115] In response to detecting a sliding operation on the area where the object to be displayed is located in the real-time screen, and the existence of a closed sliding trajectory, the area indicated by the closed sliding trajectory in the real-time screen is determined as the marked position indicated by the first touch operation.

[0116] In this embodiment of the disclosure, the first touch operation is a sliding operation on the area of ​​the object to be displayed in the real-time screen. The user can operate the object to be displayed to slide in the real-time screen. The electronic device detects the trajectory formed by the user's sliding operation on the object to be displayed in real time and performs closed trajectory detection on the sliding trajectory. A closed sliding trajectory can be a closed shape formed by the starting and ending points of the sliding trajectory being very close or overlapping, for example... Figure 2 L4 in the text can also refer to a continuous and closed shape formed by a portion of the sliding trajectory. The closed trajectory detection function can be implemented using methods such as path continuity-based closure judgment or shape matching-based closure detection. For example, in the path continuity-based closure judgment method, the continuity of each point in the sliding trajectory is checked. If multiple points in the sliding trajectory are continuous (i.e., each point is connected to its preceding and following points), and the first and last positions of these continuous points are close to or coincident, then a closed trajectory can be determined. If the shape matching-based closure judgment method is used, the shape features of the detected sliding trajectory can be extracted, and these extracted shape features can be matched with a predefined closed trajectory template. If the matching degree reaches a preset threshold, it is determined to be a closed trajectory.

[0117] In this embodiment of the disclosure, the electronic device identifies a closed trajectory within the sliding trajectory. The closed trajectory can be referenced. Figure 2 The white area in the real-time screen can be identified as the marked position indicated by the trajectory (i.e., the area to be marked by the first touch operation). The area indicated by the closed trajectory will be highlighted or highlighted in other marking ways so that the user can clearly see the markings that the user has used to treat the displayed object.

[0118] In some other embodiments, not only is the object to be displayed in the real-time image marked at the marked location indicated by the first touch operation, but areas in the real-time image other than the marked location can also be marked in other ways to further enhance the marking effect of the marked location. It should be noted that the marking style of the marked location in the real-time image and the marking style of the areas in the real-time image other than the marked location should be as opposite as possible. For example, the marked location in the real-time image is highlighted, while the areas in the real-time image other than the marked location are darkened.

[0119] It is understood that the method of forming a closed trajectory through sliding operation in the real-time image and determining the desired mark position based on the closed trajectory in the embodiments of this disclosure makes the process of determining the mark position in the real-time image relatively simple and intuitive, and can improve the accuracy and precision of the mark position information.

[0120] In some embodiments, marking the object to be displayed in the real-time image based on the marking information and generating a marked image includes:

[0121] If the real-time screen also includes a control for indicating the marking method, the object to be displayed in the real-time screen is marked based on the marking information indicated by the control operated by the second touch operation, and a marked screen without the control is generated.

[0122] In this embodiment of the disclosure, the real-time screen includes controls for indicating the marking method. When the user selects a control for a certain marking method based on the second touch operation, and the electronic device marks the object to be displayed in the real-time screen according to the marking information indicated by the control and generates a marked screen, these controls for indicating the marking method are excluded. Thus, the generated marked screen only contains the marked object to be displayed and the original background information of the real-time screen. The receiving user can see the marked object to be displayed more simply and intuitively without being disturbed by extra controls.

[0123] It is understood that, in the embodiments of this disclosure, when the real-time screen includes a control for indicating the marking method, the control operated by the user's second touch operation marks the physical object to be displayed, generating a marked screen that does not include the control. This makes the screen presented to the receiving user during the demonstration clear, concise, and intuitive, reduces the interference of the control, and improves the user experience of the receiving user.

[0124] In some embodiments, the controls in the live feed include at least one of the following:

[0125] The first control used to indicate the highlighted content;

[0126] A second control used to indicate where to add comments;

[0127] A third control used to indicate the line drawing;

[0128] A fourth control used to indicate scaling.

[0129] In this embodiment, multiple controls for indicating marking methods are provided on the live screen, including a first control for highlighting, a second control for adding annotations, a third control for drawing lines, and a fourth control for zooming. The user selects the corresponding control on the live screen via a second touch operation, and the electronic device can perform corresponding marking processing on the object to be displayed on the live screen according to the control selected by the second touch operation. For example, if the user selects the first control for highlighting via the second touch operation, and the electronic device selects highlighting as the marking method for the object to be displayed, then a highlighted border can be automatically drawn around the object or other highlighting effects can be applied to highlight the object. If the user selects the second control for adding annotations via the second touch operation, text descriptions can be added to the object to be displayed. If the user selects the third control for supporting drawing lines via the second touch operation, lines can be used to mark the parts of the object to be displayed that need to be emphasized. The user can also select the fourth control for indicating zoom via the second touch operation to adjust the display size of the object to be displayed on the live screen.

[0130] It should be noted that when there are multiple controls in the real-time screen, the user can select different controls in the real-time screen in sequence based on the second touch operation. The operation effect shown on the screen is the marking effect of the control indication. Based on the multiple selected controls, the object to be displayed is marked, and the marking effect of the object to be displayed on the real-time screen can be superimposed and displayed to the receiving user.

[0131] It is understood that by providing a variety of marking controls in the real-time screen, users can flexibly select the marking method corresponding to the control according to their actual needs to mark the physical objects to be displayed in the real-time screen. This flexibility enables the solution to meet the diverse marking needs of users, and also improves the convenience and accuracy of marking through the intuitive control selection operation method.

[0132] Figure 3 This is a flowchart illustrating an information display method according to an exemplary embodiment. Figure 2 ,like Figure 3 As shown, the method further includes:

[0133] In step S34, in response to the failure to capture an image of the object to be displayed during the real-time online display of the marked screen, the online display screen is updated to a screen that does not include the object to be displayed and the mark of the object to be displayed;

[0134] In step S35, in response to updating the online display screen, an image of the object to be displayed is acquired, and the display of the marked screen is restored; wherein, the restored marked screen is generated based on the currently acquired image of the object to be displayed and the marking information corresponding to the marked screen before the update.

[0135] In actual online demonstration scenarios, there may be situations where the physical object to be displayed temporarily leaves the field of vision or cannot be captured by the camera. If the markers for the physical object to be displayed are retained in the marked screen, the content displayed in the marked screen may not match the actual scene.

[0136] In step S34 of this embodiment, after displaying the marking screen, the electronic device detects in real time whether it has captured an image of the object to be displayed. If it detects that the electronic device has not captured an image, it can be said that the user does not have a need to continue demonstrating the object to be displayed at the detection time point. Then the marking screen is automatically updated, and the object to be displayed and the marking content applied to it are removed from the marking screen.

[0137] In step S35 of this embodiment, if the marking screen is updated so that only the original background information of the screen is displayed, when the electronic device re-collects the object to be displayed, it indicates that the user needs to demonstrate the object to be displayed again at the detection time point. Then the electronic device will automatically restore the object to be displayed in the marking screen and the marking content thereon.

[0138] In this embodiment of the disclosure, the restored marked screen is generated based on the currently acquired image of the object to be displayed and the marking information corresponding to the marked screen before the update. If the position of the currently acquired image of the object to be displayed changes, the marking position needs to be adaptively adjusted according to the position of the currently acquired image of the object to be displayed when marking it based on the marking information corresponding to the marked screen before the update.

[0139] It is understood that in this embodiment of the disclosure, by automatically detecting whether the image of the object to be displayed has been captured, and updating or restoring the screen content according to the detection result, the marked screen content is kept consistent with the actual scene. After the object to be displayed is recaptured, the marking of the object to be displayed is restored. On the one hand, this can improve the real-time performance of the user's display process, and on the other hand, restoring the marking can also improve the intelligence of the display.

[0140] In some embodiments, the marking information includes a marking style and a marking position. Marking the object to be displayed in the real-time image based on the marking information and generating a marked image includes:

[0141] In response to detecting a size change of the object to be displayed in the real-time image, the position of the marker information is adjusted based on the changed size of the object to be displayed;

[0142] Based on the marking style indicated by the marking information, the object to be displayed in the real-time image after adjustment of size is marked according to the adjusted marking position, and the marked image is generated.

[0143] During actual demonstrations, the size of the object being displayed projected onto the live screen may change in real time. If the electronic device cannot automatically adapt to this change, it will result in the marker position being off-center or the marking effect being distorted.

[0144] In this embodiment, the size change of the object to be displayed may be caused by a change in the shooting distance (such as the movement of the camera lens of the electronic device or the movement of the object to be displayed). For example, if the object to be displayed is relatively close to the electronic device, the size change is an increase; if the object to be displayed is relatively far from the electronic device, the size change is a decrease. The size change of the object to be displayed can be determined by calculating the enlargement or reduction ratio of the object to be displayed based on the size of the object before and after the size adjustment in the real-time image. After detecting the size change of the object to be displayed, the electronic device adaptively adjusts the display position based on the display position of the marking pattern on the object to be displayed. By calculating the direction and distance that the marking position needs to move, the marking position is adjusted to remain unchanged relative to the new size of the object to be displayed.

[0145] In this embodiment of the disclosure, the electronic device marks the resized object to be displayed based on the adjusted mark position and the mark style indicated by the mark information, and generates a mark screen. In the mark screen, the mark style can be accurately attached to the resized object to be displayed, and the mark style remains consistent with the original settings.

[0146] It is understood that the embodiments of this disclosure automatically adjust the marking position of the marking style on the real-time screen by detecting changes in the size of the object to be displayed in real time, and accurately marking the object to be displayed after the size change, thereby improving the accuracy and consistency of the marking effect of the object to be displayed in the marking screen.

[0147] Figure 4 This is a block diagram illustrating an information display device according to an exemplary embodiment. For example... Figure 4 As shown, the device mainly includes:

[0148] The acquisition module 401 is configured to acquire a real-time image of the object to be displayed and the marking information of the object to be displayed;

[0149] The marking module 402 is configured to mark the physical object to be displayed in the real-time image based on the marking information and generate a marked image;

[0150] The display module 403 is configured to display the marked screen online in real time.

[0151] In some embodiments, the acquisition module 401 is further configured to acquire images of the object to be displayed in real time, and display a real-time screen including the object to be displayed based on the images; and acquire the marking information of the object to be displayed in response to detecting a touch operation on the real-time screen.

[0152] In some embodiments, the acquisition module 401 is further configured to acquire marking information associated with the first touch operation in response to detecting a first touch operation on the area where the object to be displayed is located in the real-time screen; and / or, the real-time screen further includes a control for indicating the marking method, and acquire marking information associated with the second touch operation in response to detecting a second touch operation on the control in the real-time screen.

[0153] In some embodiments, the acquisition module 401 is further configured to, in response to detecting a first touch operation on the area where the object to be displayed is located in the real-time screen, acquire the mark position indicated by the first touch operation; in response to detecting a second touch operation on the control in the real-time screen, acquire the mark style indicated by the control operated by the second touch operation; and acquire the mark information based on the mark position and the mark style.

[0154] In some embodiments, the acquisition module 401 is further configured to, in response to detecting a sliding operation on the area where the object to be displayed is located in the real-time image and a closed sliding trajectory exists, determine the area indicated by the closed sliding trajectory in the real-time image as the marked position indicated by the first touch operation.

[0155] In some embodiments, the marking module 402 is further configured to, when the real-time screen also includes a control for indicating the marking method, mark the object to be displayed in the real-time screen based on the marking information indicated by the control operated by the second touch operation and generate a marked screen that does not include the control.

[0156] In some embodiments, the controls in the live feed include at least one of the following:

[0157] The first control used to indicate the highlighted content;

[0158] A second control used to indicate where to add comments;

[0159] A third control used to indicate the line drawing;

[0160] A fourth control used to indicate scaling.

[0161] In some embodiments, the apparatus further includes:

[0162] The update module is configured to update the online display screen to a screen that does not include the object to be displayed and its markings in response to the failure to capture an image of the object to be displayed during the real-time online display of the marked screen.

[0163] The recovery module is configured to recover the display of the marked screen after acquiring an image of the physical object to be displayed in response to an update of the online display screen; wherein the recovered marked screen is generated based on the currently acquired image of the physical object to be displayed and the marking information corresponding to the marked screen before the update.

[0164] In some embodiments, the tagging information includes tagging style and tagging position.

[0165] The marking module 402 is further configured to, in response to detecting a size change of the object to be displayed in the real-time image, adjust the marking position of the marking information based on the changed size of the object to be displayed; mark the object to be displayed in the real-time image after size adjustment according to the marking style indicated by the marking information and the adjusted marking position, and generate the marking image.

[0166] In some embodiments, the display module 403 is further configured to send the marked image to a receiving device; wherein the marked image is used to display in real time on the receiving device. Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

[0167] Figure 5 This is a structural block diagram of an electronic device 500 according to an exemplary embodiment. For example, device 500 may be the aforementioned terminal device, or a cloud device, relay device, etc. connected to the terminal device.

[0168] Reference Figure 5 The device 500 may include one or more of the following components: processing component 502, memory 504, power supply component 506, multimedia component 508, audio component 510, input / output (I / O) interface 512, sensor component 514, and communication component 516.

[0169] Processing component 502 typically controls the overall operation of device 500, such as operations associated with at least one of display, telephone call, data communication, camera operation, and recording operation. Processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.

[0170] Memory 504 is configured to store various types of data to support operation on device 500. Examples of such data include at least one of the following: instructions for any application or method operating on device 500, contact data, phonebook data, messages, pictures, and videos. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0171] Power supply component 506 provides power to various components of device 500. Power supply component 506 may include at least one of the following: a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 500.

[0172] Multimedia component 508 includes a screen that provides an output interface between device 500 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0173] Audio component 510 is configured to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) configured to receive external audio signals when device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.

[0174] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, and buttons. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0175] Sensor assembly 514 includes one or more sensors for providing state assessments of various aspects of device 500. For example, sensor assembly 514 may detect the on / off state of device 500, the relative positioning of components such as the display and keypad of device 500, changes in the position of device 500 or one of its components, the presence or absence of user contact with device 500, orientation or acceleration / deceleration of device 500, and temperature changes of device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include, but is not limited to, at least one of the following: an accelerometer, a gyroscope, a magnetometer, a pressure sensor, and a temperature sensor.

[0176] Communication component 516 is configured to facilitate wired or wireless communication between device 500 and other devices. Device 500 can access wireless networks based on communication standards, such as Wi-Fi, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0177] In an exemplary embodiment, device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0178] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including executable instructions or a computer program, which can be executed by the processor 520 of the device 500 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0179] A non-transitory computer-readable storage medium, wherein when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is able to perform any of the information display methods described above in the embodiments of this disclosure.

[0180] This disclosure provides a computer program product comprising a computer program or executable instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or executable instructions from the computer-readable storage medium and executes the computer program or executable instructions, causing the computer device to perform any of the information display methods described above in this disclosure.

[0181] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0182] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An information display method, characterized in that, The method includes: Acquire real-time images of the object to be displayed, as well as the marking information of the object to be displayed; The objects to be displayed in the real-time image are marked based on the marking information, and a marked image is generated. The marked screen is displayed online in real time.

2. The method according to claim 1, characterized in that, The acquisition includes real-time images of the object to be displayed and the marking information of the object to be displayed, including: Real-time acquisition of images of the object to be displayed, and display of a real-time screen including the object to be displayed based on the images; In response to the detection of a touch operation on the real-time screen, the marking information of the object to be displayed is obtained.

3. The method according to claim 2, characterized in that, The step of obtaining the marking information of the object to be displayed in response to detecting a touch operation on the real-time screen includes: In response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image, acquire marker information associated with the first touch operation; and / or, The real-time screen also includes a control for indicating the marking method, and in response to detecting a second touch operation on the control in the real-time screen, the marking information associated with the second touch operation is obtained.

4. The method according to claim 3, characterized in that, In response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image, and in response to detecting a second touch operation on a control in the real-time image, the acquisition of marker information associated with the first touch operation and the second touch operation includes: In response to detecting a first touch operation on the area where the object to be displayed is located in the real-time image, the marked position indicated by the first touch operation is obtained; In response to detecting a second touch operation on the control in the real-time screen, the mark style indicated by the control operated by the second touch operation is obtained; The marker information is obtained based on the marker position and the marker style.

5. The method according to claim 4, characterized in that, The step of responding to detecting a first touch operation on the area where the object to be displayed is located in the real-time image, and obtaining the marked position indicated by the first touch operation, includes: In response to detecting a sliding operation on the area where the object to be displayed is located in the real-time screen, and the existence of a closed sliding trajectory, the area indicated by the closed sliding trajectory in the real-time screen is determined as the marked position indicated by the first touch operation.

6. The method according to claim 3, characterized in that, The step of marking the object to be displayed in the real-time image based on the marking information and generating a marked image includes: If the real-time screen also includes a control for indicating the marking method, the object to be displayed in the real-time screen is marked based on the marking information indicated by the control operated by the second touch operation, and a marked screen without the control is generated.

7. The method according to claim 3, characterized in that, The controls in the real-time display include at least one of the following: The first control used to indicate the highlighted content; A second control used to indicate where to add comments; A third control used to indicate the line drawing; A fourth control used to indicate scaling.

8. The method according to claim 2, characterized in that, The method further includes: in response to the failure to capture an image of the object to be displayed during the real-time online display of the marked screen, updating the online display screen to a screen that does not include the object to be displayed and the markings on the object to be displayed; In response to updating the online display screen, an image of the physical object to be displayed is captured, and the display of the marked screen is restored; wherein, the restored marked screen is generated based on the currently captured image of the physical object to be displayed and the marking information corresponding to the marked screen before the update.

9. The method according to any one of claims 1 to 8, characterized in that, The marking information includes marking style and marking position. Marking the object to be displayed in the real-time image based on the marking information and generating a marked image includes: In response to detecting a size change of the object to be displayed in the real-time image, the position of the marker information is adjusted based on the changed size of the object to be displayed; Based on the marking style indicated by the marking information, the object to be displayed in the real-time image after adjustment of size is marked according to the adjusted marking position, and the marked image is generated.

10. The method according to claim 1, characterized in that, The real-time online display of the marked screen includes: The marked image is sent to the receiving device; wherein the marked image is used to be displayed in real time on the receiving device.

11. An information display device, characterized in that, The device includes: The acquisition module is configured to acquire a real-time image of the object to be displayed and the marking information of the object to be displayed; The marking module is configured to mark the physical object to be displayed in the real-time image based on the marking information and generate a marked image; The display module is configured to display the marked screen online in real time.

12. An electronic device, characterized in that, include: processor; Memory used to store computer programs or instructions; The processor executes the computer program or instructions to implement the steps of the method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium storing a computer program or instructions, characterized in that, When the computer program or instructions in the storage medium are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 10.

Images