Content display method, content display device, storage medium, and electronic device

Abstract

The present disclosure relates to the technical field of display, and in particular to a content display method, a content display device, a storage medium and an electronic device. The content display method comprises: determining a visible area corresponding to a viewer in a display device according to spatial position information of the viewer and a visual angle; determining a writing area of a writer in the display device, and determining a display area outside the writing area in the display device as a candidate display area; determining a target display area corresponding to the writing area in the candidate display area for the viewer according to the visible area; and displaying writing content of the writing area in the target display area. The method can improve the listening effect of the viewer to a certain extent.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of display, and in particular, to a content display method, a content display device, a computer readable storage medium and an electronic device. BACKGROUND

[0002] At present, more and more display devices have writing function. However, due to the height, arm length and other factors of the writer, the writer can only write in part of the area of the display device; at the same time, the viewer often watches in a fixed position.

[0003] Further, other viewers in front of the viewer's seat and writers near the display device and other factors can interfere with the viewer watching the writing content, thereby reducing the listening effect of the viewer and affecting the user experience.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The present disclosure provides a content display method, a content display device, an electronic device and a computer readable storage medium, thereby at least to some extent ensuring that the writing content is smoothly conveyed to the viewer and improving the listening effect of the viewer.

[0006] According to one aspect of the present disclosure, a content display method is provided, comprising: determining a visible area in a display device corresponding to a viewer according to spatial position information and a visual angle of the viewer; determining a writing area of a writer in the display device and determining a display area outside the writing area in the display device as a candidate display area; determining a target display area corresponding to the writing area in the candidate display area for the viewer according to the visible area; and displaying writing content of the writing area in the target display area.

[0007] In an exemplary embodiment of the present disclosure, the method further comprises: acquiring a depth image of a scene, determining point cloud data of the scene according to the depth image, and creating a three-dimensional space coordinate system according to the point cloud data; collecting pose information of the viewer and determining the spatial position information of the viewer in the three-dimensional space coordinate system according to the pose information of the viewer.

[0008] In an example embodiment of the present disclosure, the determining the visible area corresponding to the viewer in the display device comprises: determining the visual field of the viewer according to the spatial position information of the viewer and the visible angle; determining the projection area of the visual field of the viewer on the display device; and determining the projection area as the visible area corresponding to the viewer.

[0009] In an example embodiment of the present disclosure, the determining the visible area corresponding to the viewer in the display device comprises: determining the visual field of the viewer according to the spatial position information of the viewer and the visible angle; determining the first projection area of the visual field of the viewer on the display device; determining the second projection area of the line-of-sight obstacle between the viewer and the display device on the display device; and determining the area in the first projection area that does not overlap with the second projection area as the visible area corresponding to the viewer.

[0010] In an example embodiment of the present disclosure, the determining the writing area of the writer on the display device comprises: obtaining the writing content of the writer on the display device; and determining the minimum rectangular area that can enclose the writing content as the writing area.

[0011] In an example embodiment of the present disclosure, the determining the target display area corresponding to the writing area for the viewer in the candidate display area comprises: dividing the candidate display area into a plurality of sub-candidate display areas; determining the number N of sub-candidate display areas that need to be contained by the target display area according to the size of the target display area; evaluating each of the sub-candidate display areas according to the visible area; and selecting the N adjacent sub-candidate display areas as the target display area according to the evaluation result of the evaluation.

[0012] In an example embodiment of the present disclosure, the method further comprises: determining the size of the target display area according to the size of the writing area; wherein the size of the target display area is not less than the size of the writing area, and the size of the target display area is an integer multiple of the size of the sub-candidate display area.

[0013] In an example embodiment of the present disclosure, the determining the size of the target display area according to the size of the writing area comprises: magnifying the writing area by a preset multiple; and determining the size of the target display area according to the size of the magnified writing area.

[0014] In an example embodiment of the present disclosure, the number of viewers is a plurality; and the evaluating each of the sub-candidate display regions according to the visual region includes: determining a sub-candidate display region corresponding to each of the viewers according to the visual region corresponding to each of the viewers; determining, for each of the sub-candidate display regions, a number of occurrences of the sub-candidate display region in the sub-candidate display regions corresponding to each of the viewers; and determining an evaluation result of the sub-candidate display region according to the number of occurrences.

[0015] In an example embodiment of the present disclosure, the method further includes: determining the viewer according to a viewer selection instruction.

[0016] According to an aspect of the present disclosure, there is provided a content display device, including: a visual region determining module configured to determine a visual region in a display device corresponding to a viewer according to spatial position information of the viewer and a visual angle; a first display region determining module configured to determine a writing region of a writer in the display device, and determine a display region outside the writing region in the display device as a candidate display region; a second display region determining module configured to determine a target display region corresponding to the writing region in the candidate display region for the viewer according to the visual region; and a content display module configured to display writing content of the writing region in the target display region.

[0017] According to an aspect of the present disclosure, there is provided an electronic device, including: a processor; and a memory storing one or more programs, when the one or more programs are executed by the processor, causing the processor to implement the method provided by some aspects of the present disclosure.

[0018] According to an aspect of the present disclosure, there is provided a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the method provided by some aspects of the present disclosure.

[0019] It should be understood that the general description above and the following detailed description are only exemplary and explanatory, and are not limiting of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure. It is readily apparent to one of ordinary skill in the art that the accompanying drawings are merely exemplary of embodiments of the present disclosure and are therefore not limiting of the scope of the present disclosure as set forth in the claims.

[0021] Figure 1An application scenario of the content display method in the embodiment of the present disclosure is shown.

[0022] Figure 2 A flowchart of the content display method in the embodiment of the present disclosure is shown.

[0023] Figure 3 A flowchart of determining the spatial position information of the viewer in the embodiment of the present disclosure is shown.

[0024] Figure 4 A three-dimensional space coordinate system in the embodiment of the present disclosure is shown.

[0025] Figure 5 A determination of the pose information corresponding to the viewer in the embodiment of the present disclosure is shown.

[0026] Figure 6 A flowchart of determining the visible area of the viewer in the embodiment of the present disclosure is shown.

[0027] Figure 7 An application scenario of the content display method in the embodiment of the present disclosure is shown.

[0028] Figure 8 A horizontal direction visual angle of the human eye in the embodiment of the present disclosure is shown.

[0029] Figure 9 A vertical direction visual angle of the human eye in the embodiment of the present disclosure is shown.

[0030] Figure 10 A visible area of the viewer in the embodiment of the present disclosure is shown.

[0031] Figure 11 An application scenario of the content display method in the embodiment of the present disclosure is shown.

[0032] Figure 12 An application scenario of the content display method in the embodiment of the present disclosure is shown.

[0033] Figure 13 A flowchart of determining the visible area of the viewer in the embodiment of the present disclosure is shown.

[0034] Figure 14 A visible area of the viewer in the embodiment of the present disclosure is shown.

[0035] Figure 15 A display device in the embodiment of the present disclosure is shown.

[0036] Figure 16 A flowchart of determining the target display area in the embodiment of the present disclosure is shown.

[0037] Figure 17 A flowchart of determining the evaluation result of the sub-candidate display area in the embodiment of the present disclosure is shown.

[0038] Figure 18 A schematic diagram of a display device in the embodiment of the present disclosure is shown.

[0039] Figure 19 A schematic diagram of a module of the content display device in the embodiment of the present disclosure is shown.

[0040] Figure 20 A structural schematic diagram of a computer system of an electronic device for implementing the embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0041] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art. The described features, structures, or characteristics can be combined in one or more implementations.

[0042] In addition, the accompanying drawings are included to provide a thorough understanding of the present disclosure and are not intended to be exhaustive or to limit the present disclosure to a precise explanation. Identical reference numerals in the figures denote the same or similar elements and no repeated description will be given. Some block diagrams in the drawings are functional entities that do not necessarily have to correspond to physically or logically independent entities. These functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0043] It should be noted that in the present disclosure, the terms "comprise", "configured to", and "provided with" are used to mean that the inclusion of the listed elements / components / etc. is not exclusive, and that additional elements / components / etc. can be present.

[0044] Figure 1A schematic diagram showing an exemplary application scenario of the content display method to which the embodiments of the present disclosure can be applied is shown. In the application scenario, at least a display device 100, a writer 200 and a viewer 300 are included. In the present example embodiment, the display device 100 can be an electronic whiteboard. The electronic whiteboard is an interactive display device with a large-size display screen. Through a specific stylus, writing operations or other operations such as editing, annotation and saving can be performed on the electronic whiteboard. The electronic whiteboard can be applied in a conference presentation or teaching environment for a user to demonstrate or explain. Of course, in other exemplary embodiments of the present disclosure, the display device 100 can also be other display devices capable of realizing writing and display functions, and the present exemplary embodiment is not limited thereto. In addition, the display device 100 can also have an operation processing mechanism, for example, including a built-in processor for realizing operation processing function or an external server for realizing operation processing function. In the application scenario, the viewer 300 can be a conference participant, a student or an audience, and the writer 200 can be a conference participant, a lecturer or a demonstrator. The writer 200 can write content to be displayed on the display device 100, such as key points, equations, flowcharts, schematic diagrams or other content. The viewer 300 can track the handwriting of the speaker and view the written content.

[0045] With reference to Figure 1 As shown, due to the height, arm length and other factors of the writer 200, the writer 200 can only write in a part of the display area of the display device 100. Meanwhile, the viewer 300 is usually at a fixed position (such as the periphery of the conference table 400) to watch. In addition, other viewers in front of the viewer's seat and the writer 200 near the display device 100 can also interfere with the viewer's viewing of the written content, thereby reducing the listening effect of the viewer and affecting the user experience.

[0046] With reference to Figure 2 To solve at least part of the above problems, in the present example embodiment, a content display method is provided. The method can be executed by an operation processing mechanism built-in or externally connected to the display device. The content display method provided in the present exemplary embodiment can include the following steps:

[0047] In step S210, a visible area corresponding to the viewer in the display device is determined according to the spatial position information and the visual angle of the viewer.

[0048] In step S220, a writing area of the writer in the display device is determined, and a display area outside the writing area in the display device is determined as a candidate display area.

[0049] Step S230: Based on the visible area, determine a target display area corresponding to the writing area for the viewer in the candidate display area.

[0050] Step S240: Display the written content of the writing area within the target display area.

[0051] Based on the content display method provided by the exemplary embodiments of this disclosure, on the one hand, since viewers can directly view the writing area (such as...) Figure 1 In addition to the writing area 120 shown, a target display area (such as...) can also be used. Figure 1 The target display area 110 is used to view the written content. Therefore, even if the writing area is obstructed, it will not affect the viewer's ability to see the written content, improving the efficiency of information transmission and enhancing the viewer's listening experience. Furthermore, since the target display area is determined by combining the viewer's spatial position information and viewing angle, it is more convenient and suitable for the viewer to view the target display area, thereby improving the viewer's viewing experience. Moreover, even if the writer is limited to writing in only a small area of ​​the display device due to height or other factors, the written content can still be smoothly conveyed to the viewer. Therefore, the writer can write freely in various areas of the display device without restriction. The viewer's spatial position information includes, for example, the viewer's current spatial location and the spatial location of certain specific features of the viewer, such as the spatial location of the viewer's head or eyes.

[0052] The following will describe in more detail the various steps of the content display method in this exemplary embodiment with reference to the accompanying drawings and embodiments.

[0053] In step S210, the visible area in the display device corresponding to the viewer is determined based on the viewer's spatial location information and viewing angle.

[0054] In this exemplary embodiment, the spatial location information of the viewer can be determined in various ways depending on the application scenario. For example, if the application scenario is a place where the viewer's position is relatively fixed (e.g., the seat and viewer's position remain unchanged for a long time), the viewer's spatial location information can be pre-marked, and then directly obtained when needed. As another example, if the application scenario is a place where the viewer's position may change, such as a conference room or classroom, the viewer's spatial location information can be obtained in real time or periodically; alternatively, the viewer's spatial location information can be obtained only after a change in the viewer's spatial location information is detected. This exemplary embodiment does not impose any special limitations on this approach.

[0055] For example, refer to Figure 3As shown, the spatial position information of the viewer can be determined through the following steps S310 and S320.

[0056] In step S310, a depth image of the scene is acquired, point cloud data of the scene is determined according to the depth image, and a three-dimensional spatial coordinate system is created according to the point cloud data.

[0057] In the example embodiment, a three-dimensional scanning device such as a laser radar, a stereo camera, a time-of-flight camera, etc. can be used to scan the scene, thereby acquiring images and corresponding depth information of each object (such as a display device, a conference table, a chair, a wall, a podium, etc.) in the scene. Further, feature information can be obtained by performing feature extraction on the images of each object in the scene, and visual tracking and motion estimation are performed using the feature information to obtain intermediate results. Then, local point cloud data is obtained using the depth information corresponding to the images of each object and the internal parameters of the three-dimensional scanning device. Finally, global point cloud data is generated using the above intermediate results and the local point cloud data, and a three-dimensional spatial coordinate system corresponding to the current scene is constructed based on the global point cloud data. For example, the three-dimensional spatial coordinate system corresponding to the current scene can be in the form shown in Figure 4

[0058] In step S320, the pose information of the viewer is acquired, and the spatial position information of the viewer in the three-dimensional spatial coordinate system is determined according to the pose information of the viewer.

[0059] In the example embodiment, a three-dimensional scanning device such as a laser radar, a stereo camera, a time-of-flight camera, etc. can be used to scan the viewer, thereby acquiring images and corresponding depth information of each viewer. Based on the images and corresponding depth information of each viewer, the pose information of the viewer can be acquired. Of course, in some example embodiments, only the images of each viewer can be acquired, and then a polygon or triangle model, etc. is converted from the images through surface reconstruction, etc. to further acquire the pose information of the viewer. After the pose information of the viewer is acquired, the position of the viewer in the above three-dimensional spatial coordinate system can be determined in combination with an algorithm such as ICP (Iterative Closest Point). As shown in Figure 5

[0060] As shown in Figure 6 ​​As shown, in the present example embodiment, the visual area in the display device corresponding to the viewer can be determined by the following steps S610 to S630. Wherein:

[0061] Step S610, according to the spatial position information of the viewer and the visual angle, the visual field of the viewer is determined.

[0062] The angle of the object in front of the human eye that can be seen at the same time is called the visual angle. The visual angle can be divided into the following categories from small to large:

[0063] Monocular visual angle one: the visual angle when one eye looks straight ahead, the eyeball cannot rotate, and the head cannot rotate forward. For example, the right eye, the upper visual angle is usually 50°, the lower visual angle is usually 70°, the left visual angle is usually 56°, and the right visual angle is usually 100°.

[0064] Monocular visual angle two: the visual angle when one eye looks straight ahead, the eyeball cannot rotate, and the head can rotate. In order to more completely represent the visual range of the eyeball, the occlusion of the eye socket and nose can be removed; for example, the right eye, the upper visual angle is usually 55°, the lower visual angle is usually 75°, the left visual angle is usually 60°, and the right visual angle is usually 100°.

[0065] Binocular visual angle one: the visual angle when both eyes look straight ahead, the eyeball cannot rotate, and the head cannot rotate forward. The upper and lower visual angles are usually 120° in total, and the left and right visual angles are usually 200° in total.

[0066] Binocular visual angle two: the visual angle when both eyes look straight ahead, the eyeball cannot rotate, and the head can rotate. The upper and lower visual angles are usually 130° in total, and the left and right visual angles are usually 200° in total.

[0067] Monocular visual angle three: the visual angle when the eyeball can rotate, and the head cannot rotate forward. For example, the right eye, the upper visual angle is usually 70°, the lower visual angle is usually 80°, the left visual angle is usually 65°, and the right visual angle is usually 115°.

[0068] Binocular visual angle three: the visual angle when the eyeball can rotate, and the head cannot rotate forward. The upper and lower visual angles are usually 150° in total, and the left and right visual angles are usually 230° in total.

[0069] Reference Figure 7 and Figure 8As shown, taking the conference scenario as an example, in the conference, the viewer usually watches the board in a comfortable posture, that is, the eyeball can rotate and the head basically does not rotate; the maximum horizontal visual angle of the viewer's single eye is up to 180°, and the maximum horizontal visual angle of the viewer's double eyes is up to 230°. However, within the range watched by the human eye, only the objects in the middle 124° visual angle (such as the middle region located between X1 and X2 in the figure) have stereoscopic effect; and the human eye vision is usually sensitive within 10°, can correctly identify information within 10°-20°, and is more sensitive to dynamic things within 20°-30°. When the vertical visual angle of the image is 20° and the horizontal visual angle is 36°, the viewer usually has good visual presence, and does not get tired because of frequent rotation of the eyeball.

[0070] Based on the above characteristics of human eye vision, referring to Figure 8 As shown, in the example embodiment, according to the above “double eye visual angle three”, ∠a=∠b=115° is selected as the horizontal visual angle of the left eye and the right eye, and the double eye coincident visual field ∠c=124° is selected to construct the eye model of the viewer. Taking the intersection A of ∠a and ∠b as the vertex, the midpoint B of the double eye line is connected to form a ray BA, and the ray BA is taken as the 0° line, and 62° angles (that is, ∠e and ∠f shown in the figure) are made on the left and right respectively, so that the ray AC and the ray AD are within the horizontal visual field of 124°. Similarly, referring to Figure 9 As shown, in the vertical direction, the upper angle 75° and the lower angle 75° are within the vertical visual field.

[0071] Step S620, determining the projection area of the visual field of the viewer on the display device.

[0072] Step S630, determining the projection area as the visual area corresponding to the viewer.

[0073] Referring to Figure 10 As shown, in the eye model of the viewer, taking the vertex A as the equivalent view point, in the case of no occlusion, the projection area of the horizontal visual field and the vertical visual field on the whiteboard plane is SQ; and then, the projection area SQ can be determined as the visual area corresponding to the viewer.

[0074] Referring to Figure 11 As shown, in the actual scene, there is often a line-of-sight obstacle 500 between the viewer 300 and the display device 100, such as the body of the speaker, the body of other viewers, tables and chairs, computers, etc. Referring to Figure 12As shown, if there is no line-of-sight obstacle, a straight line can be determined between the three-dimensional pixel point of the viewer and the display unit of the display device; if there is a line-of-sight obstacle 500, there will be an intersection point on the straight line between the three-dimensional pixel point of the viewer and the display unit of the display device; and then, it can be determined whether there is a line-of-sight obstacle between the viewer 300 and the display device 100, and the corresponding visible area of the viewer can be determined according to the line-of-sight obstacle.

[0075] Specifically, referring to Figure 13 As shown, in the example embodiment, the corresponding visible area of the viewer in the display device can be determined by the following steps S1310 to S1340. Among them:

[0076] Step S1310, determining the field of view of the viewer according to the spatial position information of the viewer and the visible angle. This step is similar to the above-mentioned step S610, and thus will not be repeated.

[0077] Step S1320, determining the first projection area SQ of the field of view of the viewer on the display device. This step is similar to the above-mentioned step S620, and thus will not be repeated.

[0078] Step S1330, determining the second projection area of the line-of-sight obstacle located between the viewer and the display device on the display device. For example, similar to the above-mentioned step S320, in the example embodiment, the image and the corresponding depth information of the line-of-sight obstacle can be collected, and the spatial position information of the line-of-sight obstacle in the three-dimensional space coordinate system can be determined based on the image and the corresponding depth information of the line-of-sight obstacle. Further, taking the equivalent visible point of the viewer as a virtual point light source, the second projection area SZ of the line-of-sight obstacle 500 on the display device 100 is calculated.

[0079] Step S1340, determining the area in the first projection area which does not overlap with the second projection area as the corresponding visible area of the viewer. For example, referring to Figure 14 As shown, the corresponding visible area of the viewer is the first projection area SQ minus the second projection area SZ. However, in some example embodiments, if the second projection area is not completely located in the first projection area, the corresponding visible area of the viewer is the first projection area SQ minus the area of the second projection area which overlaps with the first projection area; this example embodiment does not specially limit this.

[0080] Further, in order to quantify the size of the corresponding visible area of the viewer, in the example embodiment, the display device can be divided into a plurality of arrayed display units; each display unit contains the same number of sub-pixels. As Figure 15As shown, taking an example that each display unit contains one sub-pixel, the display device can be divided into m rows and n columns of display units according to the resolution of the display device. For example, the resolution of a certain 65-inch electronic whiteboard is 4320x2160, and the screen size is 1439mmx809.3mm, so the length of each display unit is 1439mm / 4320=0.333mm, the width is 809.3mm / 2160=0.3746mm, and the area size is 0.333mmx0.3746mm. Each display unit can be named P 11 , P 12 , …, P mn respectively according to the row and column in which it is located. Then the entire display device can be represented as:

[0081]

[0082] Correspondingly, in an example embodiment, Figure 14 the visual area S corresponding to the viewer as shown can be represented as:

[0083]

[0084]

[0085] In step S220, the writing area of the writer on the display device is determined, and the display area outside the writing area in the display device is determined as a candidate display area.

[0086] In this example embodiment, the writing content of the writer on the display device can be obtained first. For example, the writing traces of the writer are detected, and all the writing traces are taken as the writing content. Then, the smallest rectangular area that can enclose the writing content is determined as the writing area. Of course, in more example embodiments, other smallest convex polygonal areas (such as trapezoids, regular hexagons, etc.) or circular areas that can enclose the writing content can also be determined as the writing area, which also falls within the protection scope of the present disclosure.

[0087] In some example embodiments, the writing area can also be a fixed area in the display device, so that the area can be obtained directly without the need for re-determination. In some example embodiments, the writing area can also be determined with the assistance of the writer. For example, the writer can enclose certain areas (with or without writing content) in the display device as the above-mentioned writing area. The specific manner of determining the writing area in this example embodiment is not specially limited.

[0088] In step S230, according to the visual area, a target display area corresponding to the writing area is determined for the viewer in the candidate display area.

[0089] Reference Figure 16 As shown in the figure, in the example embodiment, the target display region corresponding to the writing region can be determined for the viewer in the candidate display region through the following steps S1610 to S1640. Wherein:

[0090] Step S1610, divide the candidate display region into a plurality of sub-candidate display regions.

[0091] In the example embodiment, the display device can be divided into a plurality of sub-regions in advance, and then the sub-regions contained in the candidate display region are determined as the sub-candidate display regions contained in the candidate display region after the candidate display region is determined; that is, the division result of the candidate display region is obtained based on the division result of the display device. The candidate display region can also be divided after the candidate display region is determined to obtain the plurality of sub-candidate display regions; this is not specially limited in the example embodiment.

[0092] For example, in the example embodiment, the display device can be divided according to a pre-set division rule; for example, each sub-region can contain 240x120 sub-pixels, so the display device with a resolution of 4320x2160 will be divided into 18x18=324 sub-regions; and then after the candidate display region is determined, each sub-region within the range of the candidate display region can be determined as the sub-candidate display region contained in the candidate display region. For example, the specific division rule can be determined according to the properties of the display device, which is also not specially limited in the example embodiment.

[0093] Step S1620, determine the number N of sub-candidate display regions that the target display region needs to contain according to the size of the target display region.

[0094] In the example embodiment, the size of the target display region can be determined according to the size of the writing region first; wherein the size of the target display region is not less than the size of the writing region, and the size of the target display region is an integer multiple of the size of the sub-candidate display region. For example, if the size of the writing region is 400x200 sub-pixels, and the size of the sub-candidate display region is 240x120 sub-pixels; since the size of the target display region is an integer multiple of the size of the sub-candidate display region, the target display region needs to contain 4 sub-candidate display regions, and the size of the target display region is 480x240 sub-pixels.

[0095] In other exemplary embodiments of the present disclosure, the writing area can also be enlarged by a preset multiple, and then the size of the target display area is determined according to the size of the enlarged writing area. For example, the size of the writing area is 400x200 sub-pixel size, in order to make the viewer more clearly identify the writing content, the writing area can be enlarged by X times; wherein X can be a positive number greater than 1, for example, 1.5, 2, 4, etc. If the writing area is enlarged by 2.25 times, the size of the target display area needs to be greater than 600x300 sub-pixel size; since the size of the above-mentioned sub-candidate display area is 240x120 sub-pixel size, and the size of the target display area is an integer multiple of the sub-candidate display area, the target display area needs to contain 9 sub-candidate display areas, and the size of the target display area is 720x360 sub-pixel size.

[0096] It should be noted that in other exemplary embodiments of the present disclosure, the number N of sub-candidate display areas that the target display area needs to contain can also be determined by other methods, and the present exemplary embodiments are not limited thereto.

[0097] Step S1630, evaluating each sub-candidate display area according to the visual area.

[0098] Reference Figure 17 In the present exemplary embodiment, each sub-candidate display area can be evaluated by the following steps S1710 to S1730. Wherein:

[0099] Step S1710, determining the sub-candidate display area corresponding to each viewer according to the visual area corresponding to each viewer. For example, referring to Figure 18As shown, it is assumed that the sub-pixels contained in the visual area determined in step 1340 above are obtained: the corresponding visual area 301 corresponding to viewer A contains sub-candidate display areas 2, 3, 8, 9, the corresponding visual area 302 corresponding to viewer B contains sub-candidate display areas 3, 4, 9, 10, and the corresponding visual area 303 corresponding to viewer C contains sub-candidate display areas 9, 10, 15, 16. In addition, in order to reduce the computational complexity of subsequent evaluation, in the example embodiment, for a sub-candidate display area not completely contained in the visual area of a viewer, it can be considered that the sub-candidate display area is not contained in the visual area of the viewer; for example, the visual area 301 of viewer A does not contain sub-candidate display areas 1, 7, 13, 14, 15, the visual area 302 of viewer B does not contain sub-candidate display areas 5, 11, 15, 16, 17, etc.; but in some example embodiments, for a sub-candidate display area not completely contained in the visual area of a viewer, it can also be counted as a sub-candidate display area contained in the visual area of the viewer according to certain rules (for example, according to an area setting coefficient), which is not specially limited in the example embodiment.

[0100] Step S1720, for each of the sub-candidate display areas, determining the number of occurrences of the sub-candidate display area in the sub-candidate display areas corresponding to each of the viewers. For example, continuing to refer to Figure 18 As shown, sub-candidate display area 3 appears in the visual area 301 of viewer A and the visual area 302 of viewer B respectively, sub-candidate display area 9 appears in the visual area 301 of viewer A, the visual area 302 of viewer B and the visual area 303 of viewer C respectively, sub-candidate display area 10 appears in the visual area 302 of viewer B and the visual area 303 of viewer C respectively, etc. After counting, the number of occurrences of sub-candidate display areas 2, 4, 8, 15, 16 in all the sub-candidate display areas corresponding to the viewers is 1, the number of occurrences of sub-candidate display areas 3, 10 in all the sub-candidate display areas corresponding to the viewers is 2, and the number of occurrences of sub-candidate display area 9 in all the sub-candidate display areas corresponding to the viewers is 3.

[0101] Step S1730, determining the evaluation result of the sub-candidate display area according to the number of occurrences. For example, continuing to refer to Figure 18As shown, the sub-candidate display region 9 appears most frequently, and thus the sub-candidate display region 9 can be assigned a higher evaluation score; correspondingly, the sub-candidate display regions 2, 4, 8, 15, and 16 appear less frequently, and thus these sub-candidate display regions can be assigned lower evaluation scores. In this example embodiment, the number of times a sub-candidate display region appears can be directly set as the evaluation score of the sub-candidate display region; for example, the evaluation score of the sub-candidate display region 9 is 3, and the evaluation scores of the sub-candidate display regions 2, 4, 8, 15, and 16 are 1. However, in other example embodiments of the present disclosure, the number of times a sub-candidate display region appears can be converted into a percentage score or a score out of ten; or, a certain weighting operation can be performed on the number of times a sub-candidate display region appears, and then the evaluation result of the sub-candidate display region can be calculated comprehensively; this is not specially limited in this example embodiment.

[0102] In step S1640, according to the evaluation result of the evaluation, N adjacent sub-candidate display regions are selected as the target display region.

[0103] With reference to the foregoing description, the following is continued. Figure 18 As shown, if the target display region 120 needs to include 3 sub-candidate display regions; in one example embodiment, since the evaluation score of the sub-candidate display region 9 is the highest, the target display region needs to include at least the sub-candidate display region 9; and the 3 adjacent sub-candidate display regions including the sub-candidate display region 9 are respectively the sub-candidate display regions 7, 8, and 9, the sub-candidate display regions 8, 9, and 10, and the sub-candidate display regions 9, 10, and 11. The total evaluation score of the sub-candidate display regions 7, 8, and 9 is 4, the total evaluation score of the sub-candidate display regions 8, 9, and 10 is 6, and the total evaluation score of the sub-candidate display regions 9, 10, and 11 is 5. Therefore, the sub-candidate display regions 8, 9, and 10 can be selected as the target display region.

[0104] However, it is easy for those skilled in the art to understand that, in other example embodiments of the present disclosure, N adjacent sub-candidate display regions can also be selected as the target display region in other manners; for example, N sub-candidate display regions with the highest total evaluation scores can be directly selected as the target display region, and the like. In other example embodiments of the present disclosure, a sub-candidate display region can also be evaluated in other manners; for example, a sub-candidate display region can be evaluated according to the position of the sub-candidate display region in the visual field of each viewer, and the like; all of these fall within the protection scope of the present disclosure.

[0105] In step S240, the writing content of the writing region is displayed in the target display region.

[0106] In this example implementation, after determining the target display area, the written content can be displayed directly in the target display area; alternatively, the written content can be processed before being displayed in the target display area. For example, the handwriting of the written content can be deepened, optimized, or aligned before being displayed in the target display area.

[0107] In some exemplary embodiments of this disclosure, the viewer can also be determined based on viewer selection instructions from the writer or other users. That is, in these exemplary embodiments, not all users of the display device may be the viewer; only the user of the designated display device is the viewer. For example, the viewer can be determined based on the user's professional field, interests, role, etc.; or the viewer can be determined based on the issues to be discussed or the people to be interacted with, as required by the writer or other users. Thus, the method in this patent application primarily determines the target display area for the viewer and displays the written content of the writing area within the target display area. This facilitates interaction between the speaker and the designated viewer, improving user experience and communication efficiency.

[0108] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0109] Furthermore, in this exemplary embodiment, a content display device is also provided. (See reference...) Figure 19 As shown, the content display device 1900 may include a viewable area determination module 1910, a first display area determination module 1920, a second display area determination module 1930, and a content display module 1940. Wherein:

[0110] The visual area determination module 1910 can be configured to determine a visual area corresponding to a viewer in a display device according to spatial position information of the viewer and a visual angle; the first display area determination module 1920 can be configured to determine a writing area of a writer in the display device, and determine a display area outside the writing area in the display device as a candidate display area; the second display area determination module 1930 can be configured to determine a target display area corresponding to the writing area for the viewer in the candidate display area according to the visual area; and the content display module 1940 can be configured to display writing content of the writing area in the target display area.

[0111] The specific details of the modules in the content display device have been described in detail in the corresponding content display method, and thus will not be described here.

[0112] It should be noted that, although several modules or units of the device for action execution are mentioned in the foregoing detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into embodied by multiple modules or units.

[0113] The various component embodiments of the present disclosure can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof.

[0114] In the exemplary embodiments of the present disclosure, an electronic device is also provided, comprising: a processor; a memory configured to store processor-executable instructions; wherein the processor is configured to perform the method described in any of the exemplary embodiments.

[0115] Figure 20 The structural schematic diagram of a computer system of an electronic device for implementing the embodiments of the present disclosure is shown. It should be noted that, Figure 20 The computer system 2000 of the electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0116] As Figure 20As shown, the computer system 2000 includes a central processing unit 2001, which can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and / or any other devices suitable for processing electronic signals. The central processing unit 2001 can execute various suitable actions and processes according to a program stored in a read-only memory 2002 or a program loaded from the storage section 2008 into a random access memory 2003. Various programs and data required for the operation of the system are also stored in the random access memory 2003. The central processing unit 2001, the read-only memory 2002, and the random access memory 2003 are connected to each other through a bus 2004. An input / output interface 2005 is also connected to the bus 2004.

[0117] Connected to the input / output interface 2005 are an input section 2006 including a keyboard, a mouse, etc.; an output section 2007 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 2008 including a hard disk, etc.; and a communication section 2009 including a network interface card such as a local area network (LAN) card, a modem, etc. The communication section 2009 performs communication processing via a network such as the Internet. A drive 2010 is also connected to the input / output interface 2005 as necessary. A removable media 2011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 2010 as necessary, so that a computer program read therefrom is installed in the storage section 2008 as necessary.

[0118] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication section 2009, and / or installed from the removable media 2011. When the computer program is executed by the central processing unit 2001, various functions defined in the apparatus of the present application are executed.

[0119] In exemplary embodiments of the present disclosure, there is also provided a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a computer, causes the computer to perform the method of any one of the above.

[0120] It should be noted that the non-transitory computer-readable storage medium shown in the present disclosure can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used or used in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, radio frequency, etc., or any suitable combination of the above.

[0121] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure following the general principles thereof and including such departures from the present disclosure that come within known use or custom in the art. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A content display method characterized by, The method comprises the following steps: determining a visible area corresponding to the viewer in the display device according to the spatial position information and the visual angle of the viewer; determining a writing area of the writer in the display device, and determining a candidate display area outside the writing area in the display device; determining a target display area corresponding to the writing area for the viewer in the candidate display area according to the visible area; displaying the writing content of the writing area in the target display area; wherein the step of determining the target display area corresponding to the writing area for the viewer in the candidate display area comprises: dividing the candidate display area into a plurality of sub-candidate display areas; determining the number N of sub-candidate display areas that the target display area needs to contain according to the size of the target display area; evaluating each sub-candidate display area according to the visible area, and selecting the adjacent N sub-candidate display areas as the target display area according to the evaluation results of the evaluation; wherein the number of viewers is multiple; the step of evaluating each sub-candidate display area according to the visible area comprises: determining the sub-candidate display area corresponding to each viewer according to the visible area corresponding to each viewer; determining the number of occurrences of each sub-candidate display area in the sub-candidate display areas corresponding to each viewer; determining the evaluation result of each sub-candidate display area according to the number of occurrences.

2. The content display method according to claim 1, characterized by, The step of determining the visible area corresponding to the viewer in the display device comprises: determining the visual field of the viewer according to the spatial position information and the visual angle of the viewer; determining the projection area of the visual field of the viewer on the display device; determining the projection area as the visible area corresponding to the viewer.

3. The content display method of claim 1, wherein, The step of determining the visible area corresponding to the viewer in the display device comprises: determining the visual field of the viewer according to the spatial position information and the visual angle of the viewer; determining the first projection area of the visual field of the viewer on the display device; determining the second projection area of the line-of-sight obstacle between the viewer and the display device on the display device; determining the area in the first projection area that does not overlap with the second projection area as the visible area corresponding to the viewer.

4. The content display method of claim 1, wherein, The step of determining the writing area of the writer in the display device comprises: obtaining the writing content of the writer in the display device; determining the minimum rectangular area that can enclose the writing content as the writing area.

5. The content display method of claim 1, wherein, The method further comprises: determining the size of the target display area according to the size of the writing area; wherein the size of the target display area is not less than the size of the writing area, and the size of the target display area is an integer multiple of the size of the sub-candidate display area.

6. The content display method according to claim 5, characterized by, The step of determining the size of the target display area according to the size of the writing area comprises enlarging the writing area by a preset multiple; determining the size of the target display area according to the size of the enlarged writing area.

7. The content display method according to any one of claims 1 to 6, characterized by, The method further comprises: acquiring a depth image of a scene, determining point cloud data of the scene according to the depth image, and creating a three-dimensional space coordinate system according to the point cloud data; acquiring pose information of the viewer, and determining the spatial position information of the viewer in the three-dimensional space coordinate system according to the pose information of the viewer.

8. The content display method according to claim 7, characterized by, The method further comprises: determining the viewer according to a viewer selection instruction.

9. A content display device, characterized by comprising: comprise: a visible region determining module configured to determine a visible region corresponding to the viewer in a display device according to the spatial position information of the viewer and a visible angle; a first display region determining module configured to determine a writing region of a writer in the display device, and determine a display region outside the writing region in the display device as a candidate display region; a second display region determining module configured to determine a target display region corresponding to the writing region for the viewer in the candidate display region according to the visible region; a content displaying module configured to display writing content of the writing region in the target display region; wherein the second display region determining module is configured to perform: dividing the candidate display region into a plurality of sub-candidate display regions; determining a number N of sub-candidate display regions that need to be contained by the target display region according to a size of the target display region; evaluating each of the sub-candidate display regions according to the visible region, and selecting N adjacent sub-candidate display regions as the target display region according to an evaluation result of the evaluation; the number of the viewers is a plurality; the second display region determining module is further configured to perform: determining a sub-candidate display region corresponding to each of the viewers according to a visible region corresponding to each of the viewers; for each of the sub-candidate display regions, determining a number of occurrences of the sub-candidate display region in the sub-candidate display regions corresponding to the viewers; determining an evaluation result of the sub-candidate display region according to the number of occurrences.

10. An electronic device, comprising: comprise: a processor; and a memory for storing one or more programs, which, when executed by the processor, cause the processor to implement the method of any one of claims 1-8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-8.

Images