Information display method and device, optical equipment and electronic equipment

Abstract

The invention provides an information display method and device, optical equipment and electronic equipment, and the method comprises the steps: determining a target region specified by a user on a printed matter, the target region being a region, corresponding to the orthographic projection of a display region of display equipment on the printed matter, in the printed matter; the display device is located on the printed matter and the display area is located on one side of the display device away from the printed matter; in a pre-constructed database, acquiring extension information corresponding to the printing content in the target area; and displaying the extended information to the user through the display area. Based on the method, when a user wants to know the extended information of some printing contents in the printed matter, the display equipment is arranged above the surface of the printed matter, so that the target area can be appointed, and the extended information corresponding to the printing contents in the target area can be seen in the display area of the display equipment.

Description

Technical Field

[0001] The present application relates to the field of information interaction technology, and in particular to an information display method and device, an optical device, and an electronic device. Background Art

[0002] With the development of network technology and the significant improvement in the convenience of using electronic devices, the popularity of digital information has increased rapidly. People have gradually become accustomed to quickly obtaining the information they need through efficient means, and have developed a strong dependence on electronic devices and digital information.

[0003] In comparison, the weakness of physical printed materials such as traditional books is that the information they can convey is limited to the printed content. When readers want to know more information related to the printed content (such as text, pictures, etc. in the printed material), they can only consult other books or use electronic devices to search and find separately. Not only is the operation cumbersome and time-consuming, but it may not be possible to find accurate information. Summary of the invention

[0004] In view of this, embodiments of the present application provide an information display method and apparatus, an optical device and an electronic device, and a computer-readable storage medium, which aim to enable users to quickly and conveniently obtain extended information related to printed content when viewing printed materials.

[0005] The first aspect of the present application provides an information display method, including: determining a target area specified by a user on a printed material, wherein the target area is an area in the printed material corresponding to the positive projection of a display area of ​​a display device on the printed material, the display device is located on the printed material and the display area is located on a side of the display device away from the printed material; obtaining extended information corresponding to the printed content in the target area from a pre-constructed database; and displaying the extended information to the user through the display area.

[0006] According to the information display method provided by the present application, when the user places the display device above the surface of the printed matter, the area of ​​the printed matter blocked by the display device in the user's line of sight can be determined, and the extended information related to the printed content of the area and / or the nearby area can be displayed on the display device. In this way, when the user wants to know the extended information of certain printed content in the printed matter, the target area can be specified by simply placing the display device between his eyes and the target area where the printed content is located (for example, holding the display device above or in front of the printed matter, or placing the display device on the surface of the printed matter, etc.), and then the extended information corresponding to the printed content in the target area can be seen on the display device.

[0007] In some embodiments, determining a target area specified by a user on a printed matter includes: collecting optical information of the surface of the printed matter through an optical sensor fixedly arranged relative to a display device; identifying a position of the display device on the printed matter based on the optical information; determining an orthographic projection of the display area on the printed matter according to the position of the display device on the printed matter; and taking an area on the printed matter corresponding to the orthographic projection as the target area.

[0008] In order to accurately determine the target area (i.e., the area of ​​the printed material blocked by the display device in the user's line of sight), the printed material can be photographed by an optical sensor that has a fixed relative position relationship with the display device to determine the position of the optical sensor on the surface of the printed material, and then the position of the display device on the printed material can be obtained based on the relative position relationship between the optical sensor and the display device.

[0009] In some embodiments, a code containing position information is printed on the surface of a printed object, and the position of a display device on the printed object is identified based on the optical information, including: acquiring the code contained in the optical information; determining the position of the optical sensor relative to the printed object based on the position information corresponding to the code; and determining the position of the display device on the printed object based on the position of the optical sensor relative to the printed object and the relative position between the optical sensor and the display device.

[0010] The surface of the printed material may be printed with a code containing position information, such as a dot code or other two-dimensional code containing coordinate information. Based on this, the position of the optical sensor on the printed material can be accurately determined according to the code contained in the optical information collected by the optical sensor.

[0011] In some embodiments, collecting optical information of the surface of a printed object by an optical sensor fixedly arranged relative to a display device includes: irradiating auxiliary light to the surface of the printed object; collecting the auxiliary light reflected by the surface of the printed object by the optical sensor to obtain optical information. In this way, by actively irradiating light to the surface of the printed object, the light reflected by the surface of the printed object can be increased, so that the optical sensor can collect more sufficient optical information, thereby being able to more accurately determine the position of the display device.

[0012] In some embodiments, the code is printed by a first material, and the printed content on the printed matter is printed by a second material, wherein the second material does not contain the first material, and the reflectivity of the auxiliary light on the first material is higher than the reflectivity on the second material.

[0013] In order to avoid the interference of printed content on the recognition of the code, the two can be printed separately using different materials, and auxiliary light that is more sensitive to the printed material of the code can be used to illuminate the printed surface, so that the optical sensor can collect more reflected light from the code. For example, the dot matrix code can be printed with carbon-containing ink, and the printed content can be printed with carbon-free ink. The auxiliary light can be infrared light. Therefore, in the optical information, the dot matrix code appears black, and the printed content and the blank area appear white, and the two can be clearly distinguished.

[0014] In some examples, in order to prevent the code on the surface of the printed matter from interfering with the visual effect of the printed content, the code can be an invisible code, printed with invisible materials. For example, the dot matrix code can be printed with fluorescent materials, and the printed content can be printed with non-fluorescent materials, and the auxiliary light can be ultraviolet light. As a result, the brightness of the dot matrix code in the optical information is significantly higher than the brightness of the printed content and the blank area, so that the dot matrix code can be clearly identified; and the naked eye can hardly see the dot matrix code on the surface of the printed matter under natural light, and the user will not be disturbed by the dot matrix code when viewing the printed matter.

[0015] In some embodiments, an optical path extension structure is provided in the incident light path of the optical sensor. When the display device is placed on the surface of a flat printed material, the optical sensor fixedly arranged relative to it may be located very close to the surface of the printed material, for example, it may directly contact the surface of the printed material. At this time, the optical sensor is limited by its own parameters (such as the minimum focal length) and may not be able to capture a clear code, resulting in the inability to achieve positioning. By providing an optical path extension structure in the optical sensor, it is equivalent to lengthening the distance between the lens of the optical sensor and the surface of the printed material, so that the optical sensor can also clearly capture the code at a position very close to the surface of the printed material, thereby accurately determining the position of the display device on the printed material.

[0016] In some embodiments, determining the orthographic projection of the display area on the printed matter based on the position of the display device on the printed matter includes: determining shape parameters of the display area; and determining the orthographic projection of the display area on the printed matter based on the position and shape parameters of the display device on the printed matter.

[0017] Taking into account that the display areas on different display devices may be different (for example, there are differences in parameters such as size, shape, edge position, etc.), after determining the position of the display device on the printed material, the orthographic projection of the display area on the printed material can be determined based on the shape parameters of the display area in the current display device, so that the information display method of the present application can be applicable to various types of display devices and can achieve good display effects on different display devices.

[0018] In some embodiments, determining the target area specified by the user on the printed matter includes: determining the area in the printed matter that overlaps with the orthographic projection of the display area on the printed matter as the target area. Through such a setting, for example, when the display device is placed on the surface of the printed matter, the display area will visually replace the surface of the printed matter it covers, so that the user can see the extended information while accurately understanding the corresponding relationship between the extended information and the printed content, further improving the display effect.

[0019] In some embodiments, the display area is a display function area of ​​a display device or a local area in the display function area. The display area can be set to the entire display function area of ​​the display device to enhance the display effect; or it can be set to an area occupying any range in the display function area, such as one or more windows in a graphical user interface (GUI), so that other areas can display other information at the same time, thereby improving the use efficiency of the display device.

[0020] In some embodiments, displaying extended information to a user via a display area of ​​a display device includes: displaying an image of printed content in the display area, and superimposing extended information on the image, wherein the extended information includes text information and / or image information for helping the user understand the printed content. Alternatively, in some embodiments, the extended information includes an enhanced image of the printed content, wherein the enhanced image has the same or corresponding visual features as the printed content, and the visual features include at least one of a pattern, text, font, and color.

[0021] Through this display method, extended information can be displayed to users while retaining the original visual effect of the printed content. The screen switching method is soft, and the display effect is natural and vivid, which greatly improves the user's visual experience.

[0022] In some embodiments, the information display method further includes: receiving a display adjustment instruction; updating and displaying the extended information according to the display adjustment instruction.

[0023] For example, when the display device moves or rotates on the printed material, it can be regarded as the user inputting a display adjustment instruction, that is, specifying a new target area on the printed material. At this time, the extended information corresponding to the printed content in the new target area can be displayed in the display area. For another example, the user can directly input various display adjustment instructions through the input device, such as screen zooming, layer switching, increase or decrease of the type of extended information, etc. At this time, the corresponding information can be obtained from the database according to the display adjustment instruction to update the extended information displayed in the display area. As a result, the functions and application scenarios of the above-mentioned information display method are further expanded, the convenience and flexibility of operation are improved, and the user experience is greatly improved.

[0024] The second aspect of the present application provides an information display device, including: a determination module, used to determine a target area specified by a user on a printed material, wherein the target area is an area in the printed material corresponding to the positive projection of a display area of ​​a display device on the printed material, the display device is located on the printed material and the display area is located on the side of the display device away from the printed material; an information acquisition module, used to obtain extended information corresponding to the printed content in the target area from a pre-constructed database; and a display module, used to display the extended information to the user through the display area of ​​the display device.

[0025] The third aspect of the present application provides an optical device, including: an optical sensor, which is configured to be fixedly arranged relative to a display device, and is used to collect optical information of the surface of a printed matter and provide the optical information to a processing device, so that the processing device determines the orthographic projection of a display area of ​​the display device on the printed matter based on the optical information, determines the area on the printed matter corresponding to the orthographic projection as a target area specified by a user on the printed matter, and displays extended information corresponding to the printed content in the target area to the user through the display area, wherein the display device is located on the printed matter and the display area is located on the side of the display device away from the printed matter.

[0026] In some embodiments, the optical device further includes: a light emitter, which is used to irradiate auxiliary light onto the surface of the printed object, so that the optical sensor obtains optical information by collecting the auxiliary light reflected from the surface of the printed object.

[0027] The fourth aspect of the present application provides an electronic device, comprising: a memory for storing computer instructions; a processor for executing computer instructions to implement the information display method provided in the first aspect of the present application.

[0028] The fifth aspect of the present application provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by a processor, the information display method provided by the first aspect of the present application is implemented.

[0029] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory and should not be construed as limiting the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail below in conjunction with the accompanying drawings. It should be understood that the drawings constitute a part of the specification and are used together with the embodiments of the present application to explain the present application, and do not constitute a limitation of the present application. Unless otherwise specified, in the drawings, the same symbols and numbers generally represent the same steps or components.

[0031] Figure 1Shown is a schematic diagram of an exemplary information display system provided in an embodiment of the present application.

[0032] Figure 2 Shown is a schematic diagram of another exemplary information display system provided in an embodiment of the present application.

[0033] Figure 3 Shown is a schematic diagram of another exemplary information display system provided in an embodiment of the present application.

[0034] Figure 4 Shown is a schematic diagram of another exemplary information display system provided in an embodiment of the present application.

[0035] Figure 5 Shown is a flow chart of an information display method provided in one embodiment of the present application.

[0036] Figure 6 Shown is a flow chart of an information display method provided in another embodiment of the present application.

[0037] Figure 7 Shown is a flow chart of an information display method provided in another embodiment of the present application.

[0038] Figure 8 Shown is a flow chart of an information display method provided in another embodiment of the present application.

[0039] Fig. 9 Shown is a flow chart of an information display method provided in another embodiment of the present application.

[0040] Fig.10 Shown is a flow chart of an information display method provided in another embodiment of the present application.

[0041] Fig.11 Shown is a schematic diagram of an exemplary application scenario of the information display method provided in an embodiment of the present application.

[0042] Fig.12 Shown is a schematic diagram of an exemplary application scenario of the information display method provided in an embodiment of the present application.

[0043] Fig.13 Shown is a schematic diagram of an exemplary application scenario of the information display method provided in an embodiment of the present application.

[0044] Fig.14 Shown is a schematic diagram of an exemplary application scenario of the information display method provided in an embodiment of the present application.

[0045] Fig.15 Shown is a schematic diagram of an exemplary application scenario of the information display method provided in an embodiment of the present application.

[0046] Fig.16 Shown is a schematic diagram of an information display device provided in one embodiment of the present application.

[0047] Fig.17 Shown is a schematic diagram of an exemplary electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] Digital information is becoming more and more open on the Internet. With the widespread use of various electronic devices and the diversification of application functions, people can obtain the information they need in a very short time through very convenient means. This development trend has brought great convenience to society while also revolutionizing the means of acquiring knowledge. Nowadays, people have become highly dependent on the way to quickly obtain information through electronic devices.

[0049] However, when reading traditional physical printed materials, since the information it can convey is limited to the printed content, if readers want to know more relevant information, they can only consult other books or use electronic devices to search and find separately, which takes a long time and leads to low learning efficiency.

[0050] This problem is particularly evident in the field of cultural relics research. When readers view pictures of cultural relics through books and other printed materials, they usually need to consult a large amount of information in order to understand the content, meaning, historical background, author and other related information of the cultural relics. Even if the investigation is conducted through the Internet, it takes a lot of time and effort, and it is difficult to ensure the accuracy of the information obtained.

[0051] In view of this, an embodiment of the present application provides an information display method, which aims to enable users who view physical printed materials to obtain required information conveniently and quickly.

[0052] Exemplary Systems

[0053] Figure 1 FIG. 1 is a schematic diagram of an exemplary information display system 100 provided in an embodiment of the present application. The information display system 100 may include a printed matter 110 , a processing device 120 , and a display device 130 .

[0054] The printed matter 110 may be printed on any material, and the surface may be printed with text, images, and other contents. For example, the printed matter 110 may be a paper printed matter such as a book, a picture album, a newspaper, a poster, and the like. In an exemplary application scenario of an embodiment of the present application, the printed content on the surface of the printed matter 110 may include an image of a cultural relic, such as a photograph, a photocopy, or a rubbing of a cultural relic. Cultural relic may include, for example, calligraphy and painting, ceramics, jade, stone tablets, ancient buildings, and the like.

[0055] The processing device 120 can be connected to the sensing device in communication, receive information provided by the sensing device, and determine the target area specified by the user on the printed matter 110 based on the information. Here, the sensing device can be used to collect information on the surface of the printed matter 110, and can also be used to collect information such as user actions and input instructions.

[0056] For example, the sensing device may be an optical sensor, which determines the target area designated by the user on the printed object 110 by collecting optical information of the surface of the printed object 110. Figure 2 In the illustrated embodiment, the information display system 100 provided by the present application may include an optical sensor 142. The optical sensor 142 may be, for example, a camera or other device that is communicatively connected to the processing device 120, and may send the collected optical information to the processing device 120. In some examples, the processing device 120 may determine the position of the optical sensor 142 on the printed matter 110 based on the optical information, and determine the target area specified by the user based on the position.

[0057] The processing device 120 can obtain the extended information of the printed matter 110 from the pre-built database 110'. The pre-built database 110' can be stored in a server in the cloud, or in a local storage device, for example, the storage device can be integrated with the processing device 120 in the same electronic device. The processing device 120 can retrieve the data stored in the database 110' by communicating with the server or storage device where the database 110' is located. For example, after determining the target area specified by the user, the processing device 120 can obtain the extended information corresponding to the printed content in the target area from the database 110'.

[0058] Furthermore, the processing device 120 may be connected in communication with the display device 130. The display device 130 may have a display function area, which may refer to all areas on the display device 130 that have a display function and are used to display information to the user. The display function area may include at least one display area, and the processing device 120 may display extended information of the printed content in the target area of ​​the printed matter 110 to the user through the display area.

[0059] It is understood that the display area can be the entire display function area, or one or more local areas in the display function area. Parameters such as the position, size, and shape of the display area can be set according to the structure of the display function area, or can be manually set by the user or developer according to needs in a specific application scenario.

[0060] In some examples, such as in Figure 1 to Figure 4In the information display system 100 shown, the display device 130 and the processing device 120 may be integrated in the same electronic device. In other examples, the display device 130 may also be a device having a display function independent of the processing device 120, which is not limited in the embodiments of the present application.

[0061] Furthermore, the processing device 120 may also be connected to an input device for communication to receive a command input by a user and update the content displayed in the display area according to the command. For example, the input device may be a touch panel, a mouse, a keyboard, a microphone, a camera, or other device that allows a user to input a command. In some examples, the optical sensor 142 may also be directly used as an input device.

[0062] For example, in Figure 3 In the illustrated embodiment, the information display system 100 provided by the present application may include a touch panel 150 as an input device. The touch panel 150 is used for a user to input instructions or other information so that the processing device 120 receives these instructions or information. The touch panel 150 may be a device independent of the processing device 120, or may be integrated with the processing device 120 in the same electronic device, or the touch panel 150 may be integrated with the display device 130 in the same display device.

[0063] For example, in Figure 3 In the information display system 100 shown, the processing device 120, the display device 130 and the touch panel 150 are all arranged in the same mobile terminal. The processing device 120 displays the extended information of the printed content of the target area on the printed material 110 through the display area of ​​the display device 130, and receives the user's instructions through the touch panel 150, and then updates the content displayed in the display area according to the instructions, such as zooming in, zooming out, rotating or switching the displayed screen.

[0064] Furthermore, in the information display system 100 having the optical sensor 142, a light emitter may be provided so that the optical sensor 142 can collect sufficient optical information so that the processing device 120 can accurately determine the target area. The light emitter may be used to actively emit auxiliary light to the surface of the printed matter 110, and the type of auxiliary light may be selected according to the working principle of the optical sensor 140, such as visible light, infrared light, or ultraviolet light.

[0065] For example, in Figure 4In the illustrated embodiment, the information display system 100 provided by the present application may include an optical device 140, in which a light emitter 144 and an optical sensor 142 may be integrated. In the optical device 140, the emission light path of the light emitter 144 and the incident light path of the optical sensor 142 may be designed in coordination, so that the auxiliary light emitted by the light emitter 144 can be effectively collected by the optical sensor 142 after being reflected by the surface of the printed object 110.

[0066] In some other examples, the light emitter 144 and the optical sensor 142 may also be provided separately, as long as the light emitter 144 enables the optical sensor 142 to collect more reflected light from the surface of the printed matter 110 , and those skilled in the art may make the arrangements according to actual needs.

[0067] It should be understood that the processing device 120 can be a processor or a server, and can be installed in a terminal device such as a mobile phone, computer, tablet computer, car computer, etc. held by the user, or can be installed in the cloud that is communicated with the terminal device. The embodiments of the present application do not limit the specific implementation method of the processing device 120.

[0068] Exemplary Methods

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0070] Figure 5 The figure is a flow chart of an information display method provided by an embodiment of the present application. The method can be, for example, Figures 1 to 4 The processing device 120 in any of the information display systems 100 shown is executed. Figure 5 As shown, the method may include the following steps S510 to S530.

[0071] S510: Determine a target area designated by a user on a printed material.

[0072] The target area is an area in the printed matter corresponding to the orthographic projection of the display area of ​​the display device on the printed matter, the display device is located on the printed matter and the display area is located on the side of the display device away from the printed matter.

[0073] When viewing a printed matter, if the user wishes to further understand the printed content in the printed matter, the area where the printed content is located can be designated as a target area. Specifically, the user can place the display device on the surface of the printed matter and bring the display device close to the target area that is desired to be designated.

[0074] It should be understood that the display device being located on the surface of the printed matter means that the display device is located between the user and the printed matter and close to the surface of the printed matter, and the embodiments of the present application do not limit whether the display device is in contact with the printed matter.

[0075] For example, when the printed matter is hung on the wall, the user can hold the display device close to the surface of the printed matter; or, when the printed matter is laid flat on the table, the user can place the display device on the surface of the printed matter. Of course, the printed matter can also be in any other posture, and the user only needs to place the display device above the surface of the printed matter and make the side of the display device with the display area face away from the printed matter (that is, the display area faces the user so that the user can see the display content in the display area).

[0076] In an embodiment of the present application, when it is determined that the orthographic projection of the display device in a direction perpendicular to the plane where its display area is located overlaps with the printed matter, it can be determined that the display device is located on the surface of the printed matter. At this time, the positional relationship between the display device and the printed matter can be detected to determine the target area specified by the user on the printed matter. Here, the target area can be an area in the printed matter corresponding to the orthographic projection of the display area of ​​the display device on the printed matter, for example, it can be an area that completely overlaps with the orthographic projection of the display area, or an area that partially overlaps with the orthographic projection of the display area, or it can be an area that has a preset positional relationship with the orthographic projection of the display area.

[0077] S520: Acquire the extended information corresponding to the printed content in the target area from the pre-built database.

[0078] In the embodiment of the present application, the pre-built database may store digital information corresponding to the printed matter, for example, the digital image and text of the printed content, as well as various extended information such as the creation background, author profile, etc. In other words, the pre-built database may store various information that the user may want to know when viewing the printed matter, based on which, by establishing a communication connection with the database, the corresponding information can be obtained according to the user's needs.

[0079] In an exemplary application scenario, the printed content on the surface of the printed matter includes a cultural relic image, such as a photograph, rubbing or photocopy of the cultural relic, etc. Correspondingly, the database may store extended information corresponding to the cultural relic image, such as an explanation of the cultural relic, a simplified form of traditional Chinese characters in the cultural relic, a version of a text written in one font (such as a calligraphy font that is difficult to recognize, such as seal script) in a cultural relic in other fonts (such as a calligraphy font that is easy to recognize, such as regular script), a restored version of unclear pictures or texts, an introduction to the characters or buildings in the picture, a simplified form of the characters in the seal, the creation era and historical background of the cultural relic, relevant information about the creator, etc.

[0080] For example, in Fig.13 In the application scenario shown, the printed matter is a paper copy of a painting, that is, the printed content on the surface of the printed matter is an image of a painting. At this time, the pre-built database can store the digital image of the painting, the original font text of the text part of the painting and the corresponding simplified text, the interpretation of the text part of the painting, the meaning and background of the painting part of the painting, the text of the seal in the painting, the author of the painting and the author's related information, the author of the inscription in the painting and the related information, the creation time and background of the painting, the introduction of the materials and pigments used in the painting, etc.

[0081] It should be understood that the type and quantity of information stored in the database can be set according to actual needs, and the embodiments of the present application do not limit this.

[0082] After determining the target area specified by the user on the printed material, the extended information corresponding to the printed content in the target area can be obtained from a pre-built database based on the correspondence between the target area and the extended information. Specifically, the information in the database can have a preset correspondence with the printed material, for example, a correspondence established based on coordinates. Based on this, after determining the position of the target area in the printed material (for example, the coordinate value of at least one vertex or center point), the information corresponding to the position (for example, information marked with the same coordinate value) can be obtained from the database as the extended information of the printed content in the target area.

[0083] S530: Displaying the extended information to the user via the display area.

[0084] After obtaining the extended information corresponding to the target area, the extended information can be displayed to the user through the display area of ​​the display device. At this time, the display area of ​​the display device is located within the user's line of sight, and the user can see the required extended information without adjusting his posture or line of sight.

[0085] According to the information display method provided by the present application, when the user places the display device above the surface of the printed matter, the area of ​​the printed matter blocked by the display device in the user's line of sight can be determined, and the extended information related to the printed content of the area and / or the nearby area can be displayed on the display device. In this way, when the user wants to know the extended information of certain printed content in the printed matter, the target area can be specified by simply placing the display device between his eyes and the target area where the printed content is located (for example, holding the display device above or in front of the printed matter, or placing the display device on the surface of the printed matter, etc.), and then the extended information corresponding to the printed content in the target area can be seen on the display device.

[0086] In some implementations of the present application, step S530 may specifically include: displaying an image of the printed content in the display area, and overlaying the extended information on the image.

[0087] Specifically, in the display area of ​​the display device, the extended information can be displayed on the original image of the printed content in a layer overlay manner. The extended information may include text information and / or image information used to help users understand the printed content, and different types of extended information may be provided in different layers. According to user needs, one or more layers may be overlaid and displayed on the original image of the printed content, as long as the contents in each layer do not interfere with each other.

[0088] Exemplarily, the extended information can overlay the corresponding text and / or image on the original image to visually replace these contents while retaining other visual elements in the original image (such as background color or texture, etc.); or, the extended information can be displayed in a window-like form (such as a balloon form, etc.) near the corresponding content on the original image for users to view and compare.

[0089] By implementing this method, the image of the printed content and the corresponding extended information can be displayed in the display area at the same time. As a result, the corresponding relationship between the extended information and the printed content can be more intuitively displayed, making it easier for users to understand the printed content. In addition, by allowing users to still see the original image of the printed content when viewing the extended information, it is possible to minimize the interference of the extended information on the user's visual experience.

[0090] In some other implementations of the present application, the extended information may include an enhanced image of the printed content, wherein the enhanced image and the printed content have the same or corresponding visual features, and the visual features may include at least one of a pattern, text, font, and color.

[0091] Specifically, in this implementation, according to the positional correspondence between the original image of the printed matter and the enhanced image, the area corresponding to the target area on the printed matter can be determined in the enhanced image, and the area can be displayed. Here, the layer of the enhanced image can be superimposed on the layer of the original image of the printed content, or only the enhanced image can be displayed without layer superposition. In this implementation, since the enhanced image and the printed content have the same or corresponding visual features, even if only the enhanced image is displayed, it can ensure that the user's visual experience is not affected. As a result, the operation of layer superposition display can be omitted, saving computing and storage resources, and improving processing efficiency.

[0092] For example, an enhanced image can be an image that emphasizes the overall or local characteristics of the original image based on the original image of the printed content, can be an image that makes an unclear original image clear, emphasizes certain specific features in the original image, or suppresses certain specific features in the original image, can be an image that enlarges the differences between the features of different objects in the original image, etc.

[0093] As an exemplary scenario, the printed content in the printed matter includes an image of a rubbing of a stone tablet. Due to its long history, the stone tablet has been corroded by wind and rain for a long time, and many words in the inscription are difficult to distinguish. Therefore, when the user views the rubbing image on the printed matter, it may be difficult to recognize these words, resulting in the inability to understand the meaning of the inscription.

[0094] In this scenario, the extended information of the stone tablet rubbing can be an enhanced image of the stone tablet rubbing, in which the damaged text has become clearly visible. The enhanced image can be, for example, a restored rubbing image of a replica of the stone tablet, or a restored image obtained by image processing of the original rubbing image.

[0095] When viewing the rubbing image in the printed matter, the user can place the display device above the area containing unclear text on the printed matter to designate the area as the target area. At this time, the processor 120 can obtain the enhanced image corresponding to the rubbing image in the database, determine the part of the enhanced image corresponding to the target area according to the coordinate information of the target area, and display the part to the user through the display area of ​​the display device.

[0096] Enhanced images can improve the image quality of the original image in the printed content, increase the amount of information, and enhance the recognition effect of the image. At the same time, they can also retain the visual characteristics of the original image and minimize the impact on the user's perception. Therefore, by using the enhanced image of the printed content as extended information for display, more visual details can be provided to the user, making the visual transition at the boundary between the display area and the printed matter smoother and the display effect more delicate.

[0097] In some implementations of the present application, the display area may be a display function area of ​​a display device. That is, the display function area on the display device may be set as a display area as a whole, so that the display screen has a larger area and enhances the visual effect. For example, Fig.11 As shown, in an application scenario where the display area is the entire display function area of ​​the display device, correspondingly, the target area on the printed matter is the area covered by the orthographic projection of the entire display function area on the printed matter.

[0098] Alternatively, in some other implementations, the display area may also be a local area in the display function area. That is, an area occupying any range in the display function area may be set as the display area for implementing the information display method provided in the embodiment of the present application, thereby improving the design flexibility of the display function area and enabling the display device to implement more functions at the same time. For example, Fig.12 As shown, in an application scenario where the display area is a local area in the display function area of ​​the display device, the target area on the printed matter is the area covered by the orthographic projection of the local area on the printed matter.

[0099] For example, the display area can be one or more windows in the graphical user interface (GUI) of an application. When the display area is used to display extended information of the target area, other areas in the display function area can synchronously display other information (for example, an input page can be displayed for users to input information, and other information related to the extended information can be displayed, etc.). This can effectively improve the utilization efficiency of the display device and further improve the user experience.

[0100] Figure 6 The figure is a flow chart of an information display method provided by another embodiment of the present application. The method can be, for example, Figure 1-Figure 4 The processing device 120 in any of the information display systems 100 shown is executed.

[0101] exist Figure 5 Based on the embodiment shown, Figure 6 The illustrated embodiment provides a specific implementation method for step S510.

[0102] Specifically, if Figure 6 As shown, in some embodiments of the present application, step S510 may include:

[0103] S610: Determine an area in the printed matter that overlaps with the orthographic projection of the display area on the printed matter as a target area.

[0104] In other words, when the display device is located above the surface of the printed matter, the area covered by the orthographic projection of the display area of ​​the display device on the printed matter is the target area on the printed matter. Intuitively speaking, when the display area is used to display the original image of the printed content in the target area, the user will feel like holding a piece of "framed glass" when viewing the display area and the printed matter. The transparent "glass" part is the display area of ​​the display device, and the opaque "frame" part is the border of the display device and the area other than the display area in the display function area.

[0105] Through such a setting, in the user's sight, the display area will visually replace the target area on the printed surface. By achieving such a visual effect, the user can intuitively and accurately understand the correspondence between the extended information and the printed content while seeing the extended information, further improving the efficiency of information transmission and achieving a better display effect.

[0106] Figure 7 The figure is a flow chart of an information display method provided by another embodiment of the present application. The method can be, for example, Figure 1-Figure 4 The processing device 120 in any of the information display systems 100 shown is executed.

[0107] When viewing printed materials, especially those with rich content and large area, users may want to know more information and thus designate multiple target areas. In addition, when the displayed text is small and difficult to read, or the orientation of the displayed content is inconvenient to view, users may want to adjust the display screen to improve comfort.

[0108] In view of this, if Figure 7 As shown, based on the aforementioned embodiments, the information display method provided in another embodiment of the present application may further include the following steps.

[0109] S710: Receive a display adjustment instruction.

[0110] For example, the display adjustment instruction may include an action of specifying a new target area by changing the position of the display device, and may also include an operation of adjusting a display image in the display area.

[0111] S720: Update and display the extended information according to the display adjustment instruction.

[0112] For example, when the display device moves or rotates on the printed matter, it can be regarded as the user inputting a display adjustment instruction, that is, specifying a new target area on the printed matter. Since the position of the display device has changed, its orthographic projection on the printed matter has also moved. At this time, S510 to S530 can be executed again, that is, the area on the printed matter corresponding to the new orthographic projection of the display device is used as the new target area, and the new extended information corresponding to the printed content in the new target area is obtained from the database, and the new extended information is displayed in the display area.

[0113] Alternatively, the user may directly input a display adjustment instruction through an input device, and the display adjustment instruction may include one or more instructions such as screen zooming, layer switching, and type increase or decrease of extended information. At this time, the corresponding information may be obtained from the database according to the display adjustment instruction and displayed in the display area, that is, the extended information displayed in the display area is updated.

[0114] For example, in Figure 3 In the information display system shown, when the user inputs a screen zoom-out instruction through the touch panel 150, the display area needs to display more extended information corresponding to the printed content. At this time, the processor 120 can determine the area range on the printed matter corresponding to the zoomed-out screen according to the instruction, obtain the corresponding extended information from the database according to the area range as the updated extended information, and display the updated extended information in the display area.

[0115] For another example, the display area may be a window area in the display function area, and other areas in the display function area display information type selection buttons. In this scenario, when the display area displays the first type of extended information (e.g., the simplified Chinese characters corresponding to the traditional Chinese characters in the printed matter), the user can operate the information type selection button and input instructions through the touch panel 150 integrated with the display device to select the additional display of the second type of extended information (e.g., text interpretation). According to the instruction, the processor 120 can obtain the second type of extended information corresponding to the printed content in the target area in the database, and display it in the display area together with the first type of extended information.

[0116] By providing users with display adjustment functions, the embodiments of the present application further expand the functions and application scenarios of the information display method, improve the convenience and flexibility of operation, and greatly enhance the user experience.

[0117] As mentioned above, in step S510, the target area is an area in the printed matter corresponding to the orthographic projection of the display area of ​​the display device on the printed matter. An exemplary implementation of step S510 (ie, determining the target area on the printed matter) is specifically provided below.

[0118] Figure 8 The figure is a flow chart of an information display method provided by another embodiment of the present application. The method can be, for example, Figure 1-Figure 4 The processing device 120 in any of the information display systems 100 shown is executed.

[0119] like Figure 8 As shown, in an embodiment of the present application, step S510 in the aforementioned embodiment may include the following steps.

[0120] S810: Collecting optical information of the surface of the printed matter through an optical sensor fixedly arranged relative to the display device.

[0121] As an optional implementation, the surface of the printed matter can be photographed by an optical sensor fixedly arranged with the display device. In this case, since the positional relationship between the optical sensor and the display device is fixed, the position of the display device on the printed matter can be calculated as long as the position of the optical sensor on the printed matter is determined, and then the area occupied by the orthographic projection of the display area on the printed matter can be calculated based on the parameters of the display area in the display device.

[0122] Exemplarily, the optical sensor may be a component integrated with the display device in the same electronic device, for example, the optical sensor may be a camera in a mobile terminal, and the display device may be a display panel of the mobile terminal. Alternatively, the optical sensor may be an external component fixed to the display device or the electronic device to which the display device belongs through a connector, for example, the display device may be a display panel of a mobile terminal, and the optical sensor may be a camera plugged into the mobile terminal through an external port of the mobile terminal (such as a USB port, a Thunderbolt port, etc.). Alternatively, the optical sensor may be fixedly arranged relative to the display device in any other manner, and the embodiments of the present application are not limited to this.

[0123] S820: Identify the position of the display device on the printed matter based on the optical information.

[0124] The optical information collected by the optical sensor contains implicit position information of the optical sensor. Based on the fixed parameters of the optical sensor, the position of the optical sensor on the printed matter can be inferred from the optical information, and then the position of the display device on the printed matter can be calculated.

[0125] In some examples, in order to determine the position of the optical sensor on the printed matter, an image recognition method can be used. Specifically, a digital image of the printed content in the printed matter can be pre-stored in a database or storage medium that is in communication with the processor 120. Based on this, the processor 120 can control the optical sensor to shoot on the surface of the printed matter to obtain the captured current area image, and compare the current area image with the pre-stored digital image of the printed content, and then determine the area range occupied by the current area image on the printed matter. Furthermore, based on the imaging parameters and other data of the optical sensor, the position of the optical sensor on the printed matter can be calculated based on the area range occupied by the current area image on the printed matter, and then the position of the positive projection of the display device on the printed matter is determined through the aforementioned process.

[0126] Alternatively, in other examples, the surface of the printed matter may be printed with a code containing position information, such as a dot matrix code or other two-dimensional code containing coordinate information. Correspondingly, the database or storage medium connected to the processor 120 may pre-store the position information corresponding to the code of the entire printed matter. Fig. 9 In the illustrated embodiment, S820 may include:

[0127] S821: Obtain the code contained in the optical information.

[0128] S822: Determine the position of the optical sensor relative to the printed matter according to the position information corresponding to the code.

[0129] S823: Determine the position of the display device on the printed matter according to the position of the optical sensor relative to the printed matter and the relative position between the optical sensor and the display device.

[0130] Specifically, when the optical sensor captures the surface of the printed matter with the code printed on it, it can collect the code within the shooting range. Based on this, after receiving the optical information collected by the optical sensor, the processor 120 can determine the code contained therein, and then determine the position of the optical sensor on the printed matter according to the pre-stored position information, and then determine the position of the orthographic projection of the display device on the printed matter through the above process.

[0131] For example, the code may be a dot matrix code representing coordinate values, which is printed at various locations on the surface of the printed object. The coordinate value may represent the location (such as a point or pixel, etc.) of the location in a two-dimensional coordinate system based on the surface of the printed object. The processor may determine the coordinate value corresponding to the dot matrix code based on the dot matrix code contained in the optical information collected by the optical sensor, and further determine the position of the optical sensor on the surface of the printed object.

[0132] S830: Determine the orthographic projection of the display area on the printed matter according to the position of the display device on the printed matter.

[0133] As mentioned above, the display area can be the entire display function area of ​​the display device, or a preset local area in the display function area. Based on this, the position of the display area relative to the outer edge of the display device can be predetermined, and after determining the position of the orthographic projection of the display device on the printed matter (that is, the area defined by the orthographic projection of the outer edge of the display device on the printed matter), the area occupied by the orthographic projection of the display area on the printed matter can be calculated.

[0134] It can be understood that the information display method provided in the embodiment of the present application can be applied to display devices of various types or models. Considering that the position parameters (such as size, shape, edge position, etc.) of the display function area on different display devices may be different, in order to be compatible with different display devices to improve the display quality, in some embodiments, S830 may specifically include: determining the shape parameters of the display area; determining the orthographic projection of the display area on the printed matter according to the position and shape parameters of the display device on the printed matter.

[0135] In the case where the display function area constitutes the display area as a whole, the shape parameters of the display function area can be directly determined according to the structural parameters of the display device. Here, the shape parameters of the display function area are parameters used to represent the position of the outer edge of the display function area relative to the outer edge of the display device. According to the shape parameters of the display function area, the coordinate value of any point on the outer edge of the display function area in the display device coordinate system (for example, a two-dimensional coordinate system with the geometric center of the display device as the origin) can be determined.

[0136] In the case where the display area is a preset local area in the display function area, the position of the display area relative to the display function area can be predetermined. At this time, the shape parameters of the display area can be determined based on the position of the display area relative to the display function area and the position of the display function area relative to the display device (that is, the shape parameters of the display function area). Here, the shape parameters of the display area are similar to the shape parameters of the display function area, and are parameters used to represent the position of the outer edge of the display area relative to the outer edge of the display device. According to the shape parameters of the display area, the coordinate value of any point on the outer edge of the display area in the display device coordinate system (for example, a two-dimensional coordinate system with the geometric center of the display device as the origin) can be determined.

[0137] After determining the shape parameters of the display area in the current display device, the orthographic projection of the display area on the printed matter can be calculated according to the position of the display device on the printed matter determined in the above steps. As a result, the information display method provided in the embodiment of the present application can be better applied to display devices of various types or models, and good display effects can be achieved on different display devices.

[0138] S840: The area on the printed matter corresponding to the orthographic projection is taken as the target area.

[0139] As mentioned above, the area in the printed material corresponding to the orthographic projection of the display area on the printed material may be, for example, an area that completely overlaps with the orthographic projection of the display area, or an area that partially overlaps with the orthographic projection of the display area, or an area that has a preset positional relationship with the orthographic projection of the display area.

[0140] After the position of the orthographic projection on the printed matter (eg, the coordinate values ​​of each vertex) is determined, the position of the target area (eg, the coordinate values ​​of each vertex) can be determined based on a preset positional relationship between the orthographic projection and the target area.

[0141] Fig.10 The figure is a flow chart of an information display method provided by another embodiment of the present application. The method can be, for example, Figure 1-Figure 4 The processing device 120 in any of the information display systems 100 shown is executed.

[0142] In order to further improve the accuracy of determining the target area, in this embodiment, the aforementioned step S810 may include the following steps.

[0143] S1010: Irradiate auxiliary light onto the surface of the printed matter.

[0144] S1020: Collect auxiliary light reflected from the surface of the printed object through an optical sensor to obtain optical information.

[0145] The auxiliary light can be irradiated onto the surface of the printed matter by a light emitter, for example. The light emitter can be arranged in an electronic device to which the display device belongs, or in a component to which the optical sensor belongs, or can be arranged independently. It should be understood that as long as the light emitter can irradiate the auxiliary light onto the surface of the printed matter, the embodiments of the present application do not limit the specific arrangement of the light emitter.

[0146] Auxiliary light can be white light, ultraviolet light or infrared light, and the type of auxiliary light can be set according to the needs of different scenes.

[0147] In some examples, when the user specifies the target area, the distance between the display device and the printed surface is small, and the optical sensor fixed to the display device may be very close to the printed surface. Therefore, the display device and the optical sensor may block a portion of the light directed to the printed surface, resulting in less reflected light being collected by the optical sensor, that is, the optical sensor may not be able to collect sufficient optical information.

[0148] At this time, white light can be used as auxiliary light. For example, the light emitter can be controlled to irradiate the auxiliary light from between the display device and the printed object to the surface of the printed object, thereby enhancing the reflected light on the surface of the printed object, so that the optical sensor can collect more sufficient optical information.

[0149] In some examples, when a code containing position information is printed on the surface of a printed object, the optical information collected by the optical sensor may include not only the code but also the printed content. Here, in order to avoid interference of the printed content in the optical information on the recognition of the code, the two can be printed separately using different printing materials, and auxiliary light that is more sensitive to the printed material of the code can be used to illuminate the surface of the printed object, so that the optical sensor can collect more reflected light from the code, and minimize the impact of the printed content on the recognition of the code.

[0150] Specifically, the code may be printed by a first material, and the printed content on the printed matter may be printed by a second material, wherein the second material does not contain the first material, and the reflectivity of the auxiliary light on the first material is higher than the reflectivity on the second material.

[0151] For example, the code (such as a dot matrix code, etc.) can be printed with carbon-containing ink, and the printed content can be printed with carbon-free ink. In this scenario, the auxiliary light can be infrared light, which has a higher reflectivity on carbon-containing ink. Based on this, in the optical information collected by the optical sensor, the code is black, while the printed content and the blank area are white, making the code easier to identify, thereby improving the accuracy of determining the target area.

[0152] For another example, in order to prevent the code on the surface of the printed material from interfering with the visual effect of the printed content, the code can be an invisible code, printed with invisible materials. In some examples, the code can be printed with fluorescent materials, and the printed content can be printed with non-fluorescent materials, and the auxiliary light can be ultraviolet light that can be reflected by the fluorescent material. Based on this, in the optical information collected by the optical sensor, the code is brighter, while the printed content and blank areas are darker, making the code easier to identify. In addition, since the code is printed with fluorescent materials that are not easily identified by the naked eye, when users view the printed material under natural light, they can hardly see the dot matrix code on the surface of the printed material. As a result, the viewing effect of the printed material is greatly improved, and the code is prevented from interfering with the user's viewing experience.

[0153] Furthermore, in some embodiments of the present application, an optical path lengthening structure may be provided in the incident light path of the optical sensor, so as to allow the reflected light from the printed surface to reach the lens of the optical sensor via a longer path. For example, the optical path lengthening structure may include a plurality of reflectors with specific angles, and the reflectors are used to make the incident light reach the imaging lens after multiple reflections, thereby achieving the effect of lengthening the optical path.

[0154] When the display device is placed close to the printed surface, the optical sensor fixed relative to it may also be located very close to the printed surface, for example, the lens of the optical sensor may directly contact the printed surface. At this time, the optical sensor is limited by its own parameters (such as minimum focal length) and may not be able to capture clear codes, resulting in inability to achieve accurate positioning.

[0155] Therefore, by setting an optical path extension structure in the optical sensor, it is equivalent to lengthening the distance between the lens of the optical sensor and the surface of the printed object, so that the optical sensor can clearly capture the code at a position very close to the surface of the printed object, thereby accurately determining the position of the display device on the printed object.

[0156] In order to make the information display method provided in the embodiment of the present application easier to understand, the following is a reference to Figures 13-15 , a specific application scenario of the information display method is described. In this application scenario, the information display method is composed of Figure 4 The processing device 120 in the information display system shown is executed.

[0157] like Figure 4 as well as Fig.14 , Fig.15As shown, the processing device 120 and the display device 130 are integrated into the same mobile terminal, and the optical device 140 is fixedly arranged relative to the mobile terminal and is communicatively connected with the processing device 120. For example, the optical device 140 may have a USB plug, and the mobile terminal may be provided with a USB port. When the USB plug of the optical device 140 is plugged into the USB port of the mobile terminal, the optical device 140 and the mobile terminal are fixedly arranged relative to each other in a preset positional relationship, and can communicate with the processing device 120 in the mobile terminal.

[0158] like Fig.13 As shown, in this application scenario, the printed matter 110 is a paper copy of a painting, and an invisible dot matrix code printed with fluorescent ink is printed on the surface of the printed matter 110. At this time, the pre-built database 110' stores the extended information corresponding to the printed matter 110, including at least the digital image of the painting, the simplified Chinese text corresponding to the original font text of the text part of the painting, and the simplified Chinese text of the seal in the painting.

[0159] like Fig.14 or Fig.15 As shown, when the user places the mobile terminal on the surface of the printed object 110, the processing device 120 can control the light emitter 144 in the optical device 140 to irradiate ultraviolet rays on the surface of the printed object 110, and control the optical sensor 142 to photograph the surface of the printed object 110. At this time, the processing device 120 can obtain the optical information of the surface of the printed object 110, and recognize the dot matrix code in the optical information according to the brightness difference, and then determine the coordinate value corresponding to the dot matrix code, that is, the position of the optical device 140 on the surface of the printed object 110.

[0160] According to the preset positional relationship between the optical device 140 and the mobile terminal and the shape parameters of the display area in the display device 130, the processing device 120 can calculate the area covered by the orthographic projection of the display area on the printed matter 110, and then determine the area corresponding to the area on the printed matter 110 as the target area specified by the user. Fig.14 or Fig.15 As shown, the display area is a rectangle, and the processing device 120 can determine the coordinate values ​​of each vertex of the orthographic projection of the display area on the surface of the printed matter 110, and then determine that the target area is a rectangle and the coordinate values ​​of each vertex of the target area are the coordinate values ​​of each vertex of the orthographic projection.

[0161] According to the coordinate information of the target area, the processing device 120 may obtain the extended information having the coordinate information from the database 110 ′, and display the extended information in the display area.

[0162] like Fig.13 and Fig.14As shown, the target area designated by the user on the printed matter 110 is area A. At this time, the processing device 120 can obtain the extended information corresponding to area A (i.e., the simplified Chinese characters of the seal in area A) from the database 110', display the digital image of area A in the display area, and overlay the extended information on the digital image. Fig.14 As shown, the simplified Chinese characters of the seal have the same color as the seal in the original image, visually replacing the traditional Chinese characters in the seal in the original image.

[0163] like Fig.13 and Fig.15 As shown in FIG. 1 , the target area designated by the user on the printed material 110 is area B. At this time, the processing device 120 can obtain the extended information corresponding to area B (i.e., the simplified characters of the characters in area B and punctuation marks that facilitate the understanding of the sentence) from the database 110', display the digital image of area B in the display area, and overlay the extended information on the digital image. Fig.15 As shown, the simplified characters have the same size and color as the traditional characters in the original image, visually replacing the traditional characters in the original image, and adding punctuation marks that facilitate readers to understand the content of the sentence.

[0164] Based on the information display method provided by the embodiment of the present application, when a user wants to know the extended information of certain printed contents in a printed matter, the user can place the display device above the surface of the printed matter to specify the target area, and then can see the extended information corresponding to the printed contents in the target area in the display area of ​​the display device. Thus, the user's learning efficiency and viewing experience of printed matters are greatly improved, and new possibilities and development space are provided for traditional paper printed matters.

[0165] It should be understood that the above steps can be combined with the steps or methods provided in the aforementioned embodiments of the present application. The specific combination method can be designed by technical personnel in this field according to actual needs, and the embodiments of the present application are not limited to this.

[0166] Exemplary Devices

[0167] Fig.16 FIG. 1 is a schematic diagram of an information display device 1600 provided in an embodiment of the present application. Fig.16 As shown, the information display device 1600 provided in the embodiment of the present application may include a determination module 1610 , an information acquisition module 1620 and a display module 1630 .

[0168] Among them, the determination module 1610 can be used to determine the target area specified by the user on the printed material, wherein the target area is the area in the printed material corresponding to the positive projection of the display area of ​​the display device on the printed material, the display device is located on the printed material and the display area is located on the side of the display device away from the printed material; the information acquisition module 1620 can be used to obtain extended information corresponding to the printed content in the target area in a pre-constructed database; the display module 1630 can be used to display the extended information to the user through the display area of ​​the display device.

[0169] Through the information display device provided by the embodiment of the present application, when the user places the display device above the surface of the printed matter, the area of ​​the printed matter blocked by the display device in the user's line of sight can be determined, and the extended information related to the printed content of the area and / or the nearby area can be displayed on the display device. In this way, when the user wants to know the extended information of certain printed content in the printed matter, the target area can be specified by simply placing the display device between his eyes and the target area where the printed content is located (for example, holding the display device above or in front of the printed matter, or placing the display device on the surface of the printed matter, etc.), and then the extended information corresponding to the printed content in the target area can be seen on the display device.

[0170] In some embodiments, the determination module 1610 may include a collection unit, a recognition unit, a calculation unit, and a determination unit. The collection unit may be used to collect optical information on the surface of the printed object through an optical sensor fixedly arranged relative to the display device; the recognition unit may be used to identify the position of the display device on the printed object based on the optical information; the calculation unit may be used to determine the orthographic projection of the display area on the printed object according to the position of the display device on the printed object; and the determination unit may be used to take the area on the printed object corresponding to the orthographic projection as the target area.

[0171] In some embodiments, a code containing position information is printed on the surface of the printed object. In this case, the identification unit can be specifically used to: obtain the code contained in the optical information; determine the position of the optical sensor relative to the printed object according to the position information corresponding to the code; determine the position of the display device on the printed object according to the position of the optical sensor relative to the printed object and the relative position between the optical sensor and the display device.

[0172] In some embodiments, the collection unit may be specifically used to: irradiate auxiliary light onto the surface of the printed matter; and collect the auxiliary light reflected by the surface of the printed matter through an optical sensor to obtain optical information.

[0173] In some embodiments, the code is printed by a first material, and the printed content on the printed matter is printed by a second material, wherein the second material does not contain the first material, and the reflectivity of the auxiliary light on the first material is higher than the reflectivity on the second material.

[0174] In some embodiments, a light path lengthening structure is provided in the incident light path of the optical sensor.

[0175] In some embodiments, the above-mentioned calculation unit can be specifically used to: determine the shape parameters of the display area; and determine the orthographic projection of the display area on the printed matter according to the position and shape parameters of the display device on the printed matter.

[0176] In some embodiments, the determination module 1610 may be configured to determine that an area in the printed matter that coincides with the orthographic projection of the display area on the printed matter is the target area.

[0177] In some embodiments, the display area is a display function area of ​​a display device or a partial area of ​​the display function area.

[0178] In some embodiments, the display module 1630 may be used to display extended information to the user through the display area of ​​the display device, including: displaying an image of the printed content in the display area, and superimposing the extended information on the image, wherein the extended information includes text information and / or image information for helping the user understand the printed content. Alternatively, in some embodiments, the extended information displayed by the display module 1630 may include an enhanced image of the printed content, wherein the enhanced image has the same or corresponding visual features as the printed content, and the visual features include at least one of a pattern, text, font, and color.

[0179] In some embodiments, the information display device 1600 further includes: a receiving module for receiving a display adjustment instruction; and an updating module for updating and displaying the extended information according to the display adjustment instruction.

[0180] It should be understood that the principles, functions, executed actions, required environmental conditions, and technical effects of all optional implementation methods of each module in the information display device 1600 provided in this embodiment can be referred to the corresponding contents in the exemplary method and will not be described in detail here.

[0181] Exemplary Devices

[0182] The present application also provides an optical device, which may include an optical sensor. Here, the optical sensor may be, for example, Figure 2-Figure 4 The optical sensor 142 in any of the information display systems 100 shown.

[0183] Specifically, the optical sensor is configured to be fixedly arranged relative to the display device, and is used to collect optical information of the surface of the printed matter and provide the optical information to the processing device, so that the processing device can perform the exemplary method provided in any of the aforementioned embodiments of the present application. That is, the processing device can identify the orthographic projection of the display area of ​​the display device on the printed matter based on the optical information provided by the optical device, determine the area on the printed matter corresponding to the orthographic projection as the target area specified by the user on the printed matter, and display the extended information corresponding to the printed content in the target area to the user through the display area, wherein the display device is located on the printed matter and the display area is located on the side of the display device away from the printed matter.

[0184] Optionally, an optical path lengthening structure may be provided in the incident light path of the optical sensor, so as to allow the reflected light from the printed surface to reach the lens of the optical sensor via a longer path. For example, the optical path lengthening structure may include a plurality of reflectors with specific angles, and the reflectors are used to make the incident light reach the imaging lens after multiple reflections, thereby achieving the effect of lengthening the optical path.

[0185] In some embodiments, the optical device may further include a light emitter. The light emitter is used to irradiate auxiliary light onto the surface of the printed object, so that the optical sensor can obtain optical information by collecting the auxiliary light reflected from the surface of the printed object. Here, the optical device may be, for example, Figure 4 In the optical device 140 of the information display system 100 shown in FIG. 1 , the light emitter may be, for example, Figure 4 The light emitter 144 in the information display system 100 is shown.

[0186] It should be understood that the principles, functions, actions performed, required environmental conditions, and technical effects of all optional implementation methods of each component in the optical device provided in this embodiment can be referred to the corresponding contents in the exemplary method, and will not be described in detail here.

[0187] Fig.17 FIG. 1 is a schematic diagram of an exemplary electronic device 1700 provided in an embodiment of the present application. Fig.17 As shown, the electronic device 1700 may include a processor 1710 and a memory 1720. The memory 1720 stores computer instructions, and the processor 1710 is used to execute the computer instructions to implement the information display method provided by any of the above embodiments. For example, the processor 1710 may be as shown in FIG. Figure 1-Figure 4 The processing device 120 in any of the information display systems 100 shown in FIG.

[0188] Exemplary computer-readable storage media

[0189] Other embodiments of the present application further provide a computer-readable storage medium, including computer instructions stored thereon, and when the computer instructions are executed by a processor, the processor executes an information display method as provided in any of the above embodiments.

[0190] Optionally, the computer storage medium can be any tangible medium, such as a floppy disk, a CD-ROM, a DVD, a hard drive, or a network medium.

[0191] Exemplary computer program products

[0192] Other embodiments of the present application further provide a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device can execute the information display method provided by any of the above embodiments of the present application.

[0193] The block diagrams of the devices, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. It will be appreciated by those skilled in the art that these devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "comprise", "include", "have", etc. are open words, to "including but not limited to", and can be used interchangeably with them, unless the context clearly indicates otherwise. The vocabulary "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0194] It should also be noted that in the apparatus, device and method of the present application, each module or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0195] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the above aspects, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0196] The above description is intended to illustrate and describe the technical solution of the present application. In addition, this description is not intended to limit the embodiments of the present application to the scope of the above disclosed forms. Although multiple exemplary aspects and embodiments have been discussed in the above content, those skilled in the art can easily obtain other variations, modifications, changes, additions and sub-combinations based on the above content.

[0197] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An information display method, characterized in that: include: Determine a target area specified by a user on a printed matter, wherein the target area is an area in the printed matter corresponding to an orthographic projection of a display area of ​​a display device on the printed matter, the display device is located on the printed matter and the display area is located on a side of the display device away from the printed matter; Acquiring extended information corresponding to the printed content in the target area from a pre-built database; The extended information is displayed to the user via the display area.

2. The information display method according to claim 1, characterized in that: The step of determining the target area specified by the user on the printed matter comprises: Collecting optical information of the surface of the printed matter by means of an optical sensor fixedly arranged relative to the display device; identifying a position of the display device on the printed matter based on the optical information; Determining the orthographic projection of the display area on the printed matter according to the position of the display device on the printed matter; The area on the printed matter corresponding to the orthographic projection is taken as the target area.

3. The information display method according to claim 2, characterized in that: The surface of the printed matter is printed with a code containing position information. The identifying the position of the display device on the printed matter based on the optical information comprises: Obtaining the code contained in the optical information; Determining the position of the optical sensor relative to the printed object according to the position information corresponding to the code; The position of the display device on the printed matter is determined according to the position of the optical sensor relative to the printed matter and the relative position between the optical sensor and the display device.

4. The information display method according to claim 3, characterized in that: The collecting of optical information on the surface of the printed object by means of an optical sensor fixedly arranged relative to the display device comprises: irradiating auxiliary light onto the surface of the printed matter; The auxiliary light reflected from the surface of the printed object is collected by the optical sensor to obtain the optical information.

5. The information display method according to claim 4, characterized in that: The code is printed by a first material, and the printed content on the printed matter is printed by a second material, wherein the second material does not contain the first material. The reflectivity of the auxiliary light on the first material is higher than the reflectivity on the second material.

6. The information display method according to any one of claims 2 to 5, characterized in that: An optical path lengthening structure is provided in the incident light path of the optical sensor.

7. The information display method according to claim 2, characterized in that: The determining the orthographic projection of the display area on the printed matter according to the position of the display device on the printed matter comprises: Determining shape parameters of the display area; The orthographic projection of the display area on the printed matter is determined according to the position of the display device on the printed matter and the shape parameters.

8. The information display method according to claim 1 or 7, characterized in that: The display area is a display function area of ​​the display device or a partial area of ​​the display function area.

9. The information display method according to claim 1, characterized in that: The step of determining the target area specified by the user on the printed matter comprises: An area in the printed matter that overlaps with the orthographic projection of the display area on the printed matter is determined as the target area.

10. The information display method according to claim 1 or 9, characterized in that: The displaying the extended information to the user through a display area of ​​a display device includes: displaying an image of the printed content in the display area and superimposing the extended information on the image, The extended information includes text information and / or image information for helping the user understand the printed content.

11. The information display method according to claim 1 or 9, characterized in that: The extended information includes an enhanced image of the printed content, wherein the enhanced image has the same or corresponding visual features as the printed content, and the visual features include at least one of a pattern, text, font, and color.

12. The information display method according to claim 1, characterized in that: Also includes: receiving display adjustment instructions; The extended information is updated and displayed according to the display adjustment instruction.

13. An information display device, characterized in that: include: a determination module, configured to determine a target area specified by a user on a printed matter, wherein the target area is an area in the printed matter corresponding to an orthographic projection of a display area of ​​a display device on the printed matter, the display device being located on the printed matter and the display area being located on a side of the display device away from the printed matter; An information acquisition module, used to acquire extended information corresponding to the printed content in the target area from a pre-built database; A display module is used to display the extended information to the user through the display area.

14. An optical device, characterized in that: include: Optical sensors, The optical sensor is configured to be fixedly arranged relative to the display device, and is used to collect optical information of the surface of the printed matter and provide the optical information to the processing device, so that the processing device determines the orthographic projection of the display area of ​​the display device on the printed matter based on the optical information, determines the area on the printed matter corresponding to the orthographic projection as the target area specified by the user on the printed matter, and displays the extended information corresponding to the printed content in the target area to the user through the display area, The display device is located on the printed matter and the display area is located on a side of the display device away from the printed matter.

15. The optical device according to claim 14, characterized in that Also includes: Light emitter, The light emitter is used to irradiate auxiliary light onto the surface of the printed object, so that the optical sensor acquires the optical information by collecting the auxiliary light reflected by the surface of the printed object.

16. An electronic device, characterized in that: include: Memory, for storing computer instructions; A processor, configured to execute the computer instructions to implement the information display method according to any one of claims 1 to 12.

17. A computer-readable storage medium, characterized in that: Computer instructions are stored, and when the computer instructions are executed by a processor, the information display method according to any one of claims 1 to 12 is implemented.

Images