Devices to assist the visually impaired or visually impaired when shopping

Abstract

An electronic price information and / or product information display device, abbreviated as display device, configured to visually reproduce product price information and / or product information and configured to store reproduction data representative of price information and / or product information, the display device comprising a first near-field communication device, in particular a first Near-Field-Communication device, abbreviated as NFC device, configured in particular for short-range contactless communication with a reproduction device having NFC functionality, the reproduction data comprising display reproduction data defined for visual reproduction of the price information and / or product information by the electronic price information and / or product information display device itself and output reproduction data provided in particular as reproduction data in audio and / or text format for output to the reproduction device by means of short-range contactless communication and enabling acoustic reproduction of the price information and / or product information by the reproduction device therein, the short-range communication device being configured to access the output reproduction data and to output the output reproduction data in the course of the short-range contactless communication.

Description

[Technical Field]

[0001] The present invention relates to a device for assisting visually impaired or visually impaired people when shopping. [Background technology]

[0002] Nowadays, in stores, product information and / or price information is displayed on electronic display units, in particular electronic shelf labels, technically called "Electronic Shelf Labels" (ESL for short). Visual recognition of such information is a great challenge for the visually impaired and is impossible for the blind.

[0003] Today, highly developed and relatively expensive dedicated devices are commercially available, such as the device developed by OrCam Technologies Ltd. and sold under the trade name OrCam MyEye (https: / / www.orcam.com / en / myeye2 / ) (see also Patent Document 1), which converts optically captured images into text and reads it to the user.

[0004] However, visually impaired people are often economically disadvantaged, which can make purchasing such specialized equipment difficult or even impossible. Furthermore, for such people, operating additional specialized equipment is another daily challenge that is certainly not desirable as such. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] European Patent Publication No. 2490155 [Patent Document 2] International Patent Publication No. 2015 / 124197 Summary of the Invention [Problem to be solved by the invention]

[0006] In view of the above, the object of the present invention is to provide an intuitively applicable solution for everyday shopping for visually impaired or visually impaired people, which solution does not require the purchase, operation and maintenance of dedicated equipment. [Means for solving the problem]

[0007] This problem is solved by a device according to claim 1. The subject of the invention is therefore an electronic price and / or product information display device, referred to in short as display device, configured to visually reproduce price and / or product information of products and configured to store reproduction data representative of price and / or product information, the display device comprising a first near-field communication device, in particular a first near-field communication device, referred to in short as NFC device, configured in particular for short-range contactless communication with a reproduction device having NFC functionality, the reproduction data comprising display reproduction data defined for the visual reproduction of the price and / or product information by the electronic price and / or product information display device itself and output reproduction data that are output by means of short-range contactless communication to the reproduction device enabling an acoustic reproduction of the price and / or product information by the reproduction device thereon, the near-field communication device being configured to access the output reproduction data and to output the output reproduction data in the course of the short-range contactless communication.

[0008] This problem is also solved by the claims 7The problem is therefore solved by a playback device according to the invention, which comprises a playback stage adapted to acoustically play back information represented in digital form, characterized in that a second near-field communication device, in particular a second Near-Field Communication device, for short, called NFC device, is provided which is adapted for short-range contactless communication with a display device, in particular an electronic price and / or product information display device having NFC functionality, the second near-field communication device being adapted to query and receive output playback data in the course of short-range contactless communication with the electronic price and / or product information display device, the output playback data enabling the acoustic playback of price and / or product information on the playback device, and the second near-field communication device is adapted to provide the output playback data, in particular provided as playback data in audio and / or text format, for the acoustic playback of price and / or product information by the playback stage.

[0009] The measures according to the present invention have the advantage that the acoustic information necessary for blind or visually impaired persons to identify products is available with significantly less effort than with known dedicated devices. This eliminates the need for the costly process of capturing an image of a product, evaluating the image until text in the captured image is recognized, and generating acoustic information from the recognized text. This process, however, presupposes that the person is able to pick up the product or at least aim the optics of the dedicated device at the product to be photographed. The risk of incorrectly recognizing image content, which is always present in the context of computer-aided image evaluation, such as fundamental recognition errors or attention to irrelevant image areas in the captured image, is also completely eliminated, since image recognition or evaluation as such is completely unnecessary. Instead, the playback data required for acoustic playback by the playback device is retrieved directly from the display device. Therefore, in a system of electronic price and / or product information display devices in which the allocation of products to display devices is known, also known in technical terms as an "Electronic Shelf Label System" (ESL system for short), incorrect provision of information required for acoustic playback is also impossible.

[0010] That is, product information inquiries for visually impaired or visually impaired people can be performed directly on the display device by simply bringing a commercially available playback device, such as a smartphone or tablet computer, within the wireless range of the display device, i.e., nearby. The commercially available device only needs a communication device that is compatible with the first short-range communication device. Once short-range contactless communication is established, an application, i.e., software running on the playback device, can be used to receive the playback data for the inquiry, and the price information and / or product information carried or referenced thereby can be acoustically reproduced by the playback device, in particular, immediately, i.e., substantially without delay.

[0011] That is, the person can concentrate on locating the display device in the store, which is usually mounted at the very front of the rack. This dramatically eases the shopping process, especially for a visually impaired person who must feel around along the rack rails to find the display device. And, because availability information about the product, i.e., the number of products still on the rack or the product that is just missing from the rack, can also be provided using the output playback data, whether the product is actually there or out of stock is even more irrelevant at this point to searching and finding the display device.

[0012] These output playback data are generated exclusively from user data, but may advantageously be embedded in or generally combined entirely with control and / or management data commonly used in communications technology, i.e., they are usually transmitted together with the control or management data. It should further be mentioned that these output playback data may exist in or be composed of two different data structures:

[0013] The first data structure may be a reference or citation, which enables the playback device to access the information represented by the playback data, particularly the playback data in text and / or audio form, within the store's local data network, for example via a customer WLAN, or via the Internet, and acoustically play back the information transmitted therewith.

[0014] The second data structure may be audio and / or textual playback data encoded in a commonly used data format, so that a playback device can acoustically play back the information directly transmitted with the data. In some cases, this second data structure may exist in a compressed form to allow optimal use of the available bandwidth of the near-field contactless communication. In particular, since a minimal data budget is expected for NFC wireless communication, it may be advantageous to limit the second data structure to textual playback data.

[0015] Optionally, a combination of the first and second data structures can also be used, so that, for example, a product description can be started directly on the playback device or using the immediately playable second data structure (audio or text playback data) (since acoustic playback of product information takes time), while in the background, the first data structure (e.g., references) can be used to load additional data from a server via the store's WLAN or the Internet, and, following the playback of the information transmitted by the audio or text playback data, start the playback of citation information, which may be significantly more comprehensive than the information provided using the second data structure.

[0016] Depending on the implementation of the present invention, the short-range communication device may be, for example, a Bluetooth-type low-energy wireless communication device with a short range (for example, up to a maximum of 50 cm or less, down to a few cm), or a wireless communication device that enables wireless communication within a range of more than a few cm (approximately 4 cm or less) based on short-range communication specifications (ISO / IEC18092, ECMA340, or ISO / IEC21481 / ECMA352). To prepare for situations where playback devices that support only one technology or only the other technology are used, the display device may use both wireless technologies.

[0017] Commercially available playback devices (smartphones and tablet computers) today are usually equipped with hardware capable of utilizing NFC technology themselves, and in order to accommodate the present invention, the received query playback data or the data referenced therefrom must be available for the playback stage of the playback device which performs the acoustic output of the information obtained and the information defined for the acoustic playback.

[0018] In short, a solution is provided for the everyday shopping of visually impaired or visually impaired people, in which, on the one hand, the information to be acoustically reproduced or a reference to that information is provided in a location-specific manner at each display location by the operator of the display system in the store, and, on the other hand, on the customer side, commercially available reproduction equipment is used, which is already in widespread circulation and whose operation is routine for visually impaired people and even for visually impaired people, i.e., the reproduction equipment of the exemplary example already mentioned here can usually even be controlled using voice commands or by human-machine dialogue.

[0019] Further particularly advantageous embodiments and refinements of the invention emerge from the dependent claims and the following description.

[0020] The inventions considered herein are primarily applied in the context of ESL systems, where a plurality of such displays, i.e., ESLs, exist for respectively visualizing price and / or product information for displayed products or groups of products.

[0021] Such an ESL can be fixed to a rack or rack rail. That is, the fixed structure for placing the ESL and the product can be configured as a rack or rack rail. It can also be fixed to a display table. That is, the fixed structure can be configured as a table. Such an ESL can also be configured as a display screen placed on a table. Such a display device can also be configured as a tag that is fixed to clothing. That is, the fixed structure can also be configured as a connecting member between the electronic display device and the product. That is, such a fixed structure can be realized, for example, as a sewing thread, a needle, or the like. To simplify the discussion of the present invention, the following will always refer to an ESL fixed to a rack, specifically a rack rail, in connection with a display device.

[0022] For the purpose of information visualization, each ESL is equipped with a screen, usually realized as an electrophoretic screen in the case of battery-powered ESLs, to enable operation as energy-saving as possible. However, other types of screens can also be used, such as LCD screens or equivalent. These ESLs can also be provided with electricity and / or communication technology in wired form. This power supply can be achieved using a wiring structure integrated into the rack rails, the power supply wiring being connected using contacts on the ESL. Power supply can also be achieved using radio signals, which is known in technical terms as "Power over WIFI".

[0023] In a radio-based ESL system, there are also multiple access points that constitute an ESL radio network, providing communication technology for addressing and transmitting data to and retrieving data from the ESLs. Typically, a group of ESLs is assigned to an access point through which they can be addressed and called. This assignment works by selecting a suitable radio channel in the ISM band used by each access point and registering the ESL with the access point responsible for that ESL. For this wireless communication, standardized wireless protocols such as ZigBee (registered trademark) or Bluetooth (registered trademark) can be used. Proprietary wireless protocols, such as the protocol disclosed in U.S. Patent No. 5,629,492, can also be used.

[0024] These access points themselves are typically connected via a hardwire to a central control device, such as a local server running a software application (e.g., shop management software) that manages and controls the ESL, or they may be controlled via a cloud-based software application that provides the aforementioned functionality for managing and controlling the ESL.

[0025] However, the display device can also be configured as a so-called video rack rail, in which case a high-resolution screen is typically used to display video content. The video rack rail can be fixed to the rack rail or can be used instead of the rack rail. When configured as a video rack rail, the power supply to the display device, as well as the provision of communication technology, is usually wired using electrical wiring. In the case of a video rack rail, it usually extends along a product group. In this case, price information and / or product information available using the output playback data can be related to the entire product assigned to the video rack rail. In this case, the information content can include the number of different products, the order of the products along the video rack rail, and the spatial discoverability of the products along the video rack rail, thereby making it possible to guide visually impaired people, even those who are blind, in a completely acoustically guided manner.

[0026] As mentioned above, the display device is a component of an ESL system. In such an ESL system, the display device communicates with a central control device, which may be configured by a central server or cloud-based software, and thereby obtains data specified for displaying price and / or product information. For this purpose, the display device is equipped with a system communication device, distinct from the short-range communication device, arranged to communicate with the control device responsible for providing the playback data. Advantageously, this system communication device is used to obtain and store the playback data for output in a memory stage of the display device. These data are stored there in a ready-to-use state, i.e., ready to be processed by the playback device after immediate output from the display device.

[0027] The acoustic reproduction of the associated information content using the output playback data can be implemented in two ways in the playback device. The playback stage can be configured to directly reproduce the audio playback data represented by the output playback data. This allows the output playback data to be used to provide one or more audio files, which is preferred when there is sufficient bandwidth available during short-range contactless communication. Alternatively, it can be provided that the playback device converts the text playback data represented by the output playback data into audio playback data, and the playback stage is configured to directly reproduce the audio playback data thus obtained. This is preferred when there is very limited bandwidth available during short-range contactless communication. In this case, a very compact text file with as little data volume as possible can be transmitted, which is synthesized into sound by known software methods in the playback device and then acoustically reproduced.

[0028] The output of the playback data for output can be performed in principle immediately after the connection with the playback device is established. However, since other devices can or should also use a short-range communication device, it has proven advantageous for the display device to be configured to detect an output playback data query command receivable in the course of short-range contactless communication, and possibly distinguish it from at least one other query command, and to output the output playback data only after detecting the output playback data query command using short-range contactless communication. Therefore, the playback device is advantageously configured to output the output playback data query command via the second short-range communication device in order to start receiving the output playback data in audio and / or text form.

[0029] This allows the output of the playback data for output to be selectively initiated as desired, so that these data are only output under command control to a playback device previously configured by the user of the data, for example, to assist visually impaired or blind people. This can be done, for example, during the installation of the application / software deployed for assistance. On the other hand, it can also be done at the operating system level, for example, if the operating system allows a setting that allows the user to identify themselves as visually impaired or even blind. Therefore, it has proven advantageous for the playback device to be further configured to output a playback data inquiry command via the second short-range communication device in order to start receiving the playback data for output (either essentially or only after its configuration).

[0030] However, the present invention can also be used to further assist visually impaired or visually impaired people during their shopping experience. It has further been found that it is advantageous for the output playback data to include product ambience description data that describes the product ambience of the product and enables a playback device to acoustically play back a description of the product ambience. Therefore, the playback device is advantageously configured to receive, as a component of the output playback data or as reference information within the output playback data, the product ambience description data that describes the product ambience of the product.

[0031] This product surroundings description data can also be provided in a format similar to the two data structures discussed above. They also serve to describe the surroundings around the display device, i.e., the surroundings of the product assigned to the display device, when the playback device plays back the product acoustically. For example, the playback device can inform the user of the products they can find to the left and right of the product they are currently looking at. This relates to each rack surface. However, this measure can also describe the products located on the rack surfaces above and / or below it, so that a visually impaired person, or even a visually impaired person, can receive an intuitive (acoustic) overview of the contents of the rack, starting from the product for which the description information is being played. In this case, the product referred to via short-range contactless communication constitutes a central target point for the person to further orient themselves around the rack.

[0032] To facilitate a user of the playback device considering the acoustically described ambience of a product, the playback device can be configured to play a section-by-section product ambience description of a product represented or referenced using product ambience description data, particularly as a result of detected user interaction. In this manner, the playback device can acoustically indicate, for example, that a product, such as a brand name XYZ toothbrush, is accompanied by another brand name ABC toothbrush. The acoustic ambience description then pauses until the user issues a command, such as "continue," which is detected or sensed by the playback device, at which point the playback device acoustically announces, "There is toothpaste to the right of the product, and the rack of oral hygiene products ends to the left of the product." The playback device then waits for a command to continue the ambience description again. It should be noted here that the product ambient environment description data can naturally also describe the area diagonally adjacent to the product, so that the description of the ambient environment can be carried out continuously, and the person can be given an overall picture of the ambient environment around the product by acoustic means. Furthermore, it can be stipulated that the description of the ambient environment can be stopped at any time by a voice command such as "enough".

[0033] In today's commonly used smartphones, user interaction is considered to be a standard feature in the context of voice recognition, the results of which are then used to selectively, i.e., section by section, reproduce information that matches the description of the surrounding environment. Naturally, gesture recognition can also be used, which is particularly advantageous when the person involved is not only visually impaired, but also has a speech impairment, or simply prefers this option.

[0034] Otherwise, gesture recognition can be advantageously utilized in the context of the present invention in two ways: in a first configuration of the playback device, an acceleration sensor in the playback device can be used to detect directional movements of the playback device, and this detected direction can be used to select an acoustically played-back environment description adapted to this direction; in a second configuration, an optical sensor (e.g., a camera system in the playback device) can be used to detect and evaluate the person's gestures, obtain directional information that can be derived therefrom, and play an environment description adapted to this direction.

[0035] Furthermore, it has proven to be highly advantageous if the display device is configured to detect a route description query command receivable in the course of short-range contactless communication, which route description query command serves to query a route description to another product, which product is characterized by product data contained in the route description query command, said route description query command originating from a playback device configured to output the route description query command via a second short-range communication device in order to initiate the reception of playback data for output which includes the route description data.

[0036] The other product may be communicated by a user interaction, such as a spoken question (e.g., "Where are the organic petit breads?") or a voice command (e.g., "Navigate me to the organic petit breads.") In this context, voice recognition of the playback device is also utilized to identify the other product and obtain product data representative of the other product.

[0037] However, other items may also be extracted from a shopping list digitally stored on the playback device, for example, where the items listed therein are processed sequentially (in order). In this case, thematic groupings of the shopping list (e.g., apples and pears belonging to the thematic group Fruits and Vegetables, and ham and steaks belonging to the thematic group Meats) may also be present on the playback device, so that the route instructions provided may direct the person to a route optimized for the thematic group.

[0038] Advantageously, the display device is configured, after detecting a route description query command, to provide output playback data including route description data and to output the route description data via the short-range communication device, the route description data being defined for acoustically playing on the playback device a description of the route to another product, while still taking into account the idea that the output of the route description data should be controlled solely by command, i.e. initiated in a manner determined by the playback device, thereby enabling the individual navigation wishes of the playback device user to be accommodated.

[0039] In this case, similar to the audio playback description of the product environment, the playback device may be configured to play a section-by-section description of a route to another product, represented or referenced using the route description data, specifically as a result of the detected user interaction. This section-by-section playback allows a visually impaired person, or even a blind person, to be directed or navigated along a route section-by-section without having to memorize the entire route from scratch; that is, the person progresses along the route while receiving section-by-section navigation instructions.

[0040] This can be accomplished by providing the person with a first acoustic instruction to move to the end of the rack to the person's right, and only when the person confirms that they have reached this waypoint by user interaction with the playback device (e.g., a voice saying "I have reached the waypoint") is a second acoustic instruction given to the person to turn right and take 18 steps down the rack aisle leading to that point.

[0041] In this case, technical measures or standardized size information are intentionally not taken into account, since this would be impossible if the person were to move. Rather, the playback device, in the process of creating a user profile, can determine in advance what size the person's average stride is, and then incorporate this knowledge into the person's specific step size information in order to enable the person to perform navigation instructions as naturally as possible in a surrounding environment defined, for example, in meters.

[0042] When the person confirms arrival at the next waypoint through user interaction with the playback device, a third acoustic prompt is given that provides the person with the information to finally find another item on the middle rack face of the left rack.

[0043] Now, to get the correct orientation on the rack, the person can bring the playback device within the wireless range of the display device installed there and use the display device to acoustically obtain information about the described product. Similarly, the previously considered acoustic description of the surrounding environment of the product can be performed, which makes it easier to find other products on the rack.

[0044] The invention allows a visually impaired person, or even a blind person, to obtain information about products within his reach in a store by means of a display device installed there, and even provides the person with the means to move "step by step", i.e. gradually along easily walkable sections, to the (different) product he is looking for, in which case a commercially widely available smartphone or tablet computer is used on his side as user device (in the sense of a playback device) that ESL uses in the store for the purpose of acoustically playing back information.

[0045] However, the present invention can also be used to verify the effectiveness of marketing measures relating to the person in question. For example, the output playback data can be used to send a marketing or advertising message encouraging the purchase of a specific product. During the short-range contactless communication, the identifier of the playback device, and possibly even the person's identifier, can be ascertained, which can be reported to the central control device of the ESL system and stored there for management purposes, so that it can then be investigated, when paying at the POS terminal, whether the person used the marketing or advertising message to purchase the product advertised thereby.

[0046] It should be further noted here that the aforementioned order of commands sent to the display device need not be observed. Rather, the commands described herein can be sent individually and independently of one another to the display device and processed separately by the display device. Thus, for example, a query to identify nearby products can be performed in any ESL, resulting in output playback data consisting solely of product-surrounding-environment description data. Furthermore, a query for route information to another product can be performed separately from a query for specific product information or separately from a query for the product's surroundings, resulting in output playback data consisting solely of product-surrounding-environment description data. That is, commands can be used independently of one another by both the playback device and the display device.

[0047] In summary, the term "product information" can be used in a variety of ways, depending on the context: Whether this information is about the product that was just inquired about to ESL via short-range contactless communication, This information is information about the surrounding environment of the product that was just queried to the ESL through short-range contactless communication, or This information is information about the route from the product that was just inquired about to the ESL via short-distance contactless communication to another product. I would like to further state that this should be understood as follows.

[0048] Finally, it should be noted that, generally speaking, the electronic devices discussed herein (e.g., ESLs, smartphones, tablet computers, video racks, etc.) or their modules naturally include electronic components. These electronic components can be composed of discrete electronic components, integrated electronic components, or a combination thereof. Microcomputers, microcontrollers, and application-specific integrated circuits (ASICs) can also be used, in some cases, in combination with analog or digital peripheral electronic components. Many of the aforementioned functions of these devices are realized by software running on the processors of the electronic components (in some cases, in cooperation with hardware components). Devices configured for wireless communication typically include antenna structures for transmitting and receiving wireless signals as part of a transceiver module. For short-range, contactless communication, in addition to the aforementioned antenna structures, coil structures or capacitor plate structures for inductive coupling can also be used. Furthermore, these electronic devices can include an internal power source, which can be realized, for example, by a replaceable or rechargeable battery. These devices can also be powered in a wired configuration by an external power grid or using "Power over LAN."

[0049] These and other aspects of the present invention will become apparent from the drawings discussed below.

[0050] The invention will now be explained in more detail again by way of example with reference to the accompanying drawings, in which the same elements in the different drawings are given the same reference numerals, and in which the invention is not limited thereto. [Brief explanation of the drawings]

[0051] [Figure 1] Schematic diagram of a rack equipped with an ESL for examining two aspects of the present invention. [Figure 2] Block wiring diagram of the rack-mounted ESL [Figure 3] Block diagram of a smartphone used as a playback device when considering the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0052] FIG. 1 illustrates a rack 1 consisting of three rack bottoms, namely, an upper rack bottom 2, a middle rack bottom 3, and a lower rack bottom 4, each of which terminates at a rack rail 5 at its front edge. Different products P1-P9 are installed on the rack bottoms 2-4, and in this case, they are distinguishable by their shapes. Electronic rack displays E1-E9, abbreviated as ESL-E1-E9 (based on the initials of the English term "Electronic Shelf Label"), are attached to the rack rail 5 in correspondence with the positions of the products P1-P9. Furthermore, FIG. 1 illustrates a central server 29 that manages the ESL-E1-E9. This server 29 is connected to an access point 30 via a cable connection. This access point 30 provides an ESL wireless network for wireless communication with the ESL-E1-E9.

[0053] In FIG. 2, the first ESL-E1 is shown as a representative of all the installed ESLs E1 to E9, or more precisely, its electronic block diagram. In the explanation of FIG. 2, the symbol E1 is omitted and only the abbreviation ESL is used.

[0054] The ESL comprises an electrophoretic display module 6 adapted to visually reproduce price and / or product information of one of the products P1-P9 in an energy-saving manner.

[0055] The ESL-E1 further comprises a first radio module 7 configured for ESL wireless communication in the ESL wireless network referred to in the general description. The ESL wireless communication of this first radio module is based on the propagation of electromagnetic waves, and therefore an antenna structure 7A is displayed within the first radio module 7 as an interface between the electronics (not shown in detail) of the first radio module 7 and the surrounding environment. Via this ESL wireless network, the ESL obtains display reproduction data AW1 representing price information and / or product information for each of the products P1 to P9. In this case, these display reproduction data AW consist of image files containing digitally coded image information that are output via an internal data bus 9 to the display module 6, where they are visualized and reproduced by this module on its electrophoretic screen (not shown in detail).

[0056] The ESL further comprises a second wireless module 8, which is configured and arranged to communicate by NFC wirelessly with a device also equipped with NFC technology that is in close proximity, within an NFC wireless communication range 10, typically of a few centimetres in any case. In the context of this discussion, this device is a smartphone 11 equipped with NFC technology (see Figures 1 and 3), which is used as the playback device. This NFC wireless communication is based on inductive coupling with this other device, and therefore a coil structure 8A is represented within the second wireless module 8 as an interface between the electronics (not shown in detail) of the second wireless module 8 and the surrounding environment.

[0057] It is noted here that the electronic components of these modules 6, 7, 8 each include a microcontroller or microprocessor which, in some cases in cooperation with other electronic components or parts, provides the functionality of each module 6 to 8. It is further noted that a central control module may be provided which coordinates or controls the cooperative operation of the individual modules 6 to 8.

[0058] In order to facilitate the shopping of visually impaired or visually impaired persons in the store where Rack 1 is located, the following data is also transmitted to the ESL during the progress of the aforementioned ESL wireless communication, namely: a) Output playback data AW2: This data describes price and / or product information in a completely general sense. In this case, it is in plain text (e.g., in ACSII code) and is transmitted to each ESL-E1 to E9 in the form of a text file, where it is stored for retrieval by the playback device. This text description could, for example, be in the form "This product is smooth flour packaged in 500 gram packets" in association with ESL-E1 located in the upper left corner, or more simply, "Flour, smooth, 500 gram packets." This greatly reduces the amount of data, facilitating and accelerating transmission using NFC technology. b) Product ambient environment description data UB: This data describes the ambient environment of each product. In this case, too, this structure is suitable for textual ambient environment information to be transmitted without problems using NFC technology. The textual description of the ambient environment (indicated by arrows U1 to U8) stored in the fifth ELS-E5 in relation to product P5 can be structured as follows, where, of course, the actual product name is quoted in place of the substitution <<product Pn>>, and the actual number of entries in the list depends on the respective position of the product and its ambient environment: "Find <<product P4>> right next to it on the left" (see arrow U1 in Figure 1) "Find <<Product P6>> right next to it" (see arrow U2 in Figure 1) "Find <<Product P7>> to the right of <<Product P6>>" (see arrow U3 in Figure 1) "Find <<product P2>> directly next to it above" (see arrow U4 in Figure 1) "Find <<Product P8>> diagonally to the lower left" (see arrow U5 in Figure 1) "Find <<Product P9>> next to you diagonally to the bottom right" (see arrow U6 in Figure 1) "Find <<Product P1>> diagonally to the upper left" (see arrow U7 in Figure 1) "Find <<Product P3>> diagonally to the upper right" (see arrow U8 in Figure 1)

[0059] These data AW1, AW2 and UB are stored in the ESL (see FIG. 2) in a memory stage 12 of the first radio module 7 and are there ready, i.e. retrieved, for subsequent use by the second radio module 8. In order to avoid the need to start up the first radio module 7 when accessing the data AW2 and UB, it may be advantageous for these data AW2 and UB to be transferred after their reception to the second radio module 8 and stored there, as indicated by the dashed line, in a second memory stage 13 of the second radio module 8. Optionally, these data AW2 and UB may also be stored in a common memory (not shown) to which both radio modules 7 and 8 have access.

[0060] These data AW2 and UB are retrieved by the smartphone 11 during the NFC wireless communication with the smartphone 11 from the ESL involved in the NFC communication (in the context of Fig. 1 this is ESL-E5) and converted there into an audio signal AS and output acoustically, so that a visually impaired person, or even a blind person, can orient themselves towards the rack based on the acoustic instructions.

[0061] In the following, we will consider a smartphone 11 used by a visually impaired person using Figure 3. In this case, it contains a microcontroller 15 that processes the programs (i.e., software) and an operating system 17, as well as software applications 18 that are installed for the smartphone's intended use. 2 It comprises a central electronics component 14 having a PROM memory 16 and a RAM memory 19 for temporary storage of data generated during the programming process.

[0062] Furthermore, the smartphone 11 comprises an audio-visual interface electronics 20 consisting essentially of a touchscreen module 21 for visualizing information and receiving user interaction in the sense of touching the screen, an audio module 22 (amplifier and loudspeaker) for outputting an audio signal AS, and a microphone module 23 for receiving user interaction in the sense of spoken words, etc.

[0063] Furthermore, the smartphone 11 includes a third wireless module 24 configured for NFC wireless communication, in this case for NFC communication with ESL-E1 to E9. Again, for the reasons described in connection with the second wireless module 8, a different coil structure 24A is shown.

[0064] Furthermore, the smartphone 11 comprises an energy supply module 25, a mobile radio module 26 for communicating with a mobile radio operator's cellular network, and further commonly used smartphone components 27, which will not be described in detail.

[0065] Blocks 24, 20 and 14 are connected using an internal data bus 28. However, they can also be integrated on a system-on-chip platform.

[0066] The software application 18 can be conceptually divided into a program code APP and commands that are defined for data query during NFC wireless communication and are also understood by ESL-E1 to E9. In this case, these commands are: An output playback data inquiry command AWB that starts output of output playback data in ESL-E1 to E9 for the purpose of acoustically playing price information and / or product information on the smartphone 11; A route description inquiry command PUB that starts output of output playback data including product surrounding environment description data in ESL-E1 to E9 for the purpose of acoustically playing back a description of the product surrounding environment on the smartphone 11; A route description inquiry command WBB that starts output of output playback data including route description data in ESL-E1 to E9 for the purpose of acoustically playing back a description of a route to another product on the smartphone 11; In this case, information about other products may be obtained, for example, from an electronic shopping list that may also be stored in the software application 18 or from another software application on the smartphone 11 .

[0067] 1, if the person holds the smartphone 11 with the software application 18 activated at the central ESL-E5, an NFC wireless connection is first established, an output playback data inquiry command AWB is sent to the ESL-E5 and detected therein, and as a result, a text file stored therein containing price information and / or product information about product P5 is sent from the ESL-E5 to the smartphone 11 during the NFC wireless communication. The text contained in this text file is converted into an audio format in the smartphone 11, for example by the software application 18, and played as an audio signal AS using the audio module 22.

[0068] Here, the software application 18 also provides a reference to ambient environment information related to the product P5. If the user confirms that this information is also of interest, a route description query command PUB is sent to the ESL-E5 via NFC wireless communication, which detects it. As a result, a text file stored therein containing ambient environment information related to the product P5 (see the above discussion of the product ambient environment description data UB) is sent from the ESL-E5 to the smartphone 11 via NFC wireless communication. The text contained in this text file is converted into audio format by the software application 18 in the smartphone 11 and played as an audio signal AS using the audio module 22. In this case, the audio playback can be stopped after each text sequence, i.e., after the playback of ambient environment information related to one of the directions U1-U8, and can be continued only after receiving a voice command such as "continue." This allows the person time to mentally process the acquired ambient environment information.

[0069] This software application 18 offers the person the option of navigating to another item by acoustically directing them. If the person confirms that they wish to navigate in this manner, a next item (referred to as the "another item") is selected from the electronic shopping list, and the associated item information (i.e., the item's name as text, for example) is sent via NFC wireless communication as a component of a route description query command WBB to the ESL-E5, which detects this command. As a result of detecting this command, the ESL-E5 contacts the central server 29 of the ESL system (generically referred to as the central controller, not shown) via the first wireless module 7, conveying the ESL-E5's identity and the name of the other item being sought. This server 29 contains a digitally stored shelf planogram from which it can read the locations of all items in the store and, consequently, the locations of all ESLs assigned to those items. A route through the store from the actual position of the ESL-E5 to another product, possibly located in another rack aisle further away, is then calculated and sent as route description data WB in the form of a text sequence sequentially describing the route between the route points via the first wireless module 7 to the ESL-E5, where it is possibly temporarily stored and from there sent via the second wireless module 8 to the smartphone 11. Here, the smartphone 11 again converts the text into synthetic speech, whereby navigation instructions (i.e. individual instruction sequences) are played out sequentially between the route points, activated under voice control, so that the person is guided between the route points stress-free and given enough time to travel.

[0070] That is, the present invention enables both intuitive acoustic provision of local information about products on shelves and acoustically guided navigation within a store, with the in-store ESL infrastructure providing the basis for reliable and error-free provision of these functions. Since the location of the installed ESLs is always known in this ESL system, a person can start the acoustically guided navigation process anew at any ESL at any time, even if the person accidentally deviates from the acoustically defined route, i.e., gets lost. This provides safety and reliability when applying this technology.

[0071] Finally, it should be pointed out once again that the figures detailed above are merely examples that can be modified in various ways by those skilled in the art without departing from the scope of the invention, and for the sake of completeness, it should also be pointed out that the use of the indefinite article "a" does not exclude that the feature in question can be present several times. The present application relates to the invention described in the claims, but also includes the following as other aspects. 1. an electronic price and / or product information display device, referred to as display device (E1-E9) for short, configured to visually reproduce price and / or product information of products (P1-P9) and configured to store reproduced data representing the price and / or product information; In particular, a display device (E1 to E9) equipped with a first near-field communication device (8), in particular a first Near-Field-Communication device abbreviated as NFC device, configured to perform short-distance contactless communication with a playback device (11) having an NFC function, These playback data are Display reproduction data (AW1) defined for visual reproduction of price information and / or product information by the electronic price information and / or product information display device (E1 to E9) itself; output reproduction data (AW2) provided as reproduction data in particular in audio and / or text format, which is output to a reproduction device (11) using short-distance contactless communication and enables the reproduction device (11) to acoustically reproduce the price information and / or product information thereon; and The display device is characterized in that the near-field communication device (8) is configured to access the output playback data (AW2) and output the output playback data (AW2) in the process of near-field contactless communication. 2. In the display device (E1 to E9) of 1 above, The display device (E1 to E9) is provided with a system communication device (7) different from the short-range communication device (8) and arranged to communicate with a control device (29) that serves to provide playback data (AW1, AW2), and the playback data (AW2) for output is acquired using the system communication device (7) and stored in memory stages (12, 13) of the display device (E1 to E9). 3. In the display device (E1 to E9) of 1 or 2 above, A display device configured to detect an output playback data inquiry command (AWB) that can be received during short-range non-contact communication, and in some cases distinguish it from at least one other inquiry command, and output output playback data (AW2) only after detecting the output playback data inquiry command (AWB) using this short-range non-contact communication. 4. In any one of the display devices (E1 to E9) 1 to 3 above, A display device in which the output playback data (AW2) has product surrounding environment description data, which describes the product surrounding environment of the product and enables acoustic playback of a description of the product surrounding environment on a playback device (11). 5. In any one of the display devices (E1 to E9) 1 to 4 above, The display device is configured to detect a route description query command (PUB) that can be received in the course of short-range contactless communication, and the route description query command (PUB) serves to query a route description to another product (P1 to P9), and the other product (P1 to P9) is characterized by the product data included in the route description query command (PUB). 6. In the above five display devices (E1 to E9), The display device is configured to provide output playback data (AW2) including route description data (WB) after detecting a route description inquiry command (PUB) and output the route description data (WB) via a short-range communication device (8), and the route description data (WB) is defined to acoustically play back a description of the route to another product (P1 to P9) on a playback device (11). 7. In the above six display devices (E1 to E9), A display device equipped with the system communication device (7) of the above 2, and configured to query output playback data (AW2) including route explanation data (WB) from a control device (29) based on the output of product data of other products (P1 to P9). 8. A reproduction device (11) comprising a reproduction stage (20) adapted to acoustically reproduce information represented in digital form, In particular, a second short-range communication device (24), in particular abbreviated as NFC device, is provided, which is configured for short-range contactless communication with the electronic price information and / or product information display device, in particular abbreviated as display device (E1-E9), having NFC functionality; the second short-range communication device (24) is configured to query and receive output playback data (AW2) in the course of short-range contactless communication with the electronic price and / or product information display devices (E1-E9), which output playback data (AW2) enable the playback device (11) to acoustically play back the price and / or product information; The second short-range communication device (24) is configured to provide output reproduction data (AW2) provided as reproduction data in audio and / or text format for acoustic reproduction of price information and / or product information by the reproduction stage (20). 9. In the playback device (11) of the above 8, A playback device in which the playback stage (20) is configured to directly play back the audio format playback data represented by the output playback data (AW2). 10. In the playback device (11) of the above 8 or 9, This playback device (11) is configured to convert the text format playback data represented by the output playback data (AW2) into audio format playback data, and the playback stage (20) is configured to directly play back the audio format playback data thus obtained. 11. In any one of the playback devices (11) of 8 to 10 above, The playback device is configured to output an output playback data inquiry command (AWB) via a second short-range communication device (24) for the purpose of initiating reception of output playback data (AW2). 12. In any one of the playback devices (11) described above in 8 to 11, A playback device configured to receive product surroundings description data (UB) describing the product surroundings of products (P1 to P9) as a component of output playback data (AW2) or as reference information within the output playback data (AW2). 13. In the above 12 playback devices (11), In particular, a playback device configured to play back a description of the product surroundings of the products (P1 to P9) in sections, which is represented or referenced using product surroundings description data (UB) as a result of detecting interaction with the user. 14. In any one of the playback devices (11) of 8 to 13 above, The playback device is configured to output a route description query command (PUB) via a second short-range communication device (24) for the purpose of initiating reception of output playback data (AW2) including route description data (WB). 15. In the above 14 playback devices (11), In particular, a playback device configured to play back, as a result of detecting interaction with a user, a description of a route to another product (P1-P9) in sections, which is represented or referenced using route description data (WB).

Claims

1. 1. An electronic price and / or product information display device, referred to as display device (E1-E9) for short, configured to visually reproduce price and / or product information of products (P1-P9) and configured to store reproduction data representative of the price and / or product information, In particular, a display device (E1 to E9) equipped with a first near-field communication device (8), in particular a first Near-Field-Communication device abbreviated as NFC device, configured to perform short-distance contactless communication with a playback device (11) having an NFC function, These playback data are Display reproduction data (AW1) defined for visual reproduction of price information and / or product information by the electronic price information and / or product information display device (E1 to E9) itself; output reproduction data (AW2) provided as reproduction data, in particular in audio and / or text format, which is output to a reproduction device (11) using short-distance contactless communication and enables the reproduction device (11) to acoustically reproduce price information and / or product information thereon; and The short-distance communication device (8) is configured to access the playback data for output (AW2) and output the playback data for output (AW2) in the process of short-distance contactless communication; the output reproduction data (AW2) has product surroundings description data, which describes the product surroundings of the product and enables acoustic reproduction of a description of the product surroundings on a reproduction device (11); A display device characterized by:

2. In the display device (E1 to E9) according to claim 1, The display device comprises a system communication device (7) different from the short-range communication device (8) and arranged to communicate with a control device (29) that serves to provide playback data (AW1, AW2), and the playback data for output (AW2) is acquired using the system communication device (7) and stored in memory stages (12, 13) of the display device (E1 to E9).

3. In the display device (E1 to E9) according to claim 1 or 2, A display device configured to detect an output playback data inquiry command (AWB) that can be received during short-range non-contact communication, and in some cases distinguish it from at least one other inquiry command, and output output playback data (AW2) only after detecting the output playback data inquiry command (AWB) using this short-range non-contact communication.

4. In the display device (E1 to E9) according to any one of claims 1 to 3, The display device is configured to detect a route description inquiry command (PUB) that can be received in the course of short-range contactless communication, and the route description inquiry command (PUB) serves to inquire about a route description to another product (P1 to P9), and the other product (P1 to P9) is characterized by the product data included in the route description inquiry command (PUB).

5. In the display device (E1 to E9) according to claim 4, The display device is configured to provide output playback data (AW2) including route description data (WB) after detecting a route description inquiry command (PUB) and output the route description data (WB) via a short-range communication device (8), the route description data (WB) being determined to acoustically play back a description of the route to another product (P1-P9) on a playback device (11).

6. In the display device (E1 to E9) according to claim 5, A display device comprising the system communication device (7) according to claim 2, and configured to query output playback data (AW2) including route explanation data (WB) from a control device (29) based on output of product data of other products (P1 to P9).

7. A reproduction device (11) comprising a reproduction stage (20) adapted to acoustically reproduce information represented in digital form, a second short-range communication device (24), in particular the second short-range communication device, for short, referred to as NFC device, is provided, which is configured for short-range contactless communication with the electronic price and / or product information display devices, for short, referred to as display devices (E1-E9), in particular having NFC functionality; said second short-range communication device (24) being adapted to query and receive output reproduction data (AW2) in the course of short-range contactless communication with the electronic price and / or product information display devices (E1-E9), said output reproduction data (AW2) enabling said reproduction device (11) to reproduce the price and / or product information acoustically; the second short-range communication device (24) is adapted to provide output reproduction data (AW2), in particular provided as reproduction data in audio and / or text format, for acoustic reproduction of price information and / or product information by the reproduction stage (20), The output reproduction data (AW2) is configured to receive, as a component of the output reproduction data (AW2) or as reference information within the output reproduction data (AW2), product surroundings description data (UB) that describes the product surroundings of the products (P1 to P9). The playback device is characterized by the above.

8. A playback device (11) according to claim 7, A playback device in which the playback stage (20) is configured to directly play back the audio format playback data represented by the output playback data (AW2).

9. A playback device (11) according to claim 7 or 8, The playback device (11) is configured to convert the text format playback data represented by the output playback data (AW2) into audio format playback data, and the playback stage (20) is configured to directly play the audio format playback data thus obtained.

10. A playback device (11) according to any one of claims 7 to 9, The playback device is configured to output an output playback data inquiry command (AWB) via a second short-range communication device (24) in order to start receiving output playback data (AW2).

11. A playback device (11) according to claim 7, In particular, the playback device is configured to play back a description of the product surroundings of the products (P1 to P9) in sections, which description is expressed using or referenced in product surroundings description data (UB) as a result of detecting an interaction with a user.

12. A playback device (11) according to any one of claims 7 to 11, The playback device is configured to output a route description query command (PUB) via a second short-range communication device (24) for the purpose of initiating reception of output playback data (AW2) including route description data (WB).

13. A playback device (11) according to claim 12, In particular, a playback device configured to play back, as a result of detecting an interaction with a user, a description of a route to another product (P1 to P9) in sections, which is represented or referenced using route description data (WB).

Images