Devices to assist the visually impaired or visually impaired when shopping

Abstract

An electronic price information and / or product information display device, abbreviated as display device, configured to visually reproduce product price information and / or product information and configured to store reproduction data representative of price information and / or product information, the display device comprising a first near-field communication device, in particular a first Near-Field-Communication device, abbreviated as NFC device, configured in particular for short-range contactless communication with a reproduction device having NFC functionality, the reproduction data comprising display reproduction data defined for visual reproduction of the price information and / or product information by the electronic price information and / or product information display device itself and output reproduction data provided in particular as reproduction data in audio and / or text format for output to the reproduction device by means of short-range contactless communication and enabling acoustic reproduction of the price information and / or product information by the reproduction device therein, the short-range communication device being configured to access the output reproduction data and to output the output reproduction data in the course of the short-range contactless communication.

Description

[Technical field]

[0001] The present invention relates to a device for assisting visually impaired or visually impaired people when shopping. [Background technology]

[0002] Nowadays, in stores, product information and / or price information is displayed on electronic display units, especially electronic shelf labels, technically called "Electronic Shelf Labels" (ESL for short). Visually recognizing such information is a big challenge for the visually impaired and is impossible for the visually impaired.

[0003] Today, highly developed and relatively expensive dedicated devices are commercially available, such as the device developed by the company "OrCam Technologies Ltd." and sold under the name "OrCam MyEye" (https: / / www.orcam.com / en / myeye2 / ) (see also Patent Document 1), which converts optically captured images into text and reads it to its user.

[0004] However, visually impaired people are often economically disadvantaged, which can make the purchase of such specialized equipment difficult or even impossible, and for such people, operating additional specialized equipment is another daily challenge that is certainly not desirable as such. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] European Patent Publication No. 2490155 [Patent Document 2] International Patent Publication No. 2015 / 124197 Summary of the Invention [Problem to be solved by the invention]

[0006] In view of the above, the object of the present invention is to provide an intuitively applicable solution for everyday shopping for visually impaired or visually impaired people, which solution does not require the purchase, operation and maintenance of dedicated equipment. [Means for solving the problem]

[0007] This problem is solved by a device according to claim 1. The subject of the invention is therefore an electronic price information and / or product information display device, referred to for short as display device, configured for visually reproducing price information and / or product information of products and configured for storing reproduction data representative of price information and / or product information, characterized in that the display device comprises a first near-field communication device, in particular a first Near-Field-Communication device, referred to for short as NFC device, configured for near-field contactless communication with a reproduction device having NFC capabilities, the reproduction data comprising display reproduction data defined for the visual reproduction of the price information and / or product information by the electronic price information and / or product information display device itself and output reproduction data which are output by means of near-field contactless communication to the reproduction device enabling an acoustic reproduction of the price information and / or product information there by the reproduction device, and in that the near-field communication device is configured for accessing the output reproduction data and for outputting the output reproduction data in the course of the near-field contactless communication.

[0008] The problem is also solved by a reproduction device according to claim 8. The invention therefore relates to a reproduction device with a reproduction stage adapted for acoustically reproducing information represented in digital form, characterized in that a second near-field communication device, in particular a second Near-Field-Communication device, for short, NFC device, is provided, adapted for short-range contactless communication with a display device, in particular an electronic price information and / or product information display device having NFC functionality, the second near-field communication device being adapted to query and receive output reproduction data in the course of short-range contactless communication with the electronic price information and / or product information display device, the output reproduction data enabling an acoustic reproduction of the price information and / or product information on the reproduction device, the second near-field communication device being adapted to provide output reproduction data, in particular provided as audio and / or textual reproduction data, for the acoustic reproduction of the price information and / or product information by the reproduction stage.

[0009] The measures according to the invention have the advantage that the acoustic information required for a blind or visually impaired person to identify an article is made available at a significantly reduced expense compared to known dedicated devices, i.e. the burdensome process of taking an image of the article, evaluating the image until a text in the taken image is recognized and generating acoustic information from the recognized text is not necessary. The disadvantage is that this process presupposes that the person is able to pick up the article or at least direct the optics of the dedicated device towards the article to be photographed. The risk of incorrectly recognizing the image content, which is always present in the context of computerized image evaluation, such as fundamental recognition errors or attention to irrelevant image areas of the taken image, is also completely eliminated, since image recognition or image evaluation as such is indeed completely unnecessary. Instead, the reproduction data required for acoustic reproduction by the reproduction device are taken directly from the display device. In this way, in a system of electronic price and / or product information display devices, in which the allocation of the articles to the display devices is known, also referred to in technical terms as "Electronic Shelf Label System", or ESL system for short, incorrect provision of the information required for acoustic reproduction is also not possible.

[0010] That is, the inquiry of product information for the visually impaired or the blind is performed directly on the display device, for which it is only necessary to bring a commercially available reproduction device, such as for example a smartphone or a tablet computer, within the radio range of the display device, i.e. close to it. This commercially available device only needs a communication device that is compatible with the first short-range communication device. Once the short-range contactless communication has been established, the inquiry reproduction data can be received by means of an application, i.e. software running on the reproduction device, and the price information and / or product information carried thereby or referenced therewith is in particular reproduced acoustically by the reproduction device immediately, i.e. substantially without delay.

[0011] That is, the person can focus on locating the display device in the store, which is usually mounted at the very front of the rack. This dramatically facilitates the shopping process, especially for the visually impaired, who must only feel their way along the rack rails to find the display device. And since availability information about the product, i.e., the number of products still on the rack or even that the product is just not on the rack, can be provided using the output playback data, whether the product is actually there or out of stock is even more irrelevant at this point to searching and finding the display device.

[0012] These output playback data are generated exclusively from user data, but can advantageously be embedded in or generally combined with the control and / or management data commonly used in communication technology, i.e., they are usually transmitted together with said control or management data. It is further noted that these output playback data can exist in or be composed of two different data structures:

[0013] The first data structure may be a reference or citation, which enables the playback device to access the information represented by the playback data, particularly the playback data in text and / or audio form, within the store's local data network, for example via a customer WLAN or via the Internet, and acoustically play back the information transmitted therewith.

[0014] The second data structure may be audio and / or textual playback data encoded in a commonly used data format, so that a playback device can acoustically play back the information directly transmitted with it. In some cases, this second data structure may be present in a compressed form in order to make optimal use of the available bandwidth of the near-field contactless communication. In particular, since a minimal data budget is expected in NFC wireless communication, it may be advantageous to limit the second data structure to textual playback data.

[0015] Optionally, a combination of the first and second data structures can also be used, so that, for example, a description of the product can be started (since an acoustic reproduction of the product information takes time) directly on the reproduction device or using the immediately reproducible second data structure (reproduction data in audio or text format), while in the background, the first data structure (e.g. references) can be used to load additional data from a server via the store's WLAN or the Internet, and, following the reproduction of the information transmitted by the reproduction data in audio or text format, a reproduction of citation information can be started, which can be significantly more comprehensive than the information provided by the second data structure.

[0016] Depending on the implementation of the present invention, for the short-range communication device, for example, a Bluetooth-type low energy wireless communication device with a short range (for example, up to 50 cm or less to a few cm) can be used, or a wireless communication device that enables wireless communication within a range of more than a few cm (about 4 cm or less) based on the short-range communication specifications (ISO / IEC18092, ECMA340 or ISO / IEC21481 / ECMA352) can be used. In order to prepare for a situation where a playback device that supports only one technology or only the other technology is used, both wireless technologies can be used together in the present display device.

[0017] Playback devices in commercial use today (smartphones and tablet computers) are usually equipped with hardware capable of utilizing NFC technology themselves, and in order to accommodate the present invention, it is necessary that the received query playback data or the data referenced therefrom is available regarding the playback stage of the playback device which performs the acoustic output of the obtained information and the information defined for the acoustic playback.

[0018] In short, a solution is provided for the everyday shopping of the visually impaired or the visually impaired, in which, on the one hand, the information to be acoustically reproduced or references to said information are provided in a location-specific manner at the location of each display by the operator of the display system in the store, and, on the other hand, on the customer's side, a commercially available reproduction device is used, which is already in large quantities and whose operation is routine for the visually impaired or even for the visually impaired, i.e. the reproduction device of the exemplary example already mentioned here can usually even be controlled by voice commands or by human-machine dialogue.

[0019] Further particularly advantageous embodiments and refinements of the invention emerge from the dependent claims and the following description.

[0020] The invention considered herein is primarily applied in the context of an ESL system, where there are a plurality of such displays, i.e. ESLs, for respectively visualizing price and / or product information relating to displayed products or groups of products.

[0021] Such an ESL can be fixed to a rack or a rack rail, i.e. the fixing structure for placing the ESL and the goods can be configured as a rack or a rack rail. It can also be fixed to a display table, i.e. the fixing structure can be configured as a table. Such an ESL can also be configured as a display screen placed on a table. Such a display device can also be configured as a tag for fixing to a garment, i.e. the fixing structure can also be configured as a connecting member between the electronic display device and the goods. That is, such a fixing structure can be realized as, for example, a sewing thread, a needle or the like. To simplify the discussion of the invention, in the following, in relation to a display device, we always refer to an ESL fixed to a rack, in particular to a rack rail.

[0022] For the purpose of information visualization, each ESL is equipped with a screen, usually realized as an electrophoretic screen in the case of battery-powered ESLs, to allow the most energy-saving operation possible, but other types of screens can also be used, for example LCD screens or equivalent. These ESLs can also be provided with electricity and / or communication technology in wired form. This power supply can be achieved by means of a wiring structure integrated in the rack rails, the power supply wiring being connected by means of contacts on the ESL. Power supply can also be achieved by means of a radio signal, known in the technical term as "Power over WIFI".

[0023] In a radio-based ESL system, there are also a number of access points constituting an ESL radio network that provide communication technology to the ESLs to address and send data to them or to address and retrieve data from them. Usually, a group of ESLs is assigned to an access point through which they can be addressed and called. This assignment works by selecting a suitable radio channel in the ISM band to be used by each access point and by registering the ESL with the access point responsible for that ESL. For this wireless communication, basically standardized radio protocols can be used, such as ZigBee or Bluetooth. Proprietary radio protocols can also be used, such as the protocol disclosed in US Pat. No. 5,399,323.

[0024] The access points themselves are typically connected via a hardwired connection to a central control device, such as a local server on which a software application (e.g., shop management software) runs that manages and controls the ESLs, or they may be controlled via a cloud-based software application that provides the aforementioned functionality for managing and controlling the ESLs.

[0025] However, the display device can also be configured as a so-called video rack rail, in which case typically a high-resolution screen is used to display the video content. The video rack rail can be fixed to the rack rail or can be used instead of the rack rail. When configured as a video rack rail, the power supply to the display device is usually achieved in a wired manner using electrical wiring, as is the provision of communication technology. In the case of a video rack rail, it usually extends along a group of products. The price information and / or product information available using the output playback data can then be related to the entirety of the products assigned to the video rack rail. In this case, the information content can include the number of different products, the order of the products along the video rack rail, the spatial discoverability of the products along the video rack rail, so that visually impaired people, or even blind people, can be guided in a completely acoustic manner.

[0026] As mentioned above, the display device is a component of an ESL system. In such an ESL system, the display device communicates with a central control device, which may be constituted by a central server or a cloud-based software, and obtains data defined for the display of price information and / or product information. For this purpose, the display device comprises a system communication device, different from the short-range communication device, arranged to communicate with the control device responsible for providing the reproduction data. Advantageously, with the help of this system communication device, the reproduction data for output is obtained and stored in a storage stage of the display device. These data are stored there in such a way that they are immediately ready for use, i.e. ready to be processed by the reproduction device after immediate output from the display device.

[0027] The acoustic reproduction of the relevant information content using the output reproduction data can be performed in the reproduction device in two ways. The reproduction stage can be configured to directly reproduce the audio reproduction data represented by the output reproduction data. This allows one or more audio files to be provided using the output reproduction data, which is to be preferred in the case of a sufficient bandwidth in the case of short-range contactless communication. On the other hand, it can be provided that the reproduction device converts the text reproduction data represented by the output reproduction data into audio reproduction data, and the reproduction stage is configured to directly reproduce the audio reproduction data thus obtained, which is to be preferred in the case of a very limited bandwidth in the case of short-range contactless communication. In this case, a very compact text file with as little data as possible can be transmitted, which is synthesized into sound by known software methods in the reproduction device and is only then reproduced acoustically.

[0028] The output of the playback data for output can essentially be performed immediately after the connection with the playback device is established. However, since the other device can or should also use the short-range communication device, it has been found to be advantageous for the display device to be configured to detect an output playback data inquiry command receivable in the course of short-range contactless communication and to output the output playback data only after detecting the output playback data inquiry command using short-range contactless communication, possibly distinguishing it from at least one other inquiry command. The playback device is therefore advantageously configured to output the output playback data inquiry command via the second short-range communication device in order to start receiving the output playback data in audio and / or text form.

[0029] Thereby, the output of the playback data for output can be selectively started as desired, so that these data are output only under command control to a playback device previously configured by the user of the data, for example to assist a visually impaired or blind person. This can be done, for example, in the course of installation of the application / software deployed for assistance. On the other hand, it can also be done at the operating system level, for example if the operating system allows a setting that allows the user to identify that he is visually impaired or even blind. It has therefore been found to be advantageous for the playback device to be further configured to output a playback data inquiry command via the second short-range communication device in order to start receiving the playback data for output (basically or only after its configuration).

[0030] However, the invention can also be used to better assist visually impaired or visually impaired people during a shopping experience. It has further been found to be advantageous for the output reproduction data to comprise product ambience description data describing the product ambience of the product and enabling the reproduction device to reproduce a description of the product ambience acoustically. The reproduction device is therefore advantageously arranged to receive, as a component of the output reproduction data or as reference information within the output reproduction data, the product ambience description data describing the product ambience of the product.

[0031] The product surroundings description data can also be provided in a similar format to the two data structures considered. They serve to describe the surroundings in the vicinity of the display device, i.e. the surroundings of the products assigned to the display device, when acoustically reproduced by the reproduction device. In this way, for example, it is possible to inform the user of the reproduction device what products he can find to the left and right of the product he is currently looking at. This is relevant for each rack surface. However, this measure also makes it possible to describe the products present in the rack surfaces above and / or below, so that a visually impaired person, or even a blind person, can be given an overview of the contents of the rack in an intuitive (acoustic) way, starting from the product about which the description information is currently being reproduced. In this case, the product referred to by the short-range contactless communication constitutes the central target point for the person to further orient himself in the rack.

[0032] To facilitate the user of the playback device to consider the acoustically described ambience of the product, the playback device may be configured to play a section-by-section product ambience description of the product represented or referenced using the product ambience description data, in particular as a result of a detected interaction with the user. In this way, the playback device may acoustically announce that under a product, such as a toothbrush with brand name XYZ, there is another toothbrush with brand name ABC. The acoustic ambience description may then be paused until the user issues a command, such as "continue", which is detected or sensed by the playback device, following which the playback device acoustically announces "there is toothpaste next to the right side of the product and the rack of oral hygiene products ends next to the left side of the product". The playback device then waits for a command to continue the ambience description again. It should be noted here that the product surroundings description data can of course also describe the area diagonally adjacent to the product, so that the description of the surroundings can be carried out continuously, and the person can convey the overall picture of the surroundings around the product by acoustic means. Furthermore, it can be stipulated that the description of the surroundings can be stopped at any time with regard to other points, for example by a voice command such as "enough".

[0033] Here, in today's commonly used smartphones, the user's interaction is considered to be a standard feature as such in the context of voice recognition, the recognition results of which are then used in this case for the purpose of selectively reproducing information that matches the description of the surroundings, i.e. section by section. Naturally, gesture recognition can also be used, which is particularly advantageous when the person concerned is not only visually impaired, but also impaired in terms of speech ability, or simply prefers this option.

[0034] Otherwise, gesture recognition can be advantageously used in the context of the present invention in two ways: in a first configuration of the playback device, an acceleration sensor of the playback device can be used to detect movements indicative of the playback device's orientation, and this detected orientation can be used to select an acoustically played-back environment description adapted to this orientation; in a second configuration, an optical sensor (such as, for example, a camera system of the playback device) can be used to detect and evaluate the person's gestures, obtain directional information that can be derived therefrom, and play an environment description adapted to this orientation.

[0035] It has further proved to be highly advantageous if the display device is adapted to detect a route description query command receivable in the course of short-range contactless communication, which route description query command serves to query a route description to another product, which product is characterised by product data contained in the route description query command, said route description query command originating from a reproduction device adapted to output the route description query command via a second short-range communication device in order to initiate the reception of reproduction data for output which comprises the route description data.

[0036] The other product may be communicated by a user interaction, such as a spoken question (e.g., "Where are the organic petit breads?") or a voice command (e.g., "Navigate me to the organic petit breads.") In this context, voice recognition of the playback device is also utilized to identify the other product and obtain product data representative of the other product.

[0037] However, other items may also be extracted from a shopping list digitally stored on the playback device, for example, where the items listed therein are processed sequentially (in order). In this case, thematic groupings of the shopping list (e.g., apples and pears in the thematic group fruits and vegetables, and ham and steaks in the thematic group meats) may also be present on the playback device, so that the route instructions provided may convey a route for the person that is optimized for the thematic group.

[0038] Advantageously, the display device is adapted after detecting a route description inquiry command to provide output playback data comprising route description data and to output the route description data via the short-range communication device, the route description data being defined for acoustically playing on the playback device a description of the route to another product, still taking into account the idea that the output of the route description data should be exclusively command-controlled, i.e. initiated in a manner determined by the playback device, so as to accommodate the individual navigation wishes of the playback device user.

[0039] In this case, as well as the acoustic playback of the product surroundings, the playback device may be configured to play a section-by-section description of a route to another product, represented or referenced using the route description data, in particular as a result of the detected interaction with the user. This section-by-section playback allows a visually impaired person, or even a blind person, to be directed or navigated along a route section-by-section without having to remember the entire route from scratch, i.e. the person progresses along the route receiving section-by-section navigation instructions.

[0040] This can be achieved by giving the person a first acoustic instruction to move to the end of the rack adjacent to the person's right side, and only when the person confirms that he or she has reached the waypoint by a user interaction with the playback device (e.g., a voice saying "I have reached the waypoint") is a second acoustic instruction given to the person to turn right and proceed 18 steps down the rack aisle leading to the waypoint.

[0041] In this case, technical measures or standardized size information are intentionally not taken into account, since this would not be possible if the person were to move. Rather, the playback device, in the course of creating a user profile, can determine in advance what size the average stride of the person is, and then incorporate this knowledge into the person-specific stride size information in order to enable the person to execute navigation instructions as naturally as possible in a surrounding environment defined, for example, in meters.

[0042] When the person confirms arrival at the next waypoint through user interaction with the playback device, a third acoustic indication is given which finally provides the person with the information to find another item on the middle rack surface of the left rack.

[0043] Now, in order to obtain a precise orientation on the rack, the person can bring the playback device within the radio range of the display device installed there and acoustically obtain information about the described product using the display device. Similarly, the already considered acoustic description of the surroundings of the product can be performed, which makes it easier to find other products on the rack.

[0044] The invention allows a visually impaired person, or even a blind person, to obtain information about products within his reach, using a display device installed in the store, and furthermore, the person is provided with the means to move "step by step", i.e. gradually along an easily walkable path section, to the (different) product he is looking for, in which case a commercially widespread smartphone or tablet computer is used on his side as user equipment (in the sense of a reproduction device) that ESL uses in the store for the purpose of acoustic information reproduction.

[0045] However, the invention can also be used to verify the effectiveness of marketing measures with respect to the person in question. Thus, for example, the playback data for output can be used to send marketing or advertising messages encouraging the purchase of certain products. In the course of the short-distance contactless communication, the identifiers of the playback devices and possibly even the identifiers of the person in question can be ascertained, which can be reported to the central control device of the ESL system and stored there for management purposes, so that it can then be investigated, when paying at the POS terminal, whether the person has used the marketing or advertising message to purchase the product promoted thereby.

[0046] It should be further noted here that the above-mentioned order of commands sent to the display device does not have to be respected. Rather, the above-mentioned commands can be sent individually and independently of each other to the display device and processed individually by the display device. Thus, for example, in any ESL, a query can be performed to identify nearby products, so that the output playback data can consist only of product surroundings description data. Furthermore, a query can be made for a path description to another product separately from the query for specific product information or separately from the query for the surroundings of that product, so that the output playback data can consist only of product surroundings description data. That is, it is possible to use commands independently of each other in both the playback device and the display device.

[0047] In summary, the term "product information" can be used in different ways depending on the context: Whether this information is about the product that was just inquired about to ESL via short-range contactless communication, This information is information about the surrounding environment of the product that was just queried to the ESL through short-distance contactless communication, or This information is information about a route from the product that was just inquired about to the ESL via short-distance contactless communication to another product. I would like to further state that this should be understood as follows.

[0048] Finally, it should be further noted that, in general terms, the electronic devices considered here (ESLs, smartphones, tablet computers, video racks, etc.) or modules thereof naturally comprise electronic components. The electronic components may consist of discrete electronic components, integrated electronic components or a combination thereof. Microcomputers, microcontrollers, application specific integrated circuits (ASICs) may also be used, possibly in combination with analog or digital electronic peripheral components. Many of the aforementioned functions of these devices are realized by means of software running on the processors of the electronic components (possibly in cooperation with hardware components). Devices configured for wireless communication usually comprise, as a component of a transceiver module, an antenna structure for transmitting and receiving radio signals. For short-range contactless communication, besides the aforementioned antenna structures, coil structures or capacitor plate structures for inductive coupling may also be used. Furthermore, these electronic devices may comprise an internal power supply, which may be realized, for example, by a replaceable or rechargeable battery. These devices may also be powered in wired form by an external power grid part or by using "Power over LAN".

[0049] The above and other aspects of the present invention will become apparent from the drawings considered below.

[0050] The invention will now be described in more detail again by way of example with reference to the attached drawings, in which the invention is not limited thereto, in which the same elements are provided with the same reference numerals in the different drawings. [Brief description of the drawings]

[0051] [Figure 1] Schematic diagram of a rack with ESL for examining two aspects of the present invention. [Diagram 2] Block wiring diagram of ESL mounted in rack [Diagram 3] Block diagram of a smartphone used as a playback device when considering the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0052] FIG. 1 illustrates a rack 1 consisting of three rack bottoms, namely an upper rack bottom 2, a middle rack bottom 3, and a lower rack bottom 4, each of which terminates at its front edge in a rack rail 5. On these rack bottoms 2-4, different products P1-P9 are installed, which in this case can be distinguished by their shapes. In correspondence with the positions of these products P1-P9, electronic rack displays E1-E9, abbreviated as ESL-E1-E9 (based on the initials of the English name "Electronic Shelf Label"), are attached to the rack rail 5 as display devices. Furthermore, FIG. 1 illustrates a central server 29 that manages the ESL-E1-E9. This server 29 is connected to an access point 30 in a cabled form. This access point 30 provides an ESL wireless network for wireless communication with the ESL-E1-E9.

[0053] In FIG. 2, the first ESL-E1 is shown as a representative of all the installed ESLs-E1 to E9, or more precisely, its electronic block connection diagram is shown. In the explanation of FIG. 2, the reference symbol E1 is omitted and only the abbreviation ESL is used.

[0054] The ESL comprises an electrophoretic display module 6 adapted to visually reproduce price and / or product information of one of the products P1-P9 in an energy-saving manner.

[0055] The ESL-E1 further comprises a first radio module 7 adapted for ESL wireless communication in the ESL radio network referred to in the general description. The ESL wireless communication of this first radio module is based on the propagation of electromagnetic waves, for which reason an antenna structure 7A is represented in the first radio module 7 as an interface between the electronics (not shown in detail) of the first radio module 7 and the surrounding environment. Via this ESL radio network, the ESL obtains display reproduction data AW1 representing price information and / or product information of each product P1-P9. In this case, these display reproduction data AW are constituted by image files containing image information coded in digital form, which are outputted via an internal data bus 9 to the display module 6 and reproduced by this module in a visualised manner using its electrophoretic screen (not shown in detail).

[0056] The ESL further comprises a second radio module 8, which is configured and arranged for NFC wireless communication with a device also equipped with NFC technology, which is in close proximity, in any case within an NFC wireless communication range 10, typically a few centimetres. In the context of this discussion, this device is a smartphone 11 equipped with NFC technology (see Figures 1 and 3), which is used as the playback device. This NFC wireless communication is based on inductive coupling with this other device, so that within the second radio module 8 a coil structure 8A is represented as an interface between the electronics (not shown in detail) of the second radio module 8 and the surrounding environment.

[0057] It is noted here that the electronic components of these modules 6, 7, 8 each include a microcontroller or microprocessor which, possibly in cooperation with other electronic components or parts, provides the functionality of each module 6-8. It is further noted that a central control module may be provided which coordinates or controls the mutual cooperation of the individual modules 6-8.

[0058] In order to facilitate shopping for visually impaired or visually impaired persons in the store in which Rack 1 is located, the following data is also transmitted to the ESL during the progress of the aforementioned ESL wireless communication, namely: a) Output playback data AW2: This data describes price and / or product information in quite general terms and is in this case present as plain text (e.g. in ACSII code) and is transmitted in the form of a text file to each ESL-E1 to E9 and stored there to be retrieved by the playback device. This text description can be, for example, in the form "This product is smooth flour packaged in sachets of 500 grams" in connection with ESL-E1 located at the top left, or more simply "Flour, smooth, sachets of 500 grams". This facilitates and accelerates the transmission using NFC technology, since the amount of data is very small. b) Product surroundings description data UB: This data describes the surroundings of each product. Again, this structure is suitable for textual surroundings information for problem-free transmission using NFC technology. The textual description of the surroundings (indicated by the arrows U1-U8) stored in the fifth ELS-E5 in relation to the product P5 can be structured as follows, where, of course, in the place of the substitution of <<product Pn>>, the actual product name is cited and the actual number of entries in the list depends on the respective position of the product and its surroundings: "Find <<Product P4>> right next to you on the left" (see arrow U1 in Figure 1) "Find <<Product P6>> right next to you on the right" (see arrow U2 in Figure 1) "Find <<Product P7>> to the right of <<Product P6>>" (see arrow U3 in Figure 1) "Find <<Product P2>> directly next to you above" (see arrow U4 in Figure 1) "Find <<Product P8>> diagonally to the lower left" (see arrow U5 in Figure 1) "Find <<Product P9>> diagonally to the lower right" (see arrow U6 in Figure 1) "Find <<Product P1>> next to you diagonally to the upper left" (see arrow U7 in Figure 1) "Find <<Product P3>> diagonally to the upper right" (see arrow U8 in Figure 1)

[0059] These data AW1, AW2 and UB are stored in the ESL (see FIG. 2) in a memory stage 12 of the first radio module 7 and are there prepared, i.e. retrieved from there, for subsequent use by the second radio module 8. In order to avoid the need to start up the first radio module 7 when accessing the data AW2 and UB, it may be advantageous for these data AW2 and UB to be transferred after their reception to the second radio module 8 and stored there, as indicated by the dashed line, in a second memory stage 13 of the second radio module 8. Optionally, these data AW2 and UB may also be stored in a common memory (not shown) to which both radio modules 7 and 8 have access.

[0060] These data AW2 and UB are taken by the smartphone 11 during NFC wireless communication with the smartphone 11 from the ESL involved in the NFC communication (in the context of FIG. 1 this is ESL-E5) and converted there into an audio signal AS and output acoustically, such that a visually impaired person, or even a blind person, can orient themselves towards the rack on the basis of the acoustic description.

[0061] In the following, a smartphone 11 for use by a visually impaired person will be considered with the aid of FIG. 3. It comprises, in this case, a microcontroller 15 for processing programs (i.e. software) and an ECU 16 for storing an operating system 17 and for storing software applications 18 installed for the present use of the smartphone 11. 2 It comprises a central electronics component 14 having a PROM memory 16 and a RAM memory 19 for temporary storage of data generated during the programming process.

[0062] Furthermore, the smartphone 11 comprises an electronic component for an audio-visual interface 20 consisting essentially of a touchscreen module 21 for visualizing information and receiving user interaction in the sense of touching the screen, an audio module 22 (amplifier and loudspeaker) for outputting an audio signal AS, and a microphone module 23 for receiving user interaction in the sense of spoken words, etc.

[0063] Furthermore, the smartphone 11 comprises a third wireless module 24 configured for NFC wireless communication, in this case used for NFC communication with ESL-E1 to E9. Also in this case, for the reasons described in connection with the second wireless module 8, a different coil structure 24A is shown.

[0064] Furthermore, the smartphone 11 comprises an energy supply module 25, a mobile radio module 26 for communicating with a mobile radio operator's cellular network, and further commonly used smartphone components 27, which will not be described in detail.

[0065] The blocks 24, 20 and 14 are connected by means of an internal data bus 28. However, they can also be integrated on a system-on-chip platform.

[0066] The software application 18 can be conceptually divided into a program code APP and commands defined for data query during NFC wireless communication, which are also understood by ESL-E1 to E9. In this case, these commands are: An output playback data inquiry command AWB for starting the output of output playback data in ESL-E1 to E9 for the purpose of acoustically playing back price information and / or product information on the smartphone 11; A route description inquiry command PUB that starts output of output playback data including product surrounding environment description data in ESL-E1 to E9 for the purpose of acoustically playing a description of the product surrounding environment on the smartphone 11; A route explanation inquiry command WBB for starting output of output playback data including route explanation data in ESL-E1 to E9 for the purpose of acoustically playing an explanation of a route to another product on the smartphone 11; In this case, information about other items may be obtained, for example, from an electronic shopping list which may also be stored in the software application 18 or from another software application on the smartphone 11 .

[0067] Now, as illustrated in Fig. 1, if the person holds the smartphone 11 with the software application 18 started at the central ESL-E5, first an NFC wireless connection is established and an output playback data inquiry command AWB is sent to the ESL-E5 and detected there, so that a text file stored therein containing price information and / or product information about the product P5 is sent from the ESL-E5 to the smartphone 11 in the course of the NFC wireless communication. The text contained in this text file is converted in the smartphone 11, for example by the software application 18, into an audio format and played as an audio signal AS by means of the audio module 22.

[0068] Here, the software application 18 also provides a reference to the ambient environment information relating to the product P5. If the user confirms that he is also interested in this information, then in the NFC wireless communication a route description inquiry command PUB is sent to the ESL-E5 and detected there, so that a text file stored therein containing ambient environment information relating to the product P5 (see in this respect the above-mentioned discussion of the product ambient environment description data UB) is sent from the ESL-E5 to the smartphone 11 in the course of the NFC wireless communication. The text contained in this text file is converted by the software application 18 in the smartphone 11 into audio format and reproduced as an audio signal AS by means of the audio module 22. In this case, the acoustic reproduction can be stopped after the reproduction of each text sequence, i.e. after the reproduction of the ambient environment information relating to one of the directions U1 to U8, and can only be continued on the basis of voice control, for example after the reception of a voice command such as "continue". This gives the person time to mentally organize the acquired ambient environment information.

[0069] This software application 18 offers the person the option of navigating in the form of acoustic guidance towards another product. If the person confirms that he / she wishes to navigate in this form, then a next product (called another product) is selected from the electronic shopping list and the associated product information (i.e., for example, the product name as text) is sent via NFC wireless communication as a component of a route description query command WBB to the ESL-E5, which detects this command. Now, as a result of detecting this command, the ESL-E5 contacts, via the first wireless module 7, a central server 29 of the ESL system (called central control unit in the general description, not shown), in which it communicates the identity of the ESL-E5 and the other product it is looking for. This server 29 has a digitally stored shelf planogram from which it is possible to read the locations of all the products in the store and, as a consequence, the locations of all the ESLs assigned to those products. A route through the store from the actual position of the ESL-E5 to another product, possibly located in another rack aisle further away, is then calculated and sent as route description data WB in the form of a text sequence sequentially describing the route between the route points via the first wireless module 7 to the ESL-E5, where it is possibly temporarily stored and from there sent via the second wireless module 8 to the smartphone 11, where again the text is converted into synthetic voice, with the playback of navigation instructions (i.e. individual instruction sequences) being triggered under voice control and sequentially between the route points so that the person is guided stress-free between the route points and given enough time to move.

[0070] That is, the present invention allows for the acoustic provision of local information on products on the racks in an intuitive manner and for acoustically guided navigation in the store, with the in-store ESL infrastructure forming the basis for providing these functions reliably and without errors. Since the location of the installed ESLs is always known in this ESL system, a person can start the acoustically guided navigation process anew at any time at any ESL, i.e. even if the person accidentally deviates from the acoustically defined route, i.e. gets lost. This provides safety and reliability when applying this technology.

[0071] Finally, it is to be pointed out once again that the figures detailed above are merely examples which a person skilled in the art can modify in various ways without departing from the scope of the invention, and for the sake of completeness, it is also to be pointed out that the use of the indefinite article "a" does not exclude that the feature in question can be present several times.

Claims

1. an electronic price and / or product information display device, referred to for short as display device (E1-E9), adapted for visually reproducing price and / or product information of products (P1-P9) and adapted for storing reproduction data representative of the price and / or product information, In particular, in a display device (E1 to E9) equipped with a first near-field communication device (8) configured to perform short-distance non-contact communication with a playback device (11) having an NFC function, in particular a first Near-Field-Communication device abbreviated as NFC device, These playback data, Display reproduction data (AW1) defined for visual reproduction of price information and / or product information by the electronic price information and / or product information display device (E1 to E9) itself; output reproduction data (AW2) provided as reproduction data, in particular in audio and / or text format, which is output to a reproduction device (11) by means of short-distance contactless communication and enables the reproduction device (11) to acoustically reproduce the price information and / or product information thereon; and The display device is characterized in that the short-range communication device (8) is configured to access the output reproduction data (AW2) and output the output reproduction data (AW2) during the short-range non-contact communication.

2. In the display device (E1 to E9) according to claim 1, The display device includes a system communication device (7) that is different from the short-range communication device (8) and is arranged to communicate with a control device (29) that serves to provide playback data (AW1, AW2), and the playback data for output (AW2) is acquired using the system communication device (7) and stored in memory stages (12, 13) of the display device (E1 to E9).

3. In the display device (E1 to E9) according to claim 1 or 2, A display device configured to detect an output playback data inquiry command (AWB) that can be received during short-range non-contact communication, and, in some cases, distinguish it from at least one other inquiry command, and output output playback data (AW2) only after detecting the output playback data inquiry command (AWB) using short-range non-contact communication.

4. In the display device (E1 to E9) according to any one of claims 1 to 3, A display device in which the output reproduction data (AW2) has product surroundings description data that describes the product surroundings of a product and enables an acoustic reproduction of a description of the product surroundings on a reproduction device (11).

5. In the display device (E1 to E9) according to any one of claims 1 to 4, The display device is configured to detect a route description inquiry command (PUB) that can be received during short-range contactless communication, and the route description inquiry command (PUB) serves to inquire about a route description to another product (P1 to P9), and the other product (P1 to P9) is characterized by product data included in the route description inquiry command (PUB).

6. In the display device (E1 to E9) according to claim 5, The display device is configured to provide output playback data (AW2) including route description data (WB) after detecting a route description inquiry command (PUB) and output the route description data (WB) via a short-range communication device (8), the route description data (WB) being defined for acoustically playing back a description of the route to another product (P1 to P9) on a playback device (11).

7. In the display device (E1 to E9) according to claim 6, A display device comprising the system communication device (7) according to claim 2, and configured to inquire from a control device (29) about output playback data (AW2) including route description data (WB) based on output of product data of another product (P1 to P9).

8. A reproduction device (11) comprising a reproduction stage (20) adapted to acoustically reproduce information represented in digital form, A second short-range communication device (24), in particular abbreviated as NFC device, is provided, adapted for short-range contactless communication with electronic price and / or product information display devices, in particular abbreviated as display devices (E1-E9), having NFC functionality, said second short-range communication device (24) being adapted to query and receive output reproduction data (AW2) in the course of short-range contactless communication with said electronic price and / or product information display devices (E1-E9), said output reproduction data (AW2) enabling said price and / or product information to be acoustically reproduced in said reproduction device (11); The second short-range communication device (24) is configured to provide output reproduction data (AW2) provided in particular as reproduction data in audio and / or text format for acoustic reproduction of price information and / or product information by the reproduction stage (20).

9. A playback device (11) according to claim 8, A playback device, the playback stage (20) being configured to directly play back the audio format playback data represented by the output playback data (AW2).

10. A reproduction device (11) according to claim 8 or 9, The playback device (11) is configured to convert the text-format playback data represented by the output playback data (AW2) into audio-format playback data, and the playback stage (20) is configured to directly play the audio-format playback data thus obtained.

11. In a playback device (11) according to any one of claims 8 to 10, The playback device is configured to output an output playback data inquiry command (AWB) via a second short-range communication device (24) in order to start receiving output playback data (AW2).

12. In a playback device (11) according to any one of claims 8 to 11, A playback device configured to receive product surroundings description data (UB) describing the product surroundings of products (P1-P9) as a component of output playback data (AW2) or as reference information within the output playback data (AW2).

13. A playback device (11) according to claim 12, In particular, the playback device is configured to play back, as a result of detecting an interaction with a user, a description of the product surroundings of the products (P1 to P9) in sections, which description is represented or referenced using product surroundings description data (UB).

14. A playback device (11) according to any one of claims 8 to 13, The playback device is configured to output a route description query command (PUB) via a second short-range communication device (24) for the purpose of initiating reception of output playback data (AW2) including route description data (WB).

15. A playback device (11) according to claim 14, In particular, the playback device is configured to play back, as a result of detecting an interaction with a user, a description of a route to another product (P1 to P9) in sections, the description being represented or referenced using route description data (WB).

Images