How to position playback content within the viewing area of ​​the video rack rail screen

The method automates the alignment of playback content on video rack rails using image processing, addressing manual alignment inefficiencies and enhancing workflow efficiency.

JP2024544463AActive Publication Date: 2024-12-03ヴジョングループ·ゲゼルシャフト·ミト·ベシュレンクテル·ハフツング
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2024518116
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2021-10-20
Publication Date
2024-12-03
Estimated Expiration
2041-10-20

AI Technical Summary

Technical Problem

Existing video rack rail systems require manual intervention for aligning playback content with products, which interferes with employee tasks and is inefficient.

Method used

A method using image processing to automatically detect placement indicators within a digital image, generating placement data to position playback content accurately on video rack rails, allowing for automatic alignment without manual assistance.

Benefits of technology

Enables automatic and efficient placement of dynamic or static content on video rack rails, reducing interference with employee tasks and improving workflow efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024544463000001_ABST
    Figure 2024544463000001_ABST
Patent Text Reader

Abstract

A method for positioning playback content within the display area of ​​a screen 9 of a video rack rail 2, in which a digital image DA of the screen 9 and / or product display areas PBA, PBB, PBC, PBD extending along the screen 9 is captured using a camera 12, a positioning indicator and its position are automatically detected in the digital image DA using an image processing device 16, this positioning indicator indicates or defines the location of a product to be displayed along the screen 9, positioning data PBA, PBB, PBC, PBD are generated using the image processing means 16 based on the detected position of the positioning indicator, this positioning data PBA, PBB, PBC, PBD defines the location where the playback content is to be played within the display area 8, and the video rack rail 2 plays the playback content within the display area 8 of the screen 9 based on the positioning data PBA, PBB, PBC, PBD.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a method for positioning playback content within the display area of ​​a screen of a video rack rail. [Background technology]

[0002] Nowadays, in stores, product information and / or price information is displayed on electronic display units, in particular electronic shelf labels, technically called "Electronic Shelf Labels" or ESLs for short. These physical ESLs are manually fixed to rack rails constituting the front part of the rack bottom plate at positions corresponding to the products to be placed on the rack bottom plate, and typically display essentially static product information and / or price information using electrophoretic screens. Furthermore, so-called video rack rails are known from Patent Document 1, in which a video screen constituting the front part of the rack rail is used to display video information (videos or video clips), in particular related to products, and virtual ESLs are embedded in these video clips and displayed corresponding to the product positions. In this case, the positioning of these virtual ESLs is performed manually by a remote control, for example a tablet computer.

[0003] In the case of physical ESLs, misalignments between the configuration of each ESL and the corresponding products can only be removed by manual intervention, i.e. by manually shifting the ESLs, which can be easily done by an employee walking around the rack aisles, visually inspecting the configuration and immediately intervening in case of said misalignments, which the employee can easily do in conjunction with, i.e. in parallel with, his main task, e.g. replenishing the racks with products.

[0004] In the case of a virtual ESL, the adaptation of the configuration is not so easy to implement, since the employee must always carry the remote control with him / her, must perform the operations by menu items when operating the remote control, and must perform the ESL configuration operations by operating the user interface of the remote control. However, the process significantly interrupts the employee during the main operations, such as the aforementioned stocking of products, which is undesirable in the workflow. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Patent Publication No. 2019 / 091566 Summary of the Invention [Problem to be solved by the invention]

[0006] SUMMARY OF THE PRESENT EMBODIMENT In view of the above, it is an object of the present invention to provide a method for overcoming the aforementioned drawbacks when using video rack rails. [Means for solving the problem]

[0007] This problem is solved by a method according to claim 1. The subject of the invention is therefore a method for positioning playback content within a display area of ​​a screen of a video rack rail, comprising: A digital image of the screen and / or a product display area extending along the screen is captured using a camera; automatically detecting, using image processing equipment, a placement indicator and its location within the digital image, the placement indicator indicating or defining a location of an item to be displayed along the screen; Using the image processing device, placement data is generated based on the detected position of the placement indicator, the placement data defining where within the display area to play the playback content; The video rack rail plays the playback content within the display area of ​​the screen according to the arrangement data. This is the method.

[0008] The measures according to the invention make it possible for the first time to automatically position the reproduction content by automatically determining the positioning data, in particular with the advantage that it is possible completely without manual assistance from supermarket employees. The positioning data obtained in this way can also be used for the automatic positioning in different ways, in particular for changing the position of the reproduction content in the display area following a previous positioning, as will be discussed further below.

[0009] The playback content can be provided by dynamically changing content, for example videos or video clips, or by essentially static content, for example product information and / or price information, typically displayed by electronic shelf labels. In this case, each type of playback content is represented by playback data that is processed by the screen electronics of the screen of the video rack rail, so that the screen displays each playback content in a defined display area position in its display area. The individual playback contents can be arranged in individual display area positions in the display area and visually overlap or hide each other. In this case, a picture-in-picture effect can be created for the observer by superimposing a larger playback content with a smaller one. So, for example, with the video rack rail, a video can be displayed on the one hand and this video can be superimposed on the other hand in a partial area of ​​the display area by means of a virtual ESL. Thereby, for example, for one product, an emotive video can be displayed along the entire length of the display area, and at a predefined location, more precisely in the display area, i.e. in a section of the video, objective product information and / or price information can be displayed by means of a virtual ESL. Of course, the display area can also be divided along its length according to different products along which the display area extends, so that for n products, for example, n videos, each with an embedded virtual ESL associated with the product, can be played next to each other. Instead of the essentially static content of the virtual ESL, dynamic content can also be played there.

[0010] Further particularly advantageous embodiments and refinements of the invention emerge from the dependent claims and the following description.

[0011] The product display area of ​​a rack is an area in which products are displayed to the customer. This area is usually a rack base arranged next to each other or on top of each other in different configurations. In this case, the front end of such a rack base facing the customer is delimited by a video rack rail. However, a product display area can also be understood as an area in which products hung next to each other and / or products hung in a classified manner on top of each other in different configurations are displayed. Here, the video rack rail can be arranged, for example, below the hung products, also in a suspended form, or fixedly on the structure of the rack, for example on the rear end wall, the lateral bulkheads or the supports.

[0012] A digital image is represented by image data generated by a camera when capturing moving or still images of a scene within the camera's field of view, which means that a digital image can be defined by a digital still image, a sequence of digital still images (e.g., in JPG or GIF format), or by a digital video (e.g., in MP4 or MOV format).

[0013] Fully automatically detected position indicators can have very different properties or characteristics, which are discussed in more detail below.

[0014] The invention discussed here is primarily used in the context of an ESL system with video rack rails as previously described.

[0015] For the purpose of visualizing information, each of the video rack rails is equipped with a video screen. The ESL system has a number of wireless base stations, also called access points, each of which provides a group of video rack rails with dynamic and static playback content adapted to the merchandise displayed on the rack base plate defined by each video rack rail.

[0016] The wireless technical allocation is performed by the selection of a suitable wireless channel in the ISM band to be used by each access point and by the registration of the video rack rail with the access point which is responsible for this video rack rail. For wireless communication, basically standardized wireless protocols such as ZigBee, Bluetooth, WLAN networks, wired LAN networks, etc. can be used.

[0017] These access points are themselves typically connected by cables to a higher-level or central control device, such as a local server on which a software application runs that manages and controls the ESL or defines the visually recognizable playback content of the virtual ESL (e.g., shop management software), or a cloud-based software application that controls the supply of playback content to the video rack rails and possibly covers other aspects of merchandise management as well.

[0018] The method first involves automatic detection of the location indicators and their positions using computerized image content and / or pattern detection using software running on a processor that is optimized for finding, in the digital image produced by the camera, each of the location indicators to be identified, possibly having characteristics to be identified, in particular characteristics that may also be already known a priori.

[0019] In this case, in one embodiment of the method, an optical signal, in particular an optical signal in a spectral band invisible to humans, is detected as a position indicator. This optical signal makes the detection of the position indicator very easy, since the intensity or spectrum of the optical signal is easily highlighted or differentiated from the background and therefore easier to find during a digital search in a digital image than if the object was not illuminated. In a special embodiment, this optical signal can also be shifted into a spectral band invisible to humans, so that customers who remain in the store cannot sense the optical signal and therefore are not stimulated by it. Naturally, the camera is optimized for the wavelength or spectral band of the optical signal to be processed, respectively.

[0020] Particularly advantageously, this optical signal is constituted by a time- and / or spatially varying signal. This type of signal can, on the one hand, improve the possibility of detection in the digital image, since changes in the image content can often be detected automatically more easily than static image content. In addition to the position at which the optical signal is transmitted and the position at which it is detected in the image, this optical signal can also be used to provide, for example, other information that is detectable in the framework of detecting the optical signal in the digital image. These other information can, for example, be associated with goods that are found at or around the position of the optical signal.

[0021] A time-varying optical signal can be understood to be, for example, a blinking light signal, which can be, for example, emitted from a single light source. A spatially-varying optical signal can be understood to be, for example, a combination of possibly blinking light signals emitted from different light sources or positions, i.e. light signals generated by different light sources. A spatially and temporally-varying optical signal can be understood to be, for example, a cloud of time-varying light points displayed on a screen.

[0022] In one variant, this optical signal can be output from a screen. For this purpose, the screen can be equipped with light-emitting diodes or the like integrated, for example, in its housing or in a frame surrounding the display area. The display area of ​​the screen can also be used, for example, to emit a simple static light signal at a given location, or to emit a simple flashing light signal, or to display a pattern that can be changed in time and / or space. In this case, such a pattern is particularly suitable for providing additional information just by corresponding coding. The driving of the light-emitting diodes or the screen for the purpose of sending out the optical signal is performed here by the electronics of the video rack rail. In particular, when using a screen of the video rack rail for outputting an optical signal, the display area can be optically sensibly segmented along its longitudinal extension, for example, so that corresponding display segments appear along the longitudinal extension of the rack base. This can be detected in the digital image of the display area by an image processing device and can be used, for example, to generate positioning data in the form of positioning virtual ESLs in the display area of ​​the screen of the video rack rail so as to coincide with the center of each display segment.

[0023] In another embodiment, the optical signal is output by a rack separator located in the product display area. The rack separator is a substantially plate-shaped object arranged on the rack bottom plate, which separates and spatially separates two products or product groups from each other, arranged on the left and right sides of the rack bottom plate. This has the effect that the playback content displayed by the video rack rail can be reproduced without being disturbed by the output of the optical signal. In this case, the signal transmission by the optical signal is performed in a spatially separated manner from the video rack rail and therefore does not disturb the perception of the playback content by the customer. For this purpose, the rack separator is provided with one or more light-emitting diodes on its front and / or upper housing side or in the intersection area between these two housing sides, which are driven by the electronics of the rack separator housed in the housing. The front housing side is the side facing the area in front of the rack and essentially facing the video rack rail. The upper housing side is the side opposite to the rack bottom plate on which the products are displayed. Depending on the configuration of rack separator used and the spatial context of the rack, the camera should be positioned so as to detect one or more light emitting diodes of the rack separator installed within the rack.

[0024] In another implementation, products or groups of products present in the product display area are detected as position indicators. This implementation has the advantage that no additional light sources or signaling by the screen itself is required. This reduces both the technical and investment burden on the part of the video rack rail and possibly also on the part of the optionally provided rack separator or equivalent. In this example, the computerized image content and / or pattern detection is directed to products or groups of products that can be found in the rack by computerized image evaluation in order to arrive at the position data.

[0025] The following describes how the placement data is used to control how playback information is placed.

[0026] In a first implementation configuration of the layout data processing, the layout data is output from the image processing device to a higher-level control device, and this control device controls the layout of the playback content within the display area of ​​the screen of the video rack rail, in particular the video rack rail on which the digital image has been detected, based on the layout data, in particular individually controlling a large number of such video rack rails.

[0027] This higher-level control device may be a server installed in the retailer's store and running software that provides playback data to the individual video rack rails. These playback data represent the static and / or dynamic playback content to be displayed by each screen for the entire display area of ​​the screen. That is, the embedding or "merging" of the various individual playback contents taking into account the positioning data is performed in advance by the higher-level control device, and the playback data generated in this process is output to the video rack rails, so that the entire playback content represented thereby is visualized by the screen.

[0028] However, this higher level control equipment can also be provided as a cloud-based software solution hosted on a server or using a server farm in a computing center and made available to retailers over the internet for the aforementioned purposes.

[0029] In that way, for example, a background video can be played which fills the whole display area and virtual price labels can be overlaid in the display area corresponding to or belonging to different products arranged next to each other in a rack. The arrangement of the virtual price labels representing the played information to be arranged in the display area is described here by arrangement data which is used by the control device to arrange the price label / labels along the width of the display area.

[0030] The higher level control device transmits the playback data determined for each video rack rail as a data stream to the video rack rail for real-time playback therein. The higher level control device can also transmit the playback data to the video rack rail in the form of a data packet to be transmitted once, where the data packet is stored and played repeatedly (i.e. the playback content defined thereby is played repeatedly, for example as a locally stored video clip or video film).

[0031] Furthermore, for example, the aforementioned background video can also be spatially segmented along the longitudinal extension of the display area of ​​the screen of the video rack rail, each video segment constituting an individual dynamic playback content for a given product among the different products displayed next to each other in the rack. The arrangement of the individual video segments along the width of the display area is described here by arrangement data, which are used by the control device to arrange the individual video sequences along the width of the display area. In this case, the respective widths for displaying the individual video sequences can also be automatically set by the control device taking into account the arrangement data, so that, for example, the entirety of the video segments displayed next to each other fills the entire width of the display area. Dark or color-highlighted image areas can also be provided between the individual video sequences in order to optically distinguish these image areas better from each other along the width of the display area or to visually separate them. In this example too, virtual price labels can be arranged along the display area, which are then arranged, for example, centrally within the partial area of ​​the display area occupied by each video segment. In this example too, the higher level control device uses the placement data either directly to place the virtual price label, or indirectly, i.e. after placing a video sequence, uses it to place the virtual price label by referencing the placed video sequence.

[0032] In a second embodiment of the arrangement data processing, the arrangement data is output directly from the image processing device to the video rack rail which has detected a digital image of the screen or of the product display area extending along the screen, and this video rack rail controls the arrangement of the reproduced content in the display area of ​​the screen of the video rack rail based on the arrangement data. In this case, the functions considered according to the first embodiment of the arrangement data processing can still be realized, but now these functions are realized directly by the electronics of each video rack rail. For this purpose, in any case, a background video, possibly a video sequence, as well as an appropriate number of virtual rack labels, are transmitted in digital form from the central control device to each video rack rail. In particular, the transmission of the background video as well as the transmission of the video sequences can be carried out from the control device to the video rack rail as a continuous video stream or as an isolated download, as described above. The transmission of each virtual rack label is realized as a download if the virtual rack label is content-static. This does not have to be the case, but a transmission towards the video rack rail can be advantageous even if the virtual rack labels are content-dynamic. In this case, the second embodiment in particular has the advantage that the task of arranging each playback content is distributed to each video rack rail, i.e. there is no need to plan to allocate large computing power to the central control device for this, and the data traffic between the control device and the video rack rail is also limited to the provision / transmission of each playback content, possibly only to its initial arrangement. Since the video rack rail certainly already has the ability to play videos by definition, it is easy for the electronics integrated therein, and possibly also for the software used therein, to execute high-performance software that can perform both the dynamic playback content arrangement and the static playback content arrangement considered above. That is to say, the digital resources present distributed in the video rack rail, such as for example memory and computing power, are optimally utilized not only for performing pure playback, but also for performing automatic arrangement of the playback content to be played.This significantly reduces the load on upstream central control equipment in large scale installations in large supermarkets that may deploy thousands of video rack rails.

[0033] As already discussed generally, the configuration data describes the configuration of the video rack rails within the display area. In particular, the configuration data can describe or represent the configuration within the display area in at least one of the following formats: a) A single coordinate defining a position within the display area along one direction, e.g. describing the previously mentioned arrangement along the length of the display area parallel to the longitudinal extension of the underlying rack base, i.e. figuratively describing the coordinates along the different products or groups of products found along the longitudinal extension of the rack base. The definition of the arrangement thus made starts from the assumption that all the playback content is arranged, e.g. at the bottom edge of the display area or at a fixed distance therefrom, or sorted into different groups (dynamic / static or video / label) at different distances from the lower display area. b) A coordinate pair defining the position of a point within the display area, the use of which allows full flexibility in the placement of the display area relative to its periphery, whereby the playback content can be freely selected and placed within the display area. c) A line segment within the display area: The definition of a line segment also makes it possible to perform scaling of each display content, where the playback content can be scaled not only along this line segment, i.e. in one dimension, but also proportionally in the second dimension (orthogonal to this line segment). d) Planes within the display area: This definition allows the playback content to be freely selected and positioned within a given plane, whose position within the display area is also defined, or to be restricted to an area. The playback content can also be scaled for display within this plane, in at least one dimension, and advantageously proportionally in two orthogonal dimensions, to fit the defined plane. e) A portion of the display area, in particular a portion corresponding to a product display area extending along the screen. This interpretation of the positioning data is particularly advantageous when positioning the previously mentioned video sequences in different sections of the display area along its length.

[0034] The configuration data can in principle be generated at any time and can also be used persistently for configuration after its generation, however, it has proven to be particularly advantageous to use the configuration data only under the following conditions for configuring the reproduction information: a) the condition that the spatial change of the configuration defined by the configuration data compared to the currently existing configuration exceeds a predefined threshold, in particular a predefined fixed threshold or a threshold that changes dynamically over time, and / or b) the condition that the time span for taking into account the placement data exceeds a time threshold, in particular a time threshold defined by a typical dwell time of a customer in front of a video rack rail;

[0035] This aspect of the invention takes into account the situation where, as mentioned above, the immediate realization of configuration data generated directly by the video rack rail or also by a central control device is often not desirable at all, since it often only changes the position of the reproduced content in a small way in place, but still can change it frequently, which can annoy the viewers of the video rack rail. Therefore, it is advantageous to evaluate the new configuration of the reproduced content given by the configuration data and, in the evaluation, check whether the new configuration should be implemented at all.

[0036] For example, if the spatial differences compared to the previous arrangement are too small, the new arrangement can be withheld. In assessing whether this spatial difference is noteworthy or not, the dimensions of the products or product groups present along the video rack rail can play a role. For example, it can be ensured in this way that minimal spatial differences that are possible in the frame of the camera shot, for example due to changes in light conditions or shadows, can be suppressed (i.e. not noted). On the other hand, larger spatial differences, such as differences that may occur, for example, when placing or removing products already on the rack, or as a result of refilling the rack, are very well taken into account.

[0037] Furthermore, a time span can be taken into account in the evaluation of whether a difference is notable. This can be, for example, a fixed threshold value of several seconds to several minutes, in which case the identified location change is only taken into account after the time threshold has elapsed. Advantageously, however, this time threshold is related to the usual dwell time of a customer in front of the rack. This ensures that the location change of the arrangement is only realized when the starting point can be the customer's departure from the rack or from the corresponding location in front of the rack. This measure can be very advantageous, for example, when several identical products are arranged next to each other and a virtual label is arranged in the middle of these products. If an item is removed from the end area of ​​the product group and the arrangement data is used to indicate a new arrangement of the virtual label for the remaining items, this new arrangement can wait until the starting point can be the customer's departure from the rack.

[0038] Basically, the camera and the image processing device can be designed separately from each other. This can be advantageous, for example, if the cheapest possible cameras are used, and the digital images obtained with them are transferred, for example in groups, to a central image processing device for the group of cameras, where the configuration data is generated. However, it is advantageous if the camera and the image processing device are integrated in one device, in particular the image processing device is integrated in the camera, and the configuration data is output wirelessly directly to the video rack rail or to a higher-level control device, which drives a large number, in particular all, of the video rack rails in the store with the individual playback content, or both. This measure does indeed require a special camera configuration, but at the same time it results in an infrastructure that is extremely easy to install, since the number of components to be installed is reduced by the amount of the image processing devices that would otherwise be required additionally or the connections of these devices to the cameras in question.

[0039] As already mentioned, it is advantageous to apply the method in a single camera configuration for multiple playback contents determined for playback on one screen, i.e. it is possible to use a single camera to arrange different playback contents for each screen.

[0040] Furthermore, it is particularly advantageous to apply the method in the form of using one camera for multiple playback contents determined for playback on multiple screens installed on the rack base plate arranged next to each other or on top of each other. This means that using a single camera it is also possible to arrange different playback contents on multiple screens arranged vertically and horizontally next to each other, i.e. a grouping of screens for one camera is obtained. It is also possible to control the arrangement of the playback contents for an entire rack, or even for multiple racks, using one camera, as long as the rack in question is within the camera's field of view.

[0041] The screens present in the shooting area can be uniquely identified to the camera or image processing equipment, for example by a visualized symbol, so that the allocation of the playback content to these screens and the positioning data determined therefor are unambiguous.

[0042] For example, during the initialization phase of the system or "on demand", i.e. when this is signalled by the camera itself or by a central server using a command to the video rack rail, the video rack rail can also display (especially for a short time) a symbol for carrying out a calibration of the system or of each camera. This ensures that the camera can carry out an unambiguous assignment between the display area of ​​each screen and the surroundings localised adjacent to it, in particular in the rack, naturally also taking into account the respective field of view of each camera. In this case, the camera stores the position and extent of the display area of ​​each screen and captures this information during the processing of the digital images by the image processing device. The symbol displayed for this purpose can be, for example, a line displayed along the outer edge of the display area, i.e. a line indicating the extent of the available display area. In addition to this, it can also be an alphanumeric display by a bar code or QR code or an unambiguous identification chart, which is photographed together during the initialization or calibration so as to be available during the course of the subsequent operation.

[0043] Finally, it is mentioned that, in general terms, the electronic devices considered here naturally comprise electronic components. These electronic components may consist of discrete or integrated electronic components or a combination of the two. Microcomputers, microcontrollers, application specific integrated circuits (ASICs) may be used, possibly in combination with analog or digital electronic peripheral modules. By means of software running on the processors of the electronic components, many of the aforementioned functions of these devices are realized, possibly in cooperation with hardware components. Devices configured for wireless communication usually comprise an antenna arrangement for transmitting and receiving radio signals, as part of a transceiver module. Furthermore, these electronic devices may comprise an internal power supply, which may be realized by means of a replaceable or rechargeable battery. These devices may also be powered in a wired configuration by an external power supply unit, by means of "Power over LAN" or by a power-transmitting radio signal.

[0044] These and other aspects of the present invention will become apparent from the drawings discussed below.

[0045] The invention will now be described in more detail again by way of examples with reference to the attached drawings, to which the invention is not limited, in which the same components are provided with the same reference numerals in the different drawings. [Brief description of the drawings]

[0046] [Figure 1] Schematic partial view of an ESL system with a rack with video rack rails attached showing a virtual ESL initially deployed. [Diagram 2] Schematic diagram of the cameras used in the ESL system [Diagram 3] FIG. 1 is a schematic diagram of a configuration for automatically changing the layout of a virtual ESL according to a first embodiment of the present invention; [Figure 4]Schematic diagram of another configuration for automatically changing the arrangement caused by the removal of products. [Diagram 5] FIG. 1 is a schematic diagram of a configuration for automatically changing the layout of a virtual ESL according to another embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0047] 1 illustrates an electronic shelf label system 1, hereinafter referred to as "system 1" for short, that uses video rack rails 2. Such a system 1 may be based solely on video rack rails 2, or may have a mixed infrastructure consisting of video rack rails 2 capable of displaying both dynamic playback content as well as virtual electronic shelf labels (virtual ESLs for short) and physical electronic shelf labels (physical ESLs for short) that are usually capable of displaying only static playback content. The focus of the present invention is only on the racks 3 in this system 1 and the equipment required for the operation of the video rack rails 2 mounted thereon.

[0048] The system 1 comprises a central server 4 which stores the planogram of the store. The planogram is a data structure which represents or describes the products and their associated virtual ESL locations, i.e. logically linked to each product (as well as the physical ESL locations), and their three-dimensional location in the store, specifically on the racks and the rack bases attached thereto. Each ESL is identifiable by a unique ESL code. The same is true for different products or product groups. In this example, the locations of the video rack rails are also described in the planogram, and each video rack rail is also uniquely identifiable by means of a unique video rack rail code.

[0049] This server 4 provides both static and dynamic playback content for display using the video rack rails 2 and physical ESLs, although in this example only the video rack rails 2 will be discussed.

[0050] The playback data WD representing each playback content is transmitted from the server 4 to each video rack rail 2 by means of a wireless base station 5, called base station 5 for short, and stored there. The video rack rails 2 themselves each have a wireless communication stage for video rack rails, not shown in detail, which is well known per se, and are integrated into the wireless network of the base station 5 by means of this stage.

[0051] Although only a single base station 5 is illustrated in this example, in an actual installation, naturally, multiple such base stations 5 are installed in a distributed manner within the store, and the video rack rails 2 are assigned to such base stations 5 in groups by wireless communication technology. In this case, different wireless channels of one frequency band are used by spatially adjacent base stations 5 to communicate with each group of video rack rails 2.

[0052] The server 4 knows the allocation of the video rack rails 2 to the base stations 5 and can therefore address each individual video rack rail 2 as intended and transmit to it the playback content that matches the products 6A-6D displayed on each rack bottom plate 7 of the rack 3, where not all the products 6A-6D are marked in order to avoid over-decorating the drawing with product-identifying codes, but the different products 6A-6D are uniquely identifiable by their special shape. In this case, the server 4 also transmits initial positioning data PDI, which the video rack rail 2 uses to position or position the playback content within the display area 8 of its (video) screen 9. The display area 8 is delimited by a dashed line, which indicates that the screen 9 is surrounded by a (not overly noticeable) thin frame. Naturally, this frame can also be very thin. It is also possible to omit the frame altogether.

[0053] In this example, it is assumed that a background video showing a panoramic landscape scene is transmitted to the video rack rail 2 as dynamic playback content, and the video rack rail plays this background video in a manner that fills the display area 8. On each of the two rack bottom plates 7, two groups of products are displayed side by side, i.e. products 6A and 6B are displayed side by side on the upper rack bottom plate 7, and products 6C and 6D are displayed side by side on the upper rack bottom plate 7. In this example, the products displayed on the upper rack bottom plate 7 are products used for hiking, such as hiking socks 6B packaged in an elongated, rather flat package and hiking shoes 6A packaged in a rather three-dimensional package. The products 6C and 6D displayed on the lower rack bottom plate 7 also have different packaging shapes, at least for the purposes of this discussion.

[0054] Thus, in this example, the virtual ESLs 11A-11D represent the product information and / or price information for the products 6A-6D, and these virtual ESLs 11A-11D are transmitted as static playback content from the server 4 to each video rack rail 2 together with the initial placement data PDI, and then each video rack rail 2 displays the virtual ESLs 11A-11D in the display area 8 based on the position given by the initial placement data PDI. In this example, the virtual ESLs 11A-11D are initially placed at the left or right edge of the display area 8, respectively. In this case, each virtual ESL 11A-11D, which fills only a small portion of the display area, is displayed in front of the background video. That is, each background video is partially obscured by the ESLs 11A-11D.

[0055] The system 1 further comprises a camera 12 fixed to a structure 13, not described in further detail below, which may for example be a wall, a ceiling, a pillar or another rack in a store, from which an unhindered view of the rack 3 to be photographed by the camera 12 is possible.

[0056] As shown in FIG. 2, the camera 12 comprises a lens 14, as is typically provided for digital image capture, and associated camera electronics 14A, which execute at least a first software implementing an image capture stage 15, by means of which a digital image DA of the rack 3 is created with all the objects present therein, i.e. the products 6A-6D and the screen 9 of the video rack rail 2. The camera electronics 14A also execute a second software implementing an image processing device 16, by means of which the position indicators and their positions in the digital image are detected. The second software is therefore optimized for pattern recognition in the digital image. Furthermore, the camera comprises a wireless communication stage for the camera, known per se, which is omitted from the illustration. By means of this stage, the camera 12 is integrated into the wireless network of the base station 5 and can communicate both with the server 4 and with the video rack rail 2.

[0057] In this example, in the rack 3, the different products 6A-6D are directly detected by the image sensing device 16 based on their different packaging, their shape or size or their labels (or by a combination of the evaluation of these aspects) and the display areas PBA, PBB, PBC, PBD of each group of products 6A-6D are identified by determining the respective areas they occupy in groups along each screen 2 or display area 8. In a further step, the centres of each display area PBA, PBB, PBC, PBD are identified and their absolute positions along the longitudinal extension of the display area 8 (for example measured from the left edge of the display area) are generated using the individual position data PDA for the first virtual ESL 11A, the individual position data PDB for the second virtual ESL 11B, the individual position data PDC for the third virtual ESL 11C and the individual position data PDD for the fourth virtual ESL 11D.

[0058] Within the scope of the data processing of the image processing device 16, each virtual ESL 11A-11D is also automatically detected, so that the ESL 11A-11D in question is uniquely identified. This unique identification can be performed by evaluation of product information and / or price information of each virtual ESL 11A-11D, or by detection of a unique identification code displayed with each ESL 11A-11D, or by determining the position of each ESL 11A-11D in the display area 8 of each screen 8. Since the video rack rail 2 stores one or more of the product information and / or price information of the virtual ESL 11A-11D in question, stores the identification code of the ESL 11A-11D in question, and also stores the position of the ESL 11A-11D in question, which in any case obscures the video, the video rack rail 2 in question knows at least one of the three parameters necessary for the unique identification of the virtual ESL 11A-11D in question in any case.

[0059] Subsequently, the positioning data PDA-PDD generated for each of the virtual ESLs 11A-11D is transmitted from the camera 12, which is also incorporated in the wireless network of the base station 5, to the corresponding video rack rail 2, where it is used for positioning each of the virtual ESLs 11A-11D. That is, by the control action of the positioning data PDA-PDD, the positions of the virtual ESLs 11A-11D are changed so that each of the virtual ESLs 11A-11D is positioned exactly at the center of each of the exhibition areas PBA-PBD.

[0060] It should also be mentioned here that the camera 12 can be configured such that, in addition to the layout data PDA-PDD, it is also possible to output data representative of the digital image DA as such, ie raw data of the image detector or raw data which has already been preprocessed.

[0061] The new arrangement of the virtual ESLs 11A-11D is shown in FIG. 3, where the arrows FA-FD indicate the positional deviation of each virtual ESL 11A-11D along the longitudinal extension of each video rack rail 2 or the display area 8 of its screen 9 caused by each of the arrangement data PDA-PDD, compared to each of the initial positions, similar to that shown in FIG. 1.

[0062] Similarly, an automatic allocation of the available display width for four different background videos constituting the dynamic playback content can also be performed, taking again the situation according to FIG.

[0063] First, for example, four background videos are planned or displayed per rack bottom plate 7, distributed 50% / 50% along the display area. Now, if the image processing device detects that on the upper rack bottom plate 7, the first product 6A occupies only 40% of the longitudinal extension of the screen 9 or its display area 8, and the second product 6B occupies 60%, corresponding first and second arrangement data PDA, PDB are generated, which define a 40% playing width for the background video associated with the first product 6A and a 60% playing width for the background video associated with the second product 6B on the upper video rack rail 2. These two background videos are scaled accordingly and are thus displayed in a manner that fits the display areas PBA and PBB. In order to prevent possible unnatural distortions due to non-proportional scaling of the width and height of the video images, it can also be stipulated that the part of the entire video image of the background video in question that fits the respective product display area PBA or PBB is displayed there.

[0064] The same is true for the lower video rack rail 2, in which the third product 6C occupies approximately 70% and the fourth product 6D occupies approximately 30% of the longitudinal extent of the display area 8. The lower video rack rail 2 correspondingly programs the image processing device 16 to display the third background video over 70% of the longitudinal extent of the display area 8 corresponding to the third display area PBC and to display the fourth background video over 30% of the longitudinal extent of the display area 8 corresponding to the fourth display area PBD.

[0065] Subsequently, as illustrated in FIG. 4 with reference to the top rack bottom plate 7, it can be assumed that the product display area PBA has changed due to the removal of products, specifically with respect to the product 6A, the right-hand product 6A of the two products 6A-6B has been removed from the top rack bottom plate 7, so that the product display area PBA has been approximately halved in its longitudinal dimension along the display area 8 and its center has shifted towards the left. This change is detected by the image processing device 16 of the camera 12, which generates new positioning data PDA for the first virtual ESL 11A, which are transmitted to the top video rack rail 2, causing there a new positioning of the first virtual ESL 11A in the center with respect to the final remaining product 6A. This is represented by the arrow FA indicating the deviation of the first virtual ESL 11A compared to the position illustrated in FIG. 3.

[0066] In another embodiment of the invention, the arrangement of the products 6A-6D, and possibly also the arrangement of the display areas PBA-PBD assigned to them, can be represented by so-called rack separators 17A-17D illustrated in Figure 5, in which case the rack separators constitute arrangement indicators which, in this example, are automatically detected by the image processing device 16, unlike Figure 1 in which the products 11A-11D themselves constitute arrangement indicators.

[0067] In this case, the rack separators 17A-17D can be identified in the digital image DA by the image processing device 16 on the basis of their special shape, which is clearly distinguishable from the goods. Advantageously, however, the rack separators are equipped with electronic components for the rack separators (not shown), which are also configured for wireless communication in the wireless network of the base station 5. In the course of such wireless communication, identification data are transmitted to each rack separator 17A-17D for uniquely identifying the goods 6A-6D arranged adjacent to it. The electronic components of the rack separators 17A-17D furthermore comprise optical fibers connected to LEDs 18A-18D visible from the outside through the front side of the housing or also through the top side or LEDs 18A-18D integrated in the housing. By means of these LEDs 18A-18D, the electronic components for the rack separators of each rack separator 17A-17D output a varying (modulated or flashing) light signal, by means of which the unique identification data as information content is also output. Using the individual optical signals, the individual rack separators 17A-17D are detected by the image processing device 16 as placement indicators and their positions are also detected.

[0068] By definition, the rack separators 17A-17D according to this example are arranged on the right side of each of the products 6A-6D as shown in FIG. 5. The first product display area PBA extends between the adjacent rack separators 17A and 17B. The third product display area PBC extends between the adjacent rack separators 17C and 17D. The second product display area PBB extends from the second rack separator 17B to the left side to the end of the upper rack bottom plate 7 or the upper video rack rail 2. The fourth product display area PBD extends from the fourth rack separator 17D to the left side to the end of the lower rack bottom plate 7 or the lower video rack rail 2. Of course, a terminal rack separator (not shown) that sends a neutral optical signal to the left side of each rack bottom plate 7 and is used simply to define the display areas PBB and PBD can also be provided.

[0069] Based on this configuration, the image processing device 16 can generate placement data for the virtual ESLs 11A to 11D and wirelessly transmit this data to the video rack rail 2, where a new placement of the virtual ESLs 11A to 11D is performed in the same manner as in FIG. 3.

[0070] The background video placement discussed previously can also be implemented using rack separators 17A-17D in a similar manner.

[0071] In many cases, the actively light-emitting position indicators are easier to detect by the image processing device 16 than the example of using the product itself as the position indicator. This reduces the error rate and greatly accelerates the detection or identification process. Therefore, it is also possible to adopt the form of using other actively light-emitting objects as position indicators. These only need to be suitable for designating or defining the location of the product displayed along the screen.

[0072] Finally, it is pointed out once again that the figures described in detail above are merely examples which can be modified in various ways by those skilled in the art without departing from the scope of the invention. For the sake of completeness, it is also pointed out that the use of the indefinite article "ein" or "eine" does not exclude the possibility of a plurality of the features in question.

Claims

1. A method for arranging playback content within a display area (8) of a screen (9) of a video rack rail (2), comprising the steps of: A digital image (DA) of the screen (9) and / or a product display area (PBA, PBB, PBC, PBD) extending along the screen (9) is taken using a camera (12); using image processing equipment (16) to automatically detect the location indicator and its position within the digital image (DA), the location indicator indicating or defining the location of the displayed item along the screen (9); using said image processing device (16) to generate configuration data (PDA, PDB, PDC, PDD) based on the detected position of said configuration indicator, said configuration data (PDA, PDB, PDC, PDD) defining where within the display area (8) the playback content is to be played; The method, wherein the video rack rail (2) plays playback content within a display area (8) of a screen (9) based on the configuration data (PDA, PDB, PDC, PDD).

2. 10. The method of claim 1 , The method, wherein the automatic detection of the location indicators is performed using computerized image content and / or pattern detection.

3. The method according to claim 1 or 2, The method in which optical signals, particularly optical signals in spectral bands invisible to humans, are detected as position indicators.

4. 4. The method of claim 3, The method, wherein said optical signal comprises a time- and / or spatially varying signal.

5. The method according to claim 3 or 4, The method wherein said optical signal is output from a screen (9).

6. The method according to claim 3 or 4, The method, wherein the optical signals are output from rack separators (17A, 17B, 17C, 17D) located within product display areas (PBA, PBB, PBC, PBD).

7. The method according to claim 1 or 2, The method detects products (6A, 6B, 6C, 6D) or groups of products within a product display area (PBA, PBB, PBC, PBD) as placement indicators.

8. The method according to any one of claims 1 to 7, The arrangement data (PDA, PDB, PDC, PDD) is output from an image processing device (16) to a higher-level control device, and this control device controls the arrangement of the playback content within the display area of ​​the screen (9) of the video rack rail (2) based on the arrangement data (PDA, PDB, PDC, PDD), in particular the method for individually controlling multiple such video rack rails (2).

9. The method according to any one of claims 1 to 8, The method includes outputting the arrangement data (PDA, PDB, PDC, PDD) directly from an image processing device (16) to a video rack rail (2) that has detected a digital image (DA), and the video rack rail (2) controls the arrangement of the content being played within the display area of ​​the screen (9) of the video rack rail (2) based on the arrangement data (PDA, PDB, PDC, PDD).

10. 10. The method according to claim 8 or 9, The above-mentioned configuration data (PDA, PDB, PDC, PDD) is in the following formats: A single coordinate defining a position within the viewing area along one direction, A pair of coordinates defining the location of a point within the display area, A line segment in the display area, Faces in the display area, a part of the display area, in particular a part corresponding to the product display areas (PBA, PBB, PBC, PBD) extending along the screen (9); The method describes or represents the arrangement form in the display area by at least one of the above.

11. The method according to any one of claims 8 to 10, The above-mentioned configuration data (PDA, PDB, PDC, PDD) the condition that the spatial change of the configuration caused by the configuration data (PDA, PDB, PDC, PDD) in comparison with the currently existing configuration exceeds a predefined threshold, in particular a predefined fixed threshold or a threshold that changes dynamically over time, and / or the condition that the time span for taking into account said configuration data (PDA, PDB, PDC, PDD) exceeds a time threshold, in particular a time threshold defined by a typical dwell time of a customer in front of the video rack rail (2); The method is used to place playback information only under

12. The method according to any one of claims 1 to 11, The camera (12) and the image processing device (16) are integrated into one device, in particular the image processing device (16) is integrated into the camera (12), The method in which the above-mentioned configuration data (PDA, PDB, PDC, PDD) is wirelessly output directly to the video rack rail (2) and / or to a higher-level control device, and this higher-level control device controls multiple, particularly all, of the video rack rails (2) in the store with individual playback content.

13. The method according to any one of claims 1 to 12, The method is applied in a manner that uses one camera (12) for multiple playback contents determined for playback using one screen.

14. The method according to any one of claims 1 to 13, This method is particularly applied in the form of using one camera (12) for multiple playback contents determined for playback using multiple screens (9) installed on a rack base plate arranged side by side or on top of each other.

Citation Information

Patent Citations

  • Commodity monitoring device, commodity monitoring system and commodity monitoring method

    JP2017162375A

  • Display controller and program

    JP2018097462A

  • Label generation device and program

    JP2018147028A

  • Information processing device, information processing method and program

    JP2019096127A

  • Product placement suitability assessment

    JP2020518044A