Intelligent marketing and advertising platform

Abstract

The present disclosure relates to a smart marketing and advertising platform that provides a retailer with an innovative sales solution by effectively converting glass surfaces of retail product containers (e.g., doors of coolers) into non-transparent displays of shelf maps. The sales solution provides digital shelf maps and pricing management, real-time promotion updates, sales data, and the like. This may be accomplished by transforming / converting a pure glass surface of a retail product container (e.g., a cooler / freezer door) into a digital "smart" screen that provides an innovative advertising solution. The chiller / freezer door is configured to capture images using at least one camera when the door is open in order to efficiently inventory items inside the chiller / freezer.

Description

[0001] This application is a divisional application of an application with a filing date of December 17, 2019, application number 201980090632.2, and invention name “Intelligent Marketing and Advertising Platform”. Technical Field

[0002] This application is a continuation-in-part of U.S. patent application serial number 16 / 222,643 filed on December 17, 2018, which is a continuation-in-part of International Patent Application No. PCT / US18 / 46103 filed on August 9, 2018, which is a continuation-in-part of U.S. patent application serial number 15 / 674,198 filed on August 10, 2017, and the entire text of all of the foregoing applications is incorporated herein by reference.

[0003] The present invention relates generally to merchandising solutions and, more particularly, to an intelligent marketing and advertising platform that employs digital "smart" screens on retail product containers to provide innovative advertising solutions. Background Art

[0004] Although more and more people are buying items online, physical sales are still significant and are actually growing in certain market segments such as grocery stores, drug stores, and convenience stores. More than 75% of physical store purchases are impulse purchases. Research shows that impulse decisions are largely influenced by in-store messaging, and most in-store impulse purchases occur in retail product containers such as coolers and freezers.

[0005] If impulse purchases could be interrupted by content displayed on retail product containers such as cooler and freezer doors, this would provide a tremendous value creation opportunity for both consumer packaged goods brands and retailers. Coolers are probably one of the most overlooked, least promoted, least technologically advanced and most challenging parts of a retail store.

[0006] Conventional retail product containers, such as coolers, freezers, vending machines, and the like, have glass panels or doors through which customers can view the products stored in the retail product containers and those products are available for purchase. Problems with this approach include the fact that the products stored therein are often not neatly displayed, products are blocked by other products so that customers cannot easily see which products are available for purchase, etc. In addition, store employees must maintain the appearance of the products stored therein, track inventory, and install and replace promotional labels and other displays (e.g., displays related to pricing and promotional programs) to keep the items current.

[0007] The problem is that coolers and freezers in grocery stores are difficult areas for brands to conduct effective "on-the-spot" advertising and promotional programs. Typically, brands rely on other advertising media (such as print, billboards, online and television advertising, etc.) to promote products that are retailed on the shelves of coolers / freezers and / or refrigerators in retail and convenience stores. Current advertising methods are also generally considered to be ineffective and not impactful. Recognizing these inefficiencies, several display manufacturers have explored the opportunity to convert the "see-thru" glass windows on cooler doors into transparent displays that can display advertisements while allowing shoppers to still see the products and continue their daily shopping experience. Transparent display technology has been developed to begin targeting these applications and will eventually be integrated into cooler doors by several manufacturers. These systems allow these "passive" doors to be converted into advertising platforms while minimizing the interruption of the typical consumer's shopping experience. Consumers can still see through and select the items they plan to pick up, while the store owner or media platform owner can display promotional videos and advertisements on the transparent display media. However, over the past few years it has become apparent that market penetration of such systems has been very slow.

[0008] There are many factors that limit adoption, such as:

[0009] 1) The technology is very expensive and requires extremely high upfront costs;

[0010] 2) The overall visual / aesthetics are not appealing to consumers; and

[0011] 3) Lack of widespread adoption will prevent operators from bringing in significant advertising revenue to justify the business case.

[0012] Brands are constantly looking for media platforms that will enable them to reach key groups of target customers. With regard to digital billboard media platforms, this translates into a need for a very significant installed base. Summary of the invention

[0013] The purpose of the embodiments of the present invention is to provide an intelligent marketing and advertising platform.

[0014] Briefly, embodiments of the present invention provide a smart marketing and advertising platform that includes a plurality of retail product containers (e.g., coolers, freezers, and / or vending machines). Each retail product container has an internal storage space and each includes at least one non-transparent display that prohibits viewing of the internal storage space of the retail product container. Each retail product container further includes customer detection hardware and inventory counting hardware.

[0015] The platform also includes a controller / data collector in communication with the retail product containers. The controller / data collector assembly is configured to control at least one non-transparent display of each retail product container such that each non-transparent display provides a planogram related to retail products physically contained within the interior storage space of the retail product container based on inventory counted by the controller / data collector using inventory counting hardware of the retail product container.

[0016] The controller / data collector is preferably configured to control at least one non-transparent display of each retail product container to display current pricing information regarding the product physically contained within the interior storage space of the retail container, wherein the pricing is indicated by the controller / data collector.

[0017] The controller / data collector is preferably configured to detect customers using the customer detection hardware, and to display a promotional offer on at least one non-transparent display of each retail product container based on what the controller / data collector detects using the customer detection hardware.

[0018] The smart marketing and advertising platform provides retailers with innovative merchandising solutions by effectively transforming the glass surface of a retail product container (e.g., a cooler door) into a non-transparent display of a planogram. The merchandising solution disclosed herein provides digital planogram and pricing management, real-time promotion updates and sales data, and more. The innovative advertising method disclosed herein for in-store retail billboards can be effectively implemented by converting / transforming the mere glass surface of a retail product container (e.g., a cooler / freezer door) into a digital "smart" screen that provides innovative advertising solutions.

[0019] The present disclosure also relates to the following aspects:

[0020] 1) A door for a retail storage container, the door comprising:

[0021] a display; and at least one camera directed toward the retail storage container, wherein the display displays different content depending on what the at least one camera detects.

[0022] 2) The door according to 1), wherein the door assembly is controlled by a controller, and the controller is in network communication with a plurality of other doors.

[0023] 3) The door according to 1), wherein when the door is opened, the at least one camera captures at least one image.

[0024] 4) The door according to 1), wherein when the door is opened, the at least one camera captures a plurality of images.

[0025] 5) The door according to 1), wherein the door has a back, wherein the at least one camera is mounted on the back of the door, wherein the at least one camera is tilted at an angle relative to the door.

[0026] 6) The door according to 1), wherein the door comprises a mechanical frame, wherein the at least one camera is accommodated in an empty frame space between the display and the mechanical frame.

[0027] 7) The door according to 1), wherein once the at least one camera detects that the door is opened, the at least one camera starts capturing images.

[0028] 8) The door according to 1), further comprising at least one accelerometer on the door, wherein once the at least one accelerometer detects that the door is opened, the at least one camera starts capturing images.

[0029] 9) An intelligent marketing and advertising platform, comprising:

[0030] A plurality of retail product containers, each having an internal storage space, each including at least one non-transparent display that prohibits viewing of the internal storage space of the retail product container, each further including customer detection hardware, and each including inventory counting hardware; a controller / data collector in communication with the plurality of retail product containers, wherein the controller / data collector assembly is configured to control the at least one non-transparent display of each retail product container so that each non-transparent display provides information about the internal storage space physically contained in the retail product container based on inventory counted by the controller / data collector using the inventory counting hardware of the retail product container. a planogram relating to retail products physically contained within the interior storage space of the retail container, wherein the controller / data collector assembly is configured to control the at least one non-transparent display of each retail product container to display current pricing information regarding products physically contained within the interior storage space of the retail container, wherein pricing is indicated by the controller / data collector, wherein the controller / data collector assembly is configured to detect customers using the customer detection hardware, and wherein the controller / data collector assembly is configured to display promotional offers on the at least one non-transparent display of each retail product container based on what the controller / data collector detects using the customer detection hardware, wherein the controller / data collector uses the inventory counting hardware to capture at least one image when a door is opened.

[0031] 10) The intelligent marketing and advertising platform according to 9), wherein the controller / data collector combination transmits data regarding the overall inventory related to the inventory of the plurality of retail product containers to a distributor and / or store.

[0032] 11) The smart marketing and advertising platform according to 9), wherein each retail product container comprises a cooler / freezer, and the at least one non-transparent display of each retail product container comprises a door of the cooler / freezer.

[0033] 12) The smart marketing and advertising platform according to 9), wherein the inventory counting hardware includes at least one camera that captures at least one image when the door is opened.

[0034] 13) The smart marketing and advertising platform according to 12), wherein the at least one camera captures a plurality of images when the door is opened.

[0035] 14) The smart marketing and advertising platform according to 12), wherein the door has a back, wherein the at least one camera is mounted on the back of the door, and wherein the at least one camera is tilted at an angle relative to the door.

[0036] 15) The smart marketing and advertising platform according to 12), wherein the door comprises a mechanical frame, wherein the at least one camera is accommodated in an empty frame space between the display and the mechanical frame.

[0037] 16) The intelligent marketing and advertising platform according to 12), wherein once the at least one camera detects that the door is opened, the at least one camera starts capturing images.

[0038] 17) The smart marketing and advertising platform according to 12) further comprises at least one accelerometer on the door, wherein once the at least one accelerometer detects that the door is opened, the at least one camera starts to capture images. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The organization and manner of operation of the present invention, together with further objects and advantages thereof, may best be understood by referring to the following description in conjunction with the accompanying drawings, in which like reference numerals represent like elements, and in which:

[0040] Figure 1 An intelligent marketing and advertising platform is shown, which provides a controller / data collector in network communication with a plurality of retail product containers;

[0041] Figure 2 Draw Figure 1 Some components of one of the plurality of retail product containers of the illustrated intelligent marketing and advertising platform;

[0042] Figure 3 Describe the hardware stack of the smart marketing and advertising platform;

[0043] Figure 4depicting the front of the door showing the retail product container;

[0044] Figure 5 Similar to Figure 4 , but it shows the back of the door;

[0045] Figure 6 Describe the solution architecture of the intelligent marketing and advertising platform;

[0046] Figure 7 A possible platform stack for an intelligent marketing and advertising platform is illustrated;

[0047] Fig. 8A and Figure 8B A possible system architecture that can be implemented in combination with the present invention is illustrated;

[0048] Fig. 9 A "sandwich" of possible content layers for the displays disclosed herein is depicted;

[0049] Fig.10 Describe the different types of advertisements that can be displayed on a display, such as full-size ads, banner ads, hot spots, logos and tags, and spot deals;

[0050] Fig.11 showing a cross section of a portion of a door of a retail product container;

[0051] Fig.12 A portion of the door is also shown;

[0052] Fig.13 is a block diagram of a method according to an embodiment of the present invention;

[0053] Fig.14 is an example of camera spacing on a door according to one or more examples of the present disclosure; and

[0054] Fig.15 Shown are different views of the retail product container and / or a front view and a top view of the door when the door is open. DETAILED DESCRIPTION

[0055] While the invention may be embodied in different forms, the specific embodiments shown in the drawings and described herein are to be understood as illustrative modes under the principles of the invention and are not intended to limit the invention as illustrated.

[0056] Figure 1 An intelligent marketing and advertising platform 10 is shown in accordance with an embodiment of the present invention. As shown, the intelligent marketing and advertising platform 10 provides a controller / data collector 12 in network communication with a plurality of retail product containers 14.

[0057] A plurality of retail product containers 14 need not be identical, but preferably, each is a device such as a cooler, freezer and / or vending machine. In any case, each preferably has an internal storage space in which the product for purchase can be stored.

[0058] like Figure 2 As shown, each retail product container 14 preferably includes at least one display 16, such as a display on a door of the retail product container 14.

[0059] Preferably, the display 16 comprises a non-transparent display that prohibits the customer from viewing the products stored in the internal storage space through the display. The display 16 may include, for example, a touch screen LCD with which the customer can interact.

[0060] Preferably, each retail product container 14 also includes customer detection hardware 18, such as one or more proximity sensors (eg, heatmaps), cameras, facial sensors or scanners, and eye sensors (eg, iris tracking sensors).

[0061] Assuming that a camera is used, it is preferably mounted on the door of the retail product container 14. Preferably, the camera has a depth of field of twenty feet or more and has a field of view of 170 degrees, preferably with facial recognition capabilities of 150 degrees. Preferably, software is used in conjunction with the camera to monitor shopper interactions, provide relevant advertising content on the display 16, and track advertising engagement within the store. The controller / data collector 12 can be configured to independently control each camera, and collectively control multiple displays 16 to provide single-screen and / or multi-screen content and interaction. Preferably, the controller / data collector 12 and display 16 are configured to allow banner ads and full-screen ads to be displayed simultaneously on a single display 16.

[0062] Preferably, each retail product container 14 further includes inventory counting hardware 20, such as additional cameras and / or sensors disposed within the retail product container 14 and facing the product.

[0063] like Figure 1As shown, the controller / data collector 12 is in network communication (preferably a cloud network) with the retail product containers 14, such as via conventional means such as the Internet, Bluetooth, etc., via wired Ethernet, wireless LAN, or cellular network. The controller / data collector 12 is configured to control the display 16 of the retail product container 14, and is configured to receive information from the retail product container 14, such as information from the display 16 (such as information about touch screen interaction), and information from the customer detection hardware 18 and inventory counting hardware 20 of the retail product container 14. As shown, each retail product container 14 includes an interface / attachment hardware 22, which, among other things, facilitates the network connection and information (i.e., data) transmission between the controller / data collector 12 and the retail product container 14, as well as the overall control and function of the display 16, the customer detection hardware 18, and the inventory counting hardware 20.

[0064] The controller / data collector 12 may include a server having one or more processors, memory storage, a user interface, etc., and is configured to not only indicate the display content on the display 16 of the retail product container 14 and receive information and data from the retail product container 14, but is also preferably configured to perform analysis based on the received information and data.

[0065] Preferably, the controller / data collector 12 assembly is configured to control the display 16 of each retail product container 14 so that each display 16 provides a planogram of retail products physically contained in the interior storage space of the retail product container 14 (but not viewable via the display 16) based on the inventory counted by the controller / data collector 12 using the inventory counting hardware 20 of the retail product container 14. Thus, the fact that the products stored in the retail product container 14 may not be neatly stored therein or that the products stored therein may be obscured by other products stored therein is not adversely affected. The planogram displayed on the display 16 of the retail product container 14 effectively optimizes the content to be presented to the customer.

[0066] Preferably, the controller / data collector 12 assembly is configured to control the display 16 of each retail product container 14 to display current pricing information regarding the product physically contained within the interior storage space of the retail product container 14, wherein the pricing is indicated by the controller / data collector 12.

[0067] Preferably, the controller / data collector 12 is configured to detect customers using the customer detection hardware 18 and to display promotional offers on the at least one display 16 based on what the controller / data collector 12 detected using the customer detection hardware 18 .

[0068] As described above, the controller / data collector 12 effectively counts each retail product container 14 (i.e., using the inventory counting hardware 20 in conjunction with the image recognition software). Preferably, the controller / data collector 12 is configured to transmit data regarding the overall inventory related to the inventory of a plurality of retail product containers 14 to the distributor and / or store. More specifically, the overall system may be configured to issue restock notifications, provide sales updates, pricing management, real-time sales data, and behavioral consumer analytics. Thus, the system provides advertising, flawless sales, real-time pricing and promotional programs, inventory and direct store delivery solutions, and analytical capabilities.

[0069] It should be noted that Figure 2 Only some of the components of the retail product container 14 are depicted, and the retail product container 14 will inevitably include other components such as perhaps a refrigeration unit, etc., which are not specifically related to the present invention.

[0070] Preferably, a plurality of retail product containers 14 and their associated displays 16 are arranged side by side down an aisle of a retail store, such as a grocery store, and the controller / data collector 12 assembly is configured to operate the displays 16 based on what is detected by both the customer detection hardware 18 and the inventory counting hardware 20.

[0071] For example, preferably, the controller / data collector 12 combination operates the display 16 so that if no motion is detected (or no longer detected) by the customer detection hardware 18, the display 16 displays a full-screen advertisement. In other words, when a customer approaches the aisle, the customer sees a full-screen advertisement on the display 16.

[0072] Preferably, the controller / data collector 12 is configured so that if motion is detected within twenty feet of a retail product container 14 (i.e., by shopper detection hardware 18 mounted on the door of the retail product container 14), the controller / data collector 12 operates that particular display 16 to cease displaying the full screen advertisement and instead display a planogram of the products located within the retail product container 14. Preferably, a banner advertisement and pricing tag are also loaded and displayed simultaneously.

[0073] Preferably, the controller / data collector 12 combination manages the sale of advertising objects and layers. This enables it to be conveniently managed remotely or locally at individual retail locations, such as by using a content management application.

[0074] Preferably, the controller / data collector 12 is configured so that when a shopper approaches a given retail product container 14, the display 16 associated with the retail product container 14 changes again. For example, the display 16 may be changed so that it then displays a logo (e.g., organic, non-GMO, etc.) and a label (e.g., sold with a regional card, locally made, popularity, etc.). These promotional logos and labels may be activated by the controller / data collector 12 and may be selected from a set of predetermined fixed animated icons or may be customized by the retailer and / or brand based on their business and market needs. The controller / data collector 12 may be configured to incorporate customer rating scores (e.g., 1-5 stars, popularity, likeability, etc.) using one or more consumer review sources.

[0075] Preferably, the controller / data collector 12 is configured so that when a shopper stands or hovers in front of a given retail product container 14, the display 16 associated with the retail product container 14 changes again. In this regard, preferably, the controller / data collector 12 can use customer detection hardware 18 to effectively learn more about the particular customer, such as gender, age, mood, etc. The controller / data collector 12 is configured to use the information detected about the customer to determine which advertisements and other information to present to the particular customer on the display 16 associated with the retail product container 14 in front of which the customer is standing. By tracking the shopper's data and simultaneously playing advertising content on all displays 16 within the shopper's viewing range, retailers and brands can be better served, new influence opportunities can be provided when making purchase decisions, marketing expenditures can be optimized, and new revenue sources can be generated.

[0076] Preferably, the controller / data collector 12 and display 16 combination is configured so that when a shopper stands in front of a given display 16 (as detected by a camera mounted on the door of that particular retail product container 14), relevant conditional advertising, such as real-time data about news events, weather, sports games, etc., is displayed on that display 16 (i.e., in addition to information and advertising related to the contents of the retail product container 14, such as banner ads, horizontal ads, full screen ads, labels, signs, interstitials, etc.).

[0077] As described above, the controller / data collector 12 assembly is preferably configured to track the inventory of a given retail product container 14 through the use of inventory counting hardware 20, such as one or more cameras and / or sensors, etc., inside the retail product container 14. Preferably, when the controller / data collector 12 determines that a given product is out of stock, the controller / data collector 12 takes this into account when controlling the display content of the display 16 of the particular retail product container 14. For example, out of stock artwork may be displayed and / or advertisements may be exchanged so that out of stock advertising opportunities can be sold to different brands, wherein certain advertisements are displayed in certain out of stock situations.

[0078] Figure 3 The hardware stack of the platform is shown. As shown, the hardware stack consists of an LCD display (i.e., display 16), display content provided by a dedicated PC-based media player, connected to the network via an Edge router, and equipped with smart IOT devices (i.e., cameras and sensors). Figure 6 The diagrams shown depict system configuration diagrams and flow charts of the advertising content and key hardware components involved. Preferably, a dedicated UI / UX content product that is centrally (remotely) managed via the network is created. This will facilitate remote management of media scheduling and any layout modifications.

[0079] In addition, preferably all inputs collected by the IOT devices will be analyzed locally and remotely (via the cloud) to provide feedback input to the system to push more relevant / targeted content customized to the consumer. Preferably the analysis will be performed anonymously, and preferably the images captured by the camera will be processed to collect statistics about customer demographic characteristics (such as age and gender). Preferably this information will be subsequently analyzed, and these additional statistics are valuable to retailers for in-store merchandising layout design and smart merchandising, including the ability to track shopper "flow" areas, called "heat maps", i.e. areas where customers tend to concentrate more, spend more time exploring, etc.

[0080] Preferably, an inward-facing camera is used to ensure that the shelf is constantly monitored for out-of-stock alerts. When a product is sold out, the signal preferably alerts the system to update the layout on the display and alert the store operator.

[0081] Preferably, by default, the platform is preferably programmed to perform full screen advertising. Customer-facing cameras are programmed to have motion detection capabilities, analyze approaching customers, and switch the system to display a planogram of products placed on the shelf. Further analysis based on the images captured by these cameras preferably triggers customized promotional advertising.

[0082] Preferably, at least a portion of the display is configured with interactive touch capabilities to enable shoppers to browse further detailed information on various products while minimizing disruption to other shoppers who can view all of the products displayed on the planogram and who can proceed and select preferred products.

[0083] The system can be configured to track consumers' eye movements (e.g., by using powerful software tools coupled to hardware that uses infrared light sources to track eye movements) and correlate gaze points with advertisements. This amounts to a very powerful demonstration tool for measuring the success of an advertisement and its ability to influence a consumer's shopping decision.

[0084] As described above, preferably, each retail storage container 14 has a door and a display 16 mounted on the door together with customer detection hardware 18, which may be, for example, one or more proximity sensors (such as heat maps), cameras, facial sensors or scanners, and eye sensors (such as iris tracking sensors). Figure 4 The front portion 31 of the door 30 is shown, which provides a display 16 and customer detection hardware 18 such as a camera. As shown, the display 16 is preferably in the form of an LCD panel that includes one or more touch areas 32 that can interact with the customer. An access panel 34 can be provided near the bottom of the door 30 along with a media player 36. Although not specifically shown, a protective panel can generally be mounted above the display 16.

[0085] like Figure 5 As shown, a camera or other type of sensor (i.e., inventory counting hardware 20) may be provided on the rear portion 33 of the door 30, and the controller / data collector 12 (e.g., Figure 1 ) can be used to track the internal contents of retail product containers 14.

[0086] The smart marketing and advertising platform provides retailers with innovative merchandising solutions by effectively transforming the glass surface of a retail product container (e.g., a cooler door) into a non-transparent display of a planogram. The merchandising solution disclosed herein provides for digital planogram and pricing management, real-time promotion updates and sales data, etc. The innovative advertising method disclosed herein for in-store retail billboards is effectively achieved by converting / transforming the plain glass surface of a retail product container (e.g., a cooler / freezer door) into a digital "smart" screen that provides innovative advertising solutions.

[0087] As a business model, retail store cooler doors can be upgraded to "smart" cooler doors with displays, cameras and sensors that measure, react, learn and communicate in order to optimize indicators.

[0088] The smart marketing and advertising platform disclosed in this article boasts at least the following aspects: the integration of physical retail stores and e-commerce, in-store promotion programs similar to electronic markets, real-time and algorithm-driven pricing and promotion programs, self-learning / machine learning artificial intelligence algorithm-driven advertising that is personalized for given customers, behavioral responses and external data (such as weather, events, competitions, etc.), and a smart marketing and advertising platform for realizing shelf Figure 1 Smart sensors and digital sales, including consistency, automatic replenishment, in-store auditing, etc.

[0089] The advertising method disclosed herein for in-store retail billboards effectively employs two primary techniques: large-scale brand / product advertising carousels and planogram screens with non-obtrusive banner ads or hotspots (preferably activated when a customer comes within three feet).

[0090] Figure 6 Draw the solution architecture of the intelligent marketing and advertising platform disclosed in this article. Figure 7 A possible platform stack of the intelligent marketing and advertising platform disclosed in this article is illustrated, and Fig. 8A and Figure 8B A possible system architecture that can be implemented in conjunction with the present invention is shown. Fig. 9 A sandwich type layer of possible content is shown for the display 16 disclosed herein. Fig.10 Different types of advertisements that may be displayed on the display 16 are shown, such as full-size advertisements, banner advertisements, hot spots, tags and logos, and spot deals. Figures 6 to 10 It is self-explanatory.

[0091] As described above, each retail product container 14 preferably includes inventory counting hardware 20, such as a camera, disposed within the retail product container 14 and facing the product. Specifically, the camera can be disposed on the rear portion 33 of the door 30, and the controller / data collector 12 (such as Figure 1 ) can be used to track the inventory inside the relevant retail product container 14. In other words, the controller / data collector 12 is preferably configured to track the inventory of a given retail product container 14 by using a camera.

[0092] Accordingly, embodiments of the present invention effectively provide an automated capability to detect product inventory placed within each retail product container 14 and update a planogram showing the products on a display 16 (e.g., an LCD screen). If any item has gone "out of stock," the system is preferably configured to detect and update the information on the display 16 to guide consumers and store operators.

[0093] Preferably, the automatic detection system utilizes a camera that is aimed at the interior shelf of the retail product container 14 to capture images in combination with software that utilizes intelligent algorithms to perform image analysis and then extract the desired information from the analysis results.

[0094] Although embodiments are provided in which a plurality of cameras are arranged in an array to image and the individual images are combined together to form the entire shelf area by using software, an array of cameras presents challenges because all cameras must be plugged into a running PC, which typically has a limited number of inputs. Fig.11 A more preferred embodiment is shown, wherein a plurality of cameras (one of which is located at Fig.11 The camera is preferably mounted on an edge of the back 33 of the door 30 and is preferably housed in a bracket 42 that is tilted at an angle so that it can take pictures when the door 30 is open.

[0095] Although embodiments of the present invention provide for the use of cameras with wide fields of view (e.g., 180° FOV ("fisheye")) because they provide coverage of a large area, the image quality of such cameras is not suitable because objects are severely distorted at the edges, limiting the ability of the software to "recognize" the image at the image processing step. Therefore, a more preferred embodiment provides for a miniature camera with a 120° field of view (diagonal) to be mounted on a housing that is angled 45° along the edge of the handle side of the door 30. As in Fig.11 As shown in FIG. 4 , the camera is preferably housed in the empty frame space 43 between the display panel assembly 44 and the mechanical frame 46 of the unit. Any number of cameras may be disposed inside each door of each retail product container 14. For example, Fig.12 As shown, three cameras 40 may be distributed at equal distances along the height of each door 30 to allow complete coverage of the height and width of the shelf space (ie, within the retail product container 14).

[0096] Preferably, the system is configured so that during door operation (i.e., when the door 30 is opened), the camera 40 is triggered and takes pictures at various time intervals when the door 30 is opened. Preferably, the system is configured to process the images and effectively reconstruct the entire shelf. The system can be configured to trigger the camera in any number of ways. For example, the system can be configured to trigger the camera by:

[0097] 1) Motion is sensed by one or more cameras (if this is the case, it is best to configure the system so that at least one of the cameras remains powered on at all times); and / or

[0098] 2) by one or more accelerometers 48 on the door 30 (see Fig.12) effectively senses the movement of the door, wherein the system is configured to trigger the action of the camera 40 (i.e., when the door 30 is opened) depending on the various swing states and positions of the door 30.

[0099] Regardless of the exact manner in which the cameras are triggered, the system is preferably configured so that the images captured by the cameras are effectively "stitched" together to reconstruct the entire shelf space. The system is preferably configured to utilize image processing to compare the images and conclude whether any products are missing from the shelf. If any particular product is missing, the system is preferably configured to send a signal to the content management software to appropriately update the planogram (i.e., on the display 16), preferably displaying that product as "out of stock." Additionally, the system is preferably configured to notify the store inventory management team so that a quick restocking of that particular item on a particular shelf can be performed.

[0100] In the aforementioned embodiments involving stitching together multiple captured images, the disclosed system may capture multiple images using the first, second, and third cameras as the door rotates. For example, a first photo may be taken by the first camera, a second photo may be taken by the second camera, and a third photo may be taken by the third camera. A controller on or near the retail product container may combine relevant portions of the three photos into a single, composite image. Alternatively, in addition to a single photo from each of the three cameras, the first camera may take multiple photos and combine them with one or more photos taken by the second and third cameras. In other words, as the door rotates, the controller may instruct the multiple cameras to capture a number of images, thereby capturing images of the internal contents of the retail product container from different positions / angles. In one example, if the controller receives a measurement that the door swings at a higher rotational speed, the controller may change the frequency of photo capture, that is, the controller may increase the number of images captured. In another example, the controller may change the frequency and / or other parameters of the camera based on other circumstances or environmental factors. For example, in the case of low light, the operating parameters of one or more cameras may be adjusted to accommodate the demand. In other examples, moisture or condensation on a camera lens may be detected and cause the controller to modify the affected camera operating parameters.

[0101] In other examples, the plurality of cameras mounted on the rotatable door of the retail product container may be one or more motion video cameras. As such, the plurality of video cameras may capture a plurality of video frames from a starting position of the door rotation to a predetermined end position. The end position may be a set predetermined position, a predetermined range, or a predetermined relative rotation angle from the starting position (e.g., once the door has rotated 60 degrees from the starting position).

[0102] In one example, multiple images captured by the system using the first, second, and third cameras as the door rotates may be sent to an electronic controller at or near the retail product container. The controller may execute computer executable instructions to digitally combine the multiple images into a single panoramic, composite image. The size of the composite image may be large, and therefore consume a considerable amount of computer memory / storage space on the controller. In some embodiments, the composite image may be post-processed to reduce the storage space consumed by the image. In one example, artificial intelligence using a neural network may be used to identify the boundary edges of relevant portions of the contents of the retail product container, and the neural network may be, for example, a convolutional neural network. Therefore, those areas outside the boundary edges of the image may be positively discarded by a post-processing module executed on the controller. The out-of-bounds areas may be those portions of the image outside the border of the retail product container. At least one benefit of the post-processing of the foregoing example is to save memory storage capacity at the edge location. In addition, the smaller image size also results in less network bandwidth being consumed when the image is transmitted from the controller to one or more remote servers in the cloud.

[0103] In one example, a controller may be coupled to one or more cameras mounted in or on a door to automatically adjust parameters. The controller may adjust camera parameters such as, but not limited to, exposure, focus position, sensor gain / ISO speed, aperture size, and the like. The exposure value may be in units of time, and other parameters may be expressed in other units in different operating scenarios. In addition, one skilled in the art will appreciate, after reviewing the disclosure as a whole, that one or more parameters may be interrelated or interdependent. For example, an exposure of 1 / 25 second at f / 11, ISO 100 is equivalent to an exposure of 1 / 400 second at f / 2.8, ISO 100. In other words, since the shutter speed is reduced by four stops, this means that the image sensor in the camera captures less light. Therefore, the size of the aperture is increased by four stops to allow more light to enter the camera assembly. One skilled in the art will appreciate that different ways of adjusting parameters each have their own advantages and disadvantages. For example, to maximize exposure, the camera settings / parameters may be set to a large aperture, 6400 ISO, and a slow shutter speed. Instead, to minimize exposure, the camera settings / parameters are set to a small aperture, 100 ISO, and a fast shutter speed. Of course, the clarity of the captured image may be affected by the depth of field, aperture, and shutter speed parameters. For many of the examples disclosed herein involving capturing images of stationary retail products stored in retail product containers at relatively close distances, the ability to capture images without causing blur or planar warping is contemplated. Additionally, in some examples, the image sensor of the camera may be in motion at the time of image capture, thereby causing relative motion between the captured object and the camera.

[0104] With respect to blur, the controller may execute computer executable instructions (e.g., firmware, software, or an application specific integrated circuit) to perform post-processing on the captured image to reduce blur. In one example, a blur reduction module in the controller may identify portions of the captured image that are repeated across multiple captured images. And, then select appropriate portions from the repeated images where blur (and other undesirable image characteristics) are reduced. As explained herein, the portions identified by the blur reduction module may be incorporated into the final, composite image during the process of stitching the multiple captured images together.

[0105] Reference Fig.11 , when some camera types are rotated 40°, they may require a thickness of more than 6 mm due to the width of the circuit board. Fig.11 The camera is shown positioned to protrude beyond the thin cover of the display screen assembly. In some embodiments, the camera is positioned to be located on the same plane as the camera lens in the display screen assembly. Meanwhile, in other embodiments, the camera is intentionally positioned to extend beyond the edge of the LCD / LED display unit, Fig.11 A large light-colored object is shown at the lower left of the image. At least one benefit of this positioning is that the camera's view may not be obstructed. In some embodiments, this positioning may result in the camera being positioned approximately 28" from the hinge axis. In other embodiments, the positioning may be within a range including 28". At the same time, in other embodiments, the position may be outside of any such range, depending on the implementation.

[0106] In one illustrative embodiment, the camera may be four inches from the shelf, and the retail products on the shelf may be generally centered relative to the door frame. At the same time, the camera may have a diagonal field of view (FOV) of approximately 120° and be aimed in this direction so that the field of view is wider at the camera's small size. That is, the camera is placed near the handle side of the door and tilted 40° toward the hinge side without its field of view being obstructed by the display screen inside the door. Although one embodiment identifies a 4" positioning, the disclosure is not limited to this. In some examples, the camera to shelf distance may be set to a value other than four inches.

[0107] Reference Fig.12 , which illustrates a portion of a door of a retail product container. When the door is opened at any angle greater than about 25°, such as Fig.12 The three cameras shown are oriented to capture the entire shelf of retail product containers. In some other examples, the type of camera and / or the specifications of the camera may result in an angle greater or less than 25°.

[0108] In those examples where the cameras are constructed with vertically oriented sensors, two cameras are sufficient. If the cameras are located 21" from both the top and bottom edges and 30" from each other, then two cameras 28" or more from the hinge, rotated approximately 50° toward the hinge edge of the door, can view the entire scene of the contents of the retail product container. Of course, those skilled in the art will understand after reviewing the entire contents of this disclosure that this disclosure is not limited to the 21", 30", and 50 degree specifications indicated in the previous examples. Instead, it should be understood based on the basic teachings of this disclosure that the specifications are adjustable.

[0109] Fig.13 A block diagram of a method according to an embodiment of the present invention is provided and is self-illustrative given the preceding description.

[0110] Reference Fig.14 , which shows an example of the spacing of cameras on the door of a retail product container. Fig.14 In the example shown, the top and bottom cameras are 16.5" from the outer surface of the door frame. Also, the center camera is centered on the edge of the door and 20" from each of the other two cameras. In other examples where the door is not vertically centered to the products on the shelf within the retail product container, the position of one or more of the three cameras may be moved up or down accordingly. The camera may be tilted so that its FOV slightly misses the inner surface of the door. Since the hinge is close to the shelf edge, this positioning maintains the hinge side of the scene at the edge of the camera image for almost all door angles.

[0111] Reference Fig.15 , four exemplary views show different views of an exemplary retail product container and door from the front and top when the door is open approximately 44° or 25°. The camera's field of view (FOV) is shown as a transparent pyramid (triangle in the top view). The surface of the shelf is shown as a (red) interior plane. When the door is open 30° or more, the entire scene is contained in the FOV. When the door angle is reduced to 25°, the portion of the scene on the handle side begins to move out of the camera FOV. After reviewing the overall content of this disclosure, those skilled in the art will understand that this disclosure is not limited to the specific angles disclosed in the aforementioned examples. On the contrary, angles and specifications can be adjusted based on the basic teachings of this disclosure.

[0112] While particular embodiments of the present invention have been shown and described, it will be apparent to those skilled in the art that various modifications may be devised without departing from the spirit and scope of the invention.

Claims

1. A retail product container comprising: A display case door including a non-transparent display that prohibits viewing of an interior storage space of said retail product container, wherein: the display case door of the retail product container being movable from an open position to a closed position, wherein when in the closed position, one or more retail products physically contained within the retail product container are blocked from being visible from an exterior of the retail product container; customer detection hardware including one or more customer-facing cameras mounted on said display case doors of said retail product containers and configured to monitor for the presence of customers; Inventory hardware, including: at least one camera mounted on a backside of the display case door facing the retail products physically contained within the interior storage space of the retail product container, and wherein the at least one camera is triggered by door motion to capture images at predetermined time intervals during door operation, and at least one sensor disposed within the retail product container to monitor door movement during operation of the door; and a controller in communication with the retail product container, wherein the controller is configured to control the non-transparent display of the retail product container such that the non-transparent display provides a planogram associated with the retail products physically contained in the interior storage space of the retail product container based on inventory counted by the controller using the inventory counting hardware of the retail product container, wherein the controller is configured to control the non-transparent display of the retail product container to display current pricing information regarding the products physically contained in the interior storage space of the retail product container, wherein the current pricing information is indicated by the controller, wherein the controller is configured to detect customers using the customer detection hardware, and wherein the controller is configured to display a promotion on the non-transparent display of the retail product container based on what the controller detects using the customer detection hardware, wherein the controller uses the inventory counting hardware to capture at least one image of the interior storage space when the display case door is open.

2. The retail product container according to claim 1, wherein: The customer detection hardware also includes proximity sensors, cameras, face sensors, and eye sensors.

3. The retail product container of claim 1, wherein: The one or more customer-facing cameras have a depth of field of twenty feet or more and have a field of view of 170 degrees, preferably with facial recognition capabilities of 150 degrees.

4. The retail product container of claim 1, wherein: The non-transparent display provides a planogram that optimizes the presentation of retail products that are obstructed by other retail products, and wherein the non-transparent display includes a background effects layer, a permanent label layer, an inventory product asset and effects layer, a shelf space effects layer, a sign and label advertising product layer, an overlay advertising product layer, and a foreground glass effects layer.

5. The retail product container of claim 1, wherein: The display cabinet door also includes: a display holder fixed to a door frame of the display case door, and the display holder includes a mounting frame fixed to the door frame, the mounting frame having a coupling end, the coupling end being arranged to extend over an edge of the non-transparent display to fix the non-transparent display to the door frame; a transparent protective panel configured to be positioned between the front surface of the non-transparent display and the coupling end of the mounting bezel; and A covering frame is configured to be coupled with a front surface of the door frame, the covering frame being sized to overlap the front surface of the door frame with the coupling end of the mounting frame.

6. The retail product container of claim 1, wherein: The controller is configured to operate the non-transparent display to display a planogram of the retail products after determining that the shopper detection hardware has not detected motion.

7. The retail product container of claim 1, wherein: The controller is configured to operate the non-transparent display to display a planogram associated with retail products physically contained in the interior storage space of the retail product container upon determining that the shopper detection hardware detected motion.

8. The retail product container of claim 1, wherein: The controller is configured to operate the non-transparent display to display a promotional logo and label after determining that a customer is detected by the customer detection hardware within a predetermined distance of the retail product container.

9. The retail product container of claim 1, wherein: The controller is configured to operate the non-transparent display to display an advertisement specific to a particular customer after determining that the customer detection hardware has detected a particular customer loitering near the retail product container for a predetermined period of time.

10. The retail product container of claim 1, wherein: The retail product container comprises a cooler or a freezer, and the non-transparent display of the retail product container is mounted on the display case door of the cooler or the freezer.

11. The retail product container of claim 1, wherein: The inventory counting hardware includes at least one camera that captures at least one image when the display case door is open.

12. The retail product container of claim 11, wherein: The at least one camera captures a plurality of images when the display case door is opened.

13. The retail product container of claim 1, wherein: Once the at least one camera detects that the display cabinet door is opened, the at least one camera starts capturing images.

14. The retail product container of claim 1, wherein: The at least one sensor includes at least one accelerometer on the display case door, wherein once the at least one accelerometer detects that the display case door is opened, the at least one camera begins capturing images.

15. The retail product container of claim 1, wherein: The at least one camera is positioned to extend beyond an edge of the non-transparent display.

16. The retail product container of claim 14, wherein: The at least one camera is placed on the handle side of the display cabinet door and has a predetermined inclination toward the hinge side of the display cabinet door, and the field of view of the at least one camera is not blocked by the non-transparent display.

17. The retail product container of claim 16, wherein: The predetermined inclination is 40°.

18. The retail product container of claim 14, wherein: The at least one camera includes three cameras.

19. The retail product container of claim 14, wherein: The at least one camera consists of two cameras.

20. A system comprising: A server, the server comprising: one or more server processors; and a server memory storing computer executable instructions that, when executed by the one or more server processors, cause the server to: identifying inventory of one or more retail products physically contained in the retail product container via image recognition using inventory counting hardware attached to the retail product container; generating a planogram of the one or more retail products physically contained in the retail product containers; displaying the planogram on a non-transparent display on a door of the retail product container, wherein the door of the retail product container is movable from an open position to a closed position, and wherein, when in the closed position, the non-transparent display blocks visibility of the one or more retail products physically contained in the retail product container from an exterior of the retail product container; determining, in real time, current pricing information regarding the one or more retail products physically contained in the retail product container; detecting, using customer detection hardware, that a customer has lingered near the retail product container for a predetermined period of time; determining in real time a promotional offer for the customer detected by the customer detection hardware; and displaying on the non-transparent display a promotional offer for the customer and the current pricing information for the one or more retail products physically contained within the retail product container, The inventory counting hardware includes: at least one camera mounted on a back of the door of the retail product container, and the at least one camera faces the retail products physically contained in the interior storage space of the retail product container, and wherein the at least one camera is triggered by door motion to capture images at predetermined time intervals during door operation, and at least one sensor disposed within the retail product container to monitor door movement during operation of the door; and Wherein, the customer detection hardware comprises one or more customer-facing cameras mounted on the door of the retail product container and configured to monitor the presence of customers.

Images