Electronic system for intelligent container assistance
The electronic system with a container assistant and camera in shopping carts addresses shopper reluctance by ensuring valid item placement and reducing fraud, enhancing shopping efficiency.
Patent Information
- Application Number
- PCT/CN2024/078633
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-02-26
- Publication Date
- 2025-09-04
AI Technical Summary
Shoppers are reluctant to install retailer software applications on their mobile devices due to storage, ergonomic, privacy, and security concerns, and existing solutions do not adequately address theft or fraud during shopping events.
An electronic system comprising a container assistant with a camera and handheld device holder, which securely attaches to shopping carts to capture images of items and detect valid entry status, using a removable electrical connector for charging and communication.
Facilitates efficient item scanning and theft prevention by ensuring items are properly placed in containers, reducing reliance on mobile devices and minimizing fraud.
Smart Images

Figure CN2024078633_04092025_PF_FP_ABST
Abstract
Description
ELECTRONIC SYSTEM FOR INTELLIGENT CONTAINER ASSISTANCETECHNICAL FIELD
[0001] The present invention relates generally to electronic devices that assist with holding and storing objects in containers, including during shopping events within a store.BACKGROUND
[0002] Shoppers can be reluctant to install retailer software applications on their mobile devices (e.g., smartphones) for shopping. For example, shoppers may find installing such applications to be time consuming or that their mobile devices lack sufficient storage space, or may have privacy and / or security concerns installing such software applications on their mobile devices. In addition or alternatively, shoppers may be deterred from using such software applications on their mobile devices due to structural and / or operational limitations of their mobile devices, such as the mobile device is not sufficiently ergonomic to use as an electronic shopping assistant, the mobile device has too small of a screen size, the camera of the mobile device is too difficult to use for scanning, the battery of the mobile device does not provide sufficient charge to also allow the mobile device to be used as a shopping assistant, the software application is too cumbersome to use during a shopping event if the shopper receives a call while shopping, or the mobile device can break if dropped during a shopping event, as non-limiting examples. As such, ways for a retailer to provide shoppers access to one or more electronic devices, separate from their mobile devices, that address one or more of these deficiencies, and in turn encourage use of electronic devices as a shopper assistant, may be desirable. At the same time, leveraging such electronic devices to reduce or minimize theft or fraud may be desirable.
[0003] BRIEF SUMMARY
[0004] The present description describes various implementations of electronic systems, apparatuses, devices, controllers, and method related to a container assistant. In some implementations, an apparatus for valid entry detection includes: a body and a camera. The body includes a container attachment portion configured to attach to a corresponding attachment area of a container, the container attachment portion including an inner surface configured to engage with a corresponding surface of the attachment area to establish a secure attachment to the attachment area of the container; and a handheld device holder portion configured to receive and hold an electronic handheld device, the handheld device holder portion comprising a wall having an inner surface defining a volume for receipt of the electronic handheld device, the inner surface comprising a holder electrical connector configured to physically and electrically connect to a corresponding device electrical connector upon receipt of the electronic handheld device within the volume. The camera is disposed within the body, and is oriented relative to the container attachment portion such that when the container attachment portion establishes the secure attachment to the attachment area of the container, the camera is configured to establish an image capture region adjacent to an opening of the container.
[0005] In some other implementations, a system includes: an electronic handheld device and a valid entry detection apparatus for a container. The electronic handheld device includes a display, a scanner, and a handle. The valid entry detection apparatus includes: a body including: a container attachment portion configured to attach to an attachment area of the container; and a handheld device holder portion configured to removably receive and hold the handle of the electronic handheld device; and a camera disposed in the body, the camera configured to establish an image capture region adjacent to an opening of the container when the valid entry detection apparatus is securely attached to the attachment area of the container.
[0006] In some other implementations, a system includes: a container configured to hold one or more items, the container comprising an attachment area; and a valid entry detection apparatus comprising a body and a camera. The body includes a container attachment portion attached to the attachment area of the container; and a handheld device holder portion configured to removably receive and hold an electronic handheld device. The camera is disposed in the body and is configured to establish an image capture region adjacent to an opening of the container when the valid entry detection apparatus is attached to the attachment area of the container.
[0007] In some other implementations, a method of configuring a container for valid entry status detection includes: attaching a valid detection status apparatus to an attachment area of a container, wherein the valid entry status apparatus comprises a camera; and establishing an image capture region adjacent to an opening of the container.
[0008] In some other implementations, a container valid entry detection method includes: sensing, by a sensor, a presence of an object in an image capture region adjacent to an opening of a container; capturing, by a camera, at least one image of the object in the image capture region in response to the sensor sensing the presence of the object in the image capture region as the object is moving to enter into the container; detecting, by a valid entry detection controller, a valid entry status of the object in the container based on the at least one image and a scanning event; outputting, by an output device, the valid entry status of the object.
[0009] In some other implementations, a system includes: a memory storing a plurality of instructions; and a processor configured to execute the plurality of instructions, and upon execution of the plurality of instructions, is configured to: receive a captured image data set of a first object in an image capture region adjacent to an opening of a container; receive a representative image data set associated with a second object in response to a scanning event performed by a scanner on the second object; compare the captured image data set with the representative image data set to determine a degree of similarity between the first object and the second object; determine a valid entry status of the second object in a container based on whether the degree of similarity satisfies a threshold criteria; and output the valid entry status to an output device.
[0010] In some other implementations, an electronic handheld device includes: a display; an electronic scanner; a housing comprising: a display portion holding the display; a scanner portion holding the electronic scanner; an elongate handle portion extending from a first end to a second end, wherein the first end is connected to the display portion, and the second end is opposite the first end; a device electrical connector integrated with the elongate handle portion, the electrical connector comprising at least one conductive contact exposed from within the housing, wherein the electrical connector is removably coupled to the housing via at least one coupling member.
[0011] Other implementations are possible, and each of the embodiments can be used alone or together in combination. Accordingly, various embodiments will now be described with reference to the attached drawings.BRIEF DESCRIPTION OF THE DRAWINGS
[0012] FIG. 1 shows a block diagram of an electronic system.
[0013] FIG. 2A shows a perspective view of an electronic handheld device.
[0014] FIG. 2B shows another perspective view of the electronic handheld device of FIG. 2A.
[0015] FIG. 2C shows a third perspective view of the electronic handheld device of FIGs. 2A and 2B.
[0016] FIG. 2D shows a fourth perspective view of the electronic handheld device of FIGs. 2A-2C, with a removable cover detached from a handle.
[0017] FIG. 2E shows a fifth perspective view of the electronic handheld device of FIGs. 2A-2D, with a device electrical connector detached from a remainder of the electronic handheld device.
[0018] FIG. 2F shows a block diagram of electronic components of the electronic handheld device of FIGs. 2A-2E.
[0019] FIG. 3A is a perspective view of an example device storage.
[0020] FIG. 3B shows a cross-sectional side view of the device storage of FIG. 3A.
[0021] FIG. 3C shows another cross-sectional side view of the device storage in FIG. 3B, further holding an electronic handheld device.
[0022] FIG. 3D shows a top perspective view of the device storage of FIGs. 3A-3C.
[0023] FIG. 4A shows a perspective side view of a valid entry detection apparatus.
[0024] FIG. 4B shows a cross-sectional side view of the valid entry detection apparatus of FIG. 4A.
[0025] FIG. 4C shows another cross-sectional side view of the valid entry detection apparatus of FIGs. 4A, 4B, further holding an electronic handheld device.
[0026] FIG. 4D shows a top perspective view of the valid entry detection apparatus of FIGs. 4A-4C, illustrating a removably coupled holder electrical connector disposed on and securely attached to an inner surface of a handheld device holder portion via coupling members.
[0027] FIG. 4E shows a top perspective view of the valid entry detection apparatus of FIGs. 4A-4D, illustrating the removably coupled holder electrical connector detached from the inner surface of the handheld device holder portion via the coupling members being disengaged from the inner surface.
[0028] FIG. 5 shows a camera of the valid entry detection apparatus of FIGs. 4A-4E establishing an image capture region.
[0029] FIG. 6A shows a lower boundary of the image capture region of FIG. 5 relative to an opening of a container.
[0030] FIG. 6B shows a flow chart of an example method of configuring a container for valid entry detection.
[0031] FIG. 7 shows an example configuration of a database linking object identifications (IDs) with representative image data sets.
[0032] FIG. 8 shows a flow chart of an example method of valid entry detection for a container.
[0033] FIG. 9 shows a flow chart of another example method of valid entry detection for a container.
[0034] FIG. 10 shows an operational schematic diagram of an implementation of the electronic system of FIG. 1.
[0035] FIG. 11 shows a block diagram of an example computer system.DETAILED DESCRIPTION
[0036] The present description describes various embodiments of devices, apparatus, systems, and related methods for an electronic implementation of a container assistant for holding or storing items in containers, including during shopping events where the containers are shopping carts.
[0037] FIG. 1 shows an example implementation of an electronic system 100 for a container assistant. As described in further detail below, the electronic system 100 may operate to assist users locate items or goods within a certain location (e.g., by providing information to help guide or direct a user to a particular aisle, row, floor, room, section, or other particular area within a building, a facility, a warehouse, a school, a store, or a home, as non-limiting examples) , electronically scan or read items or goods to be placed in containers, detect whether items have been validly or properly placed in containers, and / or assist users of the electronic system 100 to locate lost, misplaced, or stolen electronic components of the electronic system 100, among other aspects. In particular implementations, the electronic system 100 is used in the context of shopping (e.g., retail shopping) , where the electronic system 100 operates as an electronic shopper assistant to a retailer or other seller of items and / or to shoppers who shop at a retailer’s store to potentially purchase one or more items that the retailer is selling. In such implementations, the containers include shopping carts, trolleys, baskets, or other similar containers that a retailer provides to a shopper to use to temporarily hold items that the shopper wants to purchase. As a shopper assistant, the electronic system 100 may operate to assist shoppers to locate and purchase items or goods within a store, detect whether items or objects for purchase have been validly or properly scanned before being placed in a shopping cart, and / or assist a retailer’s employees in locating lost, misplaced, or stolen electronic components of the electronic system 100, among other aspects as described in further detail below.
[0038] As shown in FIG. 1, the electronic system 100 may include an electronic handheld device 102 that is configured to be held and used by a user (e.g., a shopper) within a predetermined area or location (e.g., within a retailer’s store where one or more items or objects can be purchased by shoppers or customers) .
[0039] FIGS. 2A-2E show perspective views of an example configuration of the electronic handheld device 102 in more detail. In general, the electronic handheld device 102 includes a display 202 and a handle 204. The display 202 is an electronic device or component that may display information that assists a user while shopping during a shopping event, such as a user’s shopping list, a list of items the electronic handheld device 102 has scanned, locations (e.g., aisle, section, or floor numbers) of items within the store, guide information that can guide users to the item locations, or information about promotions, sales, or deals, as non-limiting examples. In some implementations, the size of the display is about five inches, although other sizes are possible in various of other implementations. Also, the handle 204 is an elongate structure adapted to be grasped and held by a human hand.
[0040] In addition, the electronic handheld device 102 may include a housing 206 that generally provides the outer shape and structural support of the electronic handheld device 102. The housing 206 may include a display portion 208 and a handle portion 210. The display portion 208 of the housing 206 is integrated with and / or holds the display 202. The handle portion 210 of the housing 206 is an elongate structure that forms at least a portion of the handle 204. In addition, the handle portion 210 generally extends from a first end 212 connected or attached to the display portion 208, to a second, free end 214 opposite the first end 212.
[0041] Referring to FIG. 2B, the electronic handheld device 102 may further include a scanner 216 that is configured to perform scanning events, during which the scanner 216 scans or reads markings (e.g., labels, codes (e.g., barcodes) , tags, or other similar types of markings as non-limiting examples) on items or objects, such as items for sale or purchase in a store. Upon successfully scanning or reading a marking of an item, the electronic handheld device 102 may obtain information about the item, such as identification (ID) information (e.g., an item name, an ID number (e.g., a stock keeping unit (SKU) number) ) , as well as other information related to the item, such as a purchase price of the item, a quantity of the items currently in stock, an expiration date of the item, or sale or promotional information about the item, as non-limiting examples. Additionally, in particular implementations, the scanner 216 is a barcode scanner or reader that reads and / or scans barcode labels on products, though any of various other types of scanning / reading technology (e.g., quick-response (QR) code scanning) may be used for the scanner 216 and corresponding product labels in any of various other types of implementations.
[0042] The housing 206 of the electronic handheld device 102 may include a scanner portion 218 that holds and supports the scanner 216. For at least some implementations such as shown in FIGs. 2A, 2B, the scanner portion 218 supports the scanner 216 at an end 219 of the electronic handheld device 102 that is opposite or furthest from the second, free end 214 of the handle 204.
[0043] Referring back to FIG. 1, in some implementations, the electronic system 100 includes a device storage (also called herein a cradle or a docking station) 104 for the electronic handheld device 102. In general, the device storage 104 is a device or apparatus that is configured to hold the electronic handheld device 102 when the electronic handheld device 102 is not in use by a user (e.g., a shopper during a shopping event) . For example, before a shopper obtains the electronic handheld device 102 for use while shopping, the electronic handheld device 102 may be stored or held by the device storage 104. When the electronic handheld device 102 is to be used by a shopper, the electronic handheld device 102 may be released from the device storage 104. When the shopper is finished using the electronic handheld device 102, the electronic handheld device 102 may be placed back in the device storage 104.
[0044] FIG. 3A is a perspective view of an example device storage 104. FIG. 3B shows a cross-sectional side view of the device storage 104. FIG. 3C shows the cross-sectional side view of the device storage 104 in FIG. 3B, further holding an electronic handheld device 102. As shown in FIGs. 3A-3C, the device storage 104 includes a base 302 and at least one receptacle (or hole or port) 304 disposed and / or recessed in the base 302. Each receptacle 304 is configured to receive a handle 204 of an electronic handheld device 102. Upon receipt of the handle 204, the device storage 104 is storing and / or holding the electronic handheld device 102. The example configuration of the device storage 104 in FIGs. 3A-3D include three receptacles 304 disposed in the base 302, each configured to receive and hold a respective one of three different electronic handheld devices 102. Other configurations of the device storage 104 may include more or fewer than three receptacles disposed in the base 302. Also, in some example implementations, a store or other building or facility implementing the electronic system 100 may have a device storage system in the form of a wall or other similar structure having a plurality of device storage components 104. For example, the device storage system may include a plurality of rows, where each row includes at least one base 302 having at least one receptacle 304.
[0045] Additionally, in some implementations, the device storage 104 is configured to charge a battery (e.g., a rechargeable battery) 230 (see FIG. 2D) of an electronic handheld device 102 that the device storage 104 is holding. For example, as shown in FIG. 3B, the device storage 104 may include at least one electrical connector 308, each positioned in a respective one of the at least one receptacle 304, such as at a base or bottom or the receptacle in some implementations. Additionally, in some implementations, the electrical connector 308 may be configured in the form of a pogo pin connector, although any of various other configurations may be possible. When a handle 204 of an electronic handheld device 102 is inserted into the receptacle 304 (FIG. 3C) , the associated electrical connector 308 is configured to physically contact and electrically connect to a corresponding device electrical connector 224 integrated with the handle 204 (shown and described in further detail below with respect to FIGs. 2C-2E) . The device storage 104 may be connected to a power source (e.g., an electrical wall socket or outlet) , and using power received from the power source, may charge the battery 230 via the electrical connector 308.
[0046] Referring to FIG. 3D, in some implementations, each electrical connector 308 (or at least one of the electrical connectors 308) may be removably coupled to a respective receptacle 304, such as via at least one coupling member 312. In some implementations such as in FIG. 3D, the at least one coupling member includes one or more fasteners (e.g., screws) . Within a given receptacle 304, the at least one coupling member 312 may be engaged with the receptacle 304 to secure an attachment of an electrical connector 308 in the receptacle 304. For example, as shown in FIG. 3D, two screws 312 may be screwed into corresponding threaded bores to securely attach the electrical connector 308 to the receptacle 304. Correspondingly, manipulating the at least one coupling member 312 (e.g., by unscrewing the screws out of their corresponding threaded bores) may permit the electrical connector 308 to be uncoupled or detached from the receptacle 304. FIG. 3D shows the coupling members 312 (1) , 312 (3) (e.g., screws) threadingly engaged with their corresponding bores, securely attaching respective first and third electrical connectors 308 (1) , 308 (3) in the respective first and third receptacles 304 (1) , 304 (3) . In addition, FIG. 3D shows coupling members 312 (2) (e.g., screws) unscrewed and detached from their threaded bores, allowing a second electrical connector 308 (2) to be removed and / or detached from a second receptacle 304 (2) .
[0047] In this context, the electrical connector 308 may be a modular component of the device storage 104 that can be attached to and detached from the electronic handheld device 102. Moreover, through the removable coupling and / or modular configuration, the electrical connector 308 may be a replaceable component. For example, one electrical connector 308 may be removed or detached from the device storage 104, and may be replaced by a new or at least different electrical connector 308, which may be attached or coupled to, or installed onto, the device storage 104 in the same way that first electrical connector 308 was. Also, as used herein, a coupling member for a device storage 104 is some component or apparatus that keeps the electrical connector 308 coupled or attached to the device storage 104, and that is manipulated or has a configuration changed in order for the electrical connector 308 to be uncoupled, detached or removed from device storage 104.
[0048] The removable coupling configurations of the electrical connector 308 described herein may allow or facilitate the replacement of the electrical connector 308 without having to replace the entire device storage 104. In addition or alternatively, in the implementations described herein, the removable coupling configuration may allow end users or owners of the device storage 104 to be able to replace the electrical connector 308 themselves, such as onsite (e.g., within the retailer store by an employee of the retailer) , instead of having a manufacturer or seller of the device storage 104, or another third party, perform the replacement, such as by the end user or owner sending the device storage 104 to the manufacture, seller, or third party for serving or repair. This, in turn, may provide substantial reduction in downtime of the device storage 104 and / or provide substantial cost saving to the owner of the device storage 104, especially in situations where the owner (e.g. a retailer or store owner) owns tens, hundreds, or even thousands of such device storage components (or pieces of equipment) 104 at a single time.
[0049] Additionally, in some implementations, the device storage 104 is configured to lockingly hold the electronic handheld device 102. For example, as shown in FIG. 3B, the receptacle 304 may include a latch 310 or other similar structure configured to engage with a corresponding portion of the handle 204 of the electronic handheld device 102 that locks or secures the handle 204 in the receptacle 304. As described in further detail below, the device storage 104 may be configured to receive an authorization input signal, such as in the form of an electrical signal or mechanical input, that releases the lock, and in turn allows a user to freely pull or remove the handle 204 out of the receptacle 304 to separate the electronic handheld device 102 from the device storage 104.
[0050] Also, in some implementations, the electronic handheld device 102 and the device storage 104 may be configured to establish a connection (e.g, a network connection, a wired connection, a wireless connection, or combinations thereof) between each other. In operation, the electronic handheld device 102 may communicate (transmit and / or receive) information with the device storage 104 via the connection. For example, in some implementations, the electronic handheld device 102 may transmit a current location and / or a battery status (e.g. a percent of charge) of the battery 230. For at least some of these implementations, the electronic handheld device 102 may perform the transmission periodically, at predetermined time intervals, or in response to requests from the device storage 104. Various ways or configurations for the electronic handheld device 102 to transmit its current location and / or its battery status may be possible.
[0051] Referring back to FIG. 1, in some implementations, the electronic system 100 may include a device authorization and assignment system 106 that authorizes a user (e.g., a shopper) to use an electronic handheld device 102, assigns an electronic handheld device 102 (such as from a plurality of electronic handheld devices 102) to the user, and / or that sends an authorization signal or command to the device storage 104 that causes the device storage 104 to unlock the assigned electronic handheld device 102 from the device storage 104.
[0052] For example, in some implementations, the device authorization and assignment system 106 may include a computer device, a touchscreen, an electronic kiosk or other similar electronic device or system installed and positioned in a building (e.g., a store of a retailer) . Further details of an example configuration for the device authorization and assignment system 106 is provided below with respect to the computer system 1100 of FIG. 11.
[0053] When a user (e.g., a shopper) wants to use an electronic handheld device 102 (e.g., for shopping in the store) , before the user obtains the electronic handheld device 102, the user may approach the device authorization and assignment system 106 and provide credentials to the device authorization and assignment system 106 that indicates that the user is authorized to use an electronic handheld device 102. As another example, the user may position a card (e.g., a loyalty or membership card associated with the retailer) within a predetermined proximity of the device authorization and assignment system 106, and a card reader of the device authorization and assignment system 106 may read the card to identify that the user of the card is authorized. As another example, a user may enter a login and / or password into the device authorization and assignment system 106 to obtain authorization. As still another example, a user may use his / her mobile device and a network connection (e.g., the Internet or a local area network connection within the store) to obtain authorization. For example, the user may enter credentials into a webpage of the retailer through the user’s mobile device or may use the mobile device to scan a QR code displayed by the device authorization and assignment system 106 to gain authorization, as non-limiting examples.
[0054] Additionally, upon determining that a user is authorized to use an electronic handheld device 102, the device authorization and assignment system 106 may assign an electronic handheld device 102, such as from a plurality of available (not currently in-use) electronic handheld devices 102 to the user. In some implementations, the device authorization and assignment system 106 may link the assigned electronic handheld device 102 (or a unique ID of the assigned electronic handheld device 102 with a unique ID of the user (such as one derived from the user’s credentials) for future use by one or more electronic components of the electronic system 100. In some implementations, the device authorization and assignment system 106 may dynamically maintain and / or manage a list of available (i.e., not currently in-use) electronic handheld devices 102 in the building or store. Such a list may be electronically stored and accessed from a memory located internally within the device authorization and assignment system 106, from memory located within the building or store (such as a local server for example) , from remote storage (e.g., the cloud or other storage that is remote from the building or store) . Upon determining that the user is authenticated, the device authorization and assignment system 106 may access the list, identify an available electronic handheld device 102 from the list, and assign the available electronic handheld device 102 to the user. Further, the device authorization and assignment system 106 may update the list to identify that the electronic handheld device 102 assigned to the user is no longer available.
[0055] Additionally, in some implementations, the device authorization and assignment system 106 may include circuitry configured to establish a connection (e.g., a network connection, a wired connection, a wireless connection, or combinations thereof) with circuitry of the device storage 104, and / or may establish a connection (e.g., a network connection, a wired connection, a wireless connection, or combinations thereof) with the electronic handheld device 102. Through the one or more connections, the device authorization and assignment system 106 may transmit an authorization signal to the circuitry of the device storage 104 and / or to the electronic handheld device 102 carrying information identifying the assigned electronic handheld device 102. In response to receipt of the signal, the device storage 104 and / or the electronic handheld device 102 may activate an output that indicates to the user the assigned electronic handheld device 102 being held in the device storage 104. In some implementations, the output is a light output emitted by a light source. For example, as shown in FIG. 3A, the device storage 104 includes a light source (e.g., a light emitting diode (LED) ) 306 at least partially surrounding an opening of the receptacle 304, which may be configured to light up to indicate the assigned electronic handheld device 102. In addition or alternatively, in some implementations, a light source 225 (e.g., a flashlight) of the electronic handheld device 102 (FIGs. 2C-2E) may be configured to light up to indicate the assigned electronic handheld device 102. In addition or alternatively, in some implementations, the output is an audible output that is output by a speaker. For example, the electronic handheld device 102 may include a speaker 227 configured to output a sound or audible output that indicates the assigned electronic handheld device 102. In addition or alternatively, in some implementations, the output is a visual output displayed on an electronic display. For example, the display 202 of the assigned electronic handheld device 102 may display an indication that it has been assigned.
[0056] Also, in implementations where the assigned electronic handheld device 102 is locked in the receptacle 304, the authorization signal may cause the device storage 104 to unlock or release its locked connection with the assigned electronic handheld device 102, in turn allowing the user to freely remove the assigned electronic handheld device 102 from the receptacle 304. In some implementations, the device authorization and assignment system 106 may send the authorization signal to the device storage 104 directly, which may cause the device storage 104 to unlock the assigned electronic handheld device 102. In other embodiments, the device authorization and assignment system 106 may send the authorization signal to the assigned electronic handheld device 102, which in turn may forward the authorization signal or generate and send it own authorization signal to the device storage 104 via a connection that the assigned electronic handheld device 102 has with device storage 104, such as a serial port connection or other similar connection established when the device electrical connector 224 engages with the electrical connector 308 of the device storage 104, which in turn may cause the device storage 104 to unlock the assigned electronic handheld device 102.
[0057] In addition, in some implementations, the device authorization and assignment system 106, such as via a display, may display a map or other graphical representation of receptacles 304 of the device storage 104 (or a device storage system) in a way that indicates whether or not each of the receptacles 304 is holding a respective electronic handheld device 102. In addition, in some of these implementations, the device authorization and assignment system 106 may also display a battery or charge status of the electronic handheld devices 102 being held by the device storage 104. In addition or alternatively, in some implementations, the device authorization and assignment system may display one or more battery or charge statuses of one or more electronic handheld devices 102 being held by the device storage 104 at a given point in time. In addition or alternatively, the device authorization and assignment system 106 may be configured to display the map or graphical representation in response to receipt of credentials of a user (e.g., shopper) .
[0058] Additionally, the electronic system 100 may include a valid entry detection apparatus 108 configured capture or take one or more images of an image capture region or region adjacent to a container 110 used to determine a valid entry status of an object or item in the container. In some implementations for shopping (e.g., retail shopping) in a store such as in FIG. 1, the container 110 may be a shopping cart or trolley (or more particularly a basket of a shopping cart or trolley) , such as shown in FIG. 1, or other similar apparatus provided by the store to a shopper that a shopper can temporarily use to hold items that the shopper intends to purchase or otherwise take into possession, including after the shopper leaves the store. Outside of the context of shopping, the container 110 may be used for any of various applications, such as short term or long term storage, shipping, organization, or any of various other types of applications or uses where a container is used to hold or contain one or more items.
[0059] Also, as used herein a valid entry status of an object is an indication of whether the object is acceptably or properly in (or within, contained within, or being held by) the container 110. A valid entry status of an object is valid when the object is acceptably or properly in the container 110. A valid entry status of an object is invalid when the object is unacceptably or improperly in the container 110. In particular implementations, the valid entry status of an object is valid when the object is successfully scanned by the scanner 216 before being placed in the container 110. Additionally, the valid entry status of an object is invalid when the object is not successfully scanned by the scanner 216 before being placed in the container 110.
[0060] FIG. 1 shows the valid entry detection apparatus 108 connected or attached to the shopping cart 110. FIGs. 4A-4C show perspective views of an example configuration of the valid entry detection apparatus 108. In the example configuration in FIGs. 4A-4C, the valid entry detection apparatus 108 may include a body 402 that includes a handheld device holder portion 404 and a container attachment portion 406.
[0061] As best shown in FIGs. 4B, 4C, the handheld device holder portion 404 may be configured to hold the electronic handheld device 102, such as the handle 204 of the electronic handheld device 102. In some implementations such as shown in FIGs. 4A-4E, the handheld device holder portion 404 may include a wall 408 having an inner surface 410 defining a volume 412 for receipt of the electronic handheld device 102, such as the handle 204 of the electronic handheld device 102.
[0062] Additionally, in some implementations, the inner surface 410 includes a holder electrical connector 414 (such as in the form of a pogo connector) that physically and electrically connects to a corresponding device electrical connector 224 (FIGs. 2C-2E) of the electronic handheld device 102. The electronic handheld device 102 may be configured to charge a battery 426 of the valid entry detection apparatus 108 when the handle 204 is positioned in the handheld device holder portion 404, which is described in further detail below.
[0063] Additionally, in some implementations during operation, a particular electronic handheld device 102 and a particular valid entry detection apparatus 108 may pair together in which a connection (e.g., a network connection, a wired connection, a wireless connection, or combinations thereof) in order to communicate with each other. A pairing may be established particularly in environments, such as a store, where there are several electronic handheld devices 102 and / or several valid entry detection apparatuses 108. Before a pairing is established between a given electronic handheld device 102 and a given valid entry detection apparatus 108, the handle 204 of the electronic handheld device 102 may be positioned in the handheld device holder portion 404, such as shown in FIG. 4C. At this time, the device electrical connector 224 (FIGs. 2C-2E) and the holder electrical connector 414 may become engaged, causing the electronic handheld device 102 and the valid entry detection apparatus 108 to establish a connection (e.g., a network connection, a wired connection, a wireless connection, or combinations thereof) . As a non-limiting example, the connection may be a serial port connection and / or a universal asynchronous receiver / transmitter (UART) connection. The electronic handheld device 102 and the valid entry detection apparatus 108 may also be able communicate via a wireless network connection, such as after or upon establishing the UART connection when the electronic handheld device 102 is in the handheld device holder portion 404. For example, the electronic handheld device 102 and the valid entry detection apparatus 108 may be configured to exchange Internet Protocol (IP) addresses, enabling a wireless connection. Various ways of establishing a pairing and / or one or more connections between the electronic handheld device 102 and the valid entry detection apparatus 108 so that they can communicate signals carrying data, information, and / or commands between each other, in a wired and / or wireless manner, may be possible.
[0064] Referring to FIGs. 4D-4E, in some implementations, the holder electrical connector 414 may be removably coupled to the inner surface 410 of the handheld device holder portion 404, such as via at least one coupling member 415. In some implementations, the at least one coupling member 415 includes one or more fasteners (e.g., screws) . The at least one coupling member 415 may be engaged with the handheld device holder portion 404 to secure an attachment of the holder electrical connector 414 to the inner surface 410 of the handheld device holder portion 404. For example, as shown in FIG. 4D, two screws 415 may be screwed into corresponding threaded bores to securely attach the holder electrical connector 414 to the inner surface 410 of the handheld device holder portion 404. Correspondingly, manipulating the at least one coupling member 415 (e.g., by unscrewing the screws out of their corresponding threaded bores) may permit the holder electrical connector 414 to be uncoupled or detached from the valid entry detection apparatus 108. FIG. 4D shows the coupling members 415 (e.g., screws) threadingly engaged with their corresponding bores, securely attaching the holder electrical connector 414 to the inner surface 410 of the handheld device holder portion 404. FIG. 4E shows the coupling members 415 (e.g., screws) threadingly disengaged from the corresponding bores, allowing the holder electrical connector 414 to be uncoupled or detached from the inner surface 410 of the handheld device holder portion 404.
[0065] In this context, the holder electrical connector 414 may be a modular component of the valid entry detection apparatus 108 that can be attached to and detached from the valid entry detection apparatus 108. Moreover, through the removable coupling and / or modular configuration, the holder electrical connector 414 may be a replaceable component. For example, one holder electrical connector 414 may be removed or detached from the valid entry detection apparatus 108, and may be replaced by a new or at least different holder electrical connector 414, which may be attached or coupled to, or installed onto, the inner surface 410 of the handheld device holder portion 404 in the same way that first holder electrical connector 414 was. Also, as used herein, a coupling member for the holder electrical connector 414 is a component or apparatus that keeps the holder electrical connector 414 coupled or attached to the valid entry detection apparatus 108, and that is manipulated or has a configuration changed in order for the holder electrical connector 414 to be uncoupled, detached or removed from valid entry detection apparatus 108.
[0066] The removable coupling configurations of the holder electrical connector 414 described herein may allow or facilitate the replacement of the holder electrical connector 414 without having to replace the valid entry detection apparatus 108. In addition or alternatively, in the implementations described herein, the removable coupling configuration may allow end users or owners of the valid entry detection apparatus 108 to be able to replace the holder electrical connector 414 themselves, such as onsite (e.g., within the retailer store by an employee of the retailer) , instead of having a manufacturer or seller of the valid entry detection apparatus 108, or another third party, perform the replacement, such as by the end user or owner sending the valid entry detection apparatus 108 to the manufacturer, seller, or third party for serving or repair. This, in turn, may provide substantial reduction in downtime of the valid entry detection apparatus 108 and / or provide substantial cost saving to the owner of the valid entry detection apparatus 108, especially in situations where the owner (e.g. a retailer or store owner) owns tens, hundreds, or even thousands of such valid entry detection apparatuses 108 at a single time.
[0067] Referring to FIGs. 4B and 4D, in some implementations, a through hole 417 may extend through the wall 408 of the handheld device holder portion 404, from the inner surface 410 to an outer surface 419. The through hole 417 may function as a liquid (e.g., rainwater) outlet or drain. As shown in FIGs. 4B and 4D, the through hole 417 may extend through the wall 408 from a bottom or base portion of the inner surface 410, such that if water or other liquid enters into the volume 412, the water will drain out of the volume 412 via the through hole 417. This way, if liquid is undesirably delivered to within the volume 412, such as if the container 110 (e.g., shopping cart) is left outside in the rain or a user (e.g., shopper) spills a drink into the volume 412, the valid entry detection apparatus 108 will already have a feature (i.e., the through hole 417) that can drain the liquid out of the volume 412.
[0068] Additionally, the container attachment portion 406 of the body 402 is configured to attach to a corresponding attachment area of a container. In some implementations where the container is a shopping cart, the corresponding attachment area is a handlebar 112 of the shopping cart, as shown in FIG. 1.
[0069] Additionally, in some implementations such as shown in FIGs. 4A-4C, the container attachment portion 406 includes an inner surface 416 that engages with a corresponding surface of the attachment area (e.g., an outer surface of the handlebar 112) of the container 110 to establish a secure attachment to the attachment area. In particular of these implementations such as in FIGs. 4A-4C, the container attachment portion 406 includes a clamp 418 that clamps to the attachment area. For example, the inner surface 416 of the clamp 418 surrounds the handlebar 112 and the clamp 418 exerts an inward force onto the handlebar 112 that is sufficiently high so as to prevent or inhibit rotation of the valid entry detection apparatus 108 about the handlebar 112. Here, sufficiently high refers to that the clamping force prevents movement relative to the attachment area (e.g., rotational movement about the handlebar 112) caused by weight distribution or imbalance of the valid entry detection apparatus 108 relative to the attachment area and gravitational force.
[0070] The valid entry detection apparatus 108 may also include a camera 420 disposed in the body 402. The camera 420 may be configured to take images of an image capture region adjacent to an opening of the container. For example, in the context of shopping, the image capture region may be above an opening 114 of the basket of the shopping cart 110 (FIG. 1) . When the container attachment portion 406 is securely attached to the attachment area (e.g., the handlebar 112) of the container (e.g., the shopping cart 110) , the camera 420 may be configured to establish the image capture region adjacent to the container.
[0071] The combination of the valid entry detection apparatus 108 attached to the attachment area of the container and the camera 420 may be set and / or adjusted to define horizontal and vertical boundaries of the image capture region. FIG. 5 shows the valid entry detection apparatus 108 attached to a portion of the attachment area (e.g., handlebar) 112, with the camera 420 establishing an image capture area or region 500 of which the camera 420 captures images. As shown in FIG. 5, from the perspective of the camera 420, the image capture region 500 includes an upper boundary 502, a lower boundary 504, a left boundary 506, and a right boundary 508. For implementations where the valid entry detection apparatus 108 is attached to a handlebar 112 of a shopping cart 110, when the container attachment portion 406 is attached to the handlebar 112 but not so securely such that the valid entry detection apparatus 108 can be moved or rotated about an axis 510 of the handlebar 112, such rotation can correspondingly vertically adjust the upper and lower boundaries 502, 504 of the image capture region 500. In addition or alternatively, in some implementations, the camera 420 has the ability to set and / or adjust its horizontal angle of view. Accordingly, through its ability to rotationally adjust its position about the handlebar 112, in combination its ability to change its horizontal angle of view of the camera 420, the valid entry detection apparatus 108 may be able to set and / or adjust the image capture region 500 both vertically and horizontally in order to set an optimum size and / or position of the image capture region 500 relative to the container 110.
[0072] Referring to FIG. 6A, in some implementations, the valid entry detection apparatus 108 may be installed on an attachment area of a container (e.g., the handlebar 112 of the shopping cart 110) such that when the valid entry detection apparatus 108 is securely attached to the attachment area (e.g., handlebar 112) , the lower boundary 504 of the image capture region 500 is substantially parallel with the opening 114 of the container 110. Here, substantially parallel refers to an angle θ (also called herein a mounting angle) defined by the lower boundary 504 and the opening 114 being less than 45 degrees. Additionally, in some implementations, the mounting angle θ may be set and / or adjusted depending on or according to the configuration and / or model of the container 110. For example, the mounting angle θ may be set and / or adjusted based on the model of the shopping cart 110 to which the valid entry detection apparatus 108 is attached. Accordingly, in any of various implementations, the mounting angle θ for different containers 110 may be the same as or different from each other, which may depend on the configurations, types, and / or models of the containers 110.
[0073] FIG. 6B shows a flow chart of an example method 600 of configuring a container for valid entry status detection. At block 602, a valid entry detection apparatus (e.g., the valid entry detection apparatus 108) may be attached to an attachment area of a container (e.g., the handlebar 112 of the shopping cart 110) . In some implementations of the method 600, the valid entry detection apparatus is attached to the container by clamping the valid entry detection apparatus to the container (e.g., clamping the valid entry detection apparatus 108 to the handlebar 112) . At block 604, an image capture region (e.g., the image capture region 500) may be established adjacent to an opening of the container (e.g., above the opening 114 of the basket of the shopping cart 110) . In some implementations, the image capture region is established by adjusting a rotatable position of the valid entry status about the attachment area until a lower boundary of the image capture region is substantially parallel with the opening of the container. For example, valid entry detection apparatus 108, when unsecurely clamped to the handlebar 112, may be rotated about the handlebar 112 until a lower boundary 504 of the image capture region 500 is substantially parallel with the opening 114 of the shopping cart 110. When the valid entry detection apparatus 108 is in a desired position (e.g., the lower boundary is substantially parallel with the opening 114) about the handlebar 112, the clamp 418 may be tightened to establish a secure attachment to the handlebar 112. In addition or alternatively, in some implementations, a horizontal angle of view of the camera 420 may be adjusted to establish the image capture region.
[0074] Referring back to FIGs. 4B, 4C, the valid entry detection apparatus 108 may further include a sensor 422 integrated with and / or disposed in the body 402. The sensor 422 is configured to sense a presence of objects or items in the image capture region 500. For implementations where the image capture region 500 is adjacent to the opening 114 of the container 110, the sensor 422 may be configured to sense the presence of an object as it is moving (such as falling) toward the opening 114 and into the container 110. In some implementations, the sensor 422 may be in the form of a radar device or motion sensor, although other types of configurations for the sensor 422 suitable for detection of presence of an item or object, including but not limited to where the item or object is moving in the image capture region 500 toward the opening 114.
[0075] Additionally, the camera 420 and the sensor 422 may be electrically and / or operatively connected or coupled to each other to enable the camera 420 to be controlled based on operation of the sensor 422. For example, the sensor 422 detecting the presence of an object in the image capture region 500 may cause or trigger the camera 420 to capture or take one or more images of the image capture region 500 with the object. In some implementations, the camera 420 and the sensor 422 are directly electrically and / or operatively coupled to each other. In other implementations, such as shown in FIGs. 4B and 4C, the camera 420 and the sensor 422 are electrically and / or operatively coupled to each other via circuitry 424 disposed within the body 402 of the valid entry detection apparatus 108. The circuitry 424 may include various circuit or other electronic components. For example, the circuitry 424 may include a circuit board on which one or more circuit components (e.g., one or more chips, passive components, active components, etc. ) may be mounted. In addition or alternatively, the circuitry 424 may include a controller, such as in the form of a processor configured to execute software and / or other hardware circuitry. The circuitry 424 may be configured to control one or more of various electronic operations and / or control the electronic components (e.g., the camera 420 and / or the sensor 422) of the valid entry detection apparatus 108. For example, when the sensor 422 detects the presence of an object, the sensor 422 may transmit a signal of the detection to the controller 424. The controller 424 may detect the signal, and in response, trigger the camera 420 to capture one or more images of the image capture region 500 including the object. In some implementations, the camera 420 may be configured to take three or four images in response to the sensor 422 detecting the presence of an object in the image capture region 500, although numbers of images fewer or greater than three or four may be possible in any of various other implementations.
[0076] In addition, the circuitry 424 may include network circuitry configured to connect to and / or establish a wired and / or wireless network connection, such as a local area network (LAN) , a wireless local area network (WLAN) , a personal area network (PAN) , a Bluetooth network, a wide area network (WAN) , a serial port connection, and / or a UART connection as non-limiting examples. The network connection may be established with one or more of the electronic components of the electronic system 100, such as the electronic handheld device 102, the device storage 104, the device authorization and assignment system 106, a scale 115 (discussed below) and / or a valid entry detection controller 116 (discussed below) . In addition or alternatively, the circuitry 424 is configured to facilitate a pairing between the valid entry detection apparatus 108 and an electronic handheld device 102, such as one that the valid entry detection apparatus 108 is holding, as previously described. In addition or alternatively, the circuitry 424, such as via a network connection, may be configured to detect a scanning event performed by the scanner 216 of the electronic handheld device 102. In at least some implementations, the electronic handheld device 102 may communicate the scanning event to the valid entry detection apparatus 108 via a wireless connection. In some of these situations, the electronic handheld device 102 is positioned outside of the handheld device holder portion 404, such as when it being held by a user (e.g., shopper) , at which time it is not physically attached to the valid entry detection apparatus 108. Despite not being physically attached, the electronic handheld device 102 and the valid entry detection apparatus 108 may still maintain a network connection with each other, enabling communication (e.g., such as of a scanning event) between each other. Also, the circuitry 424 of the valid entry detection apparatus 108 may detect a scanning event in situations where the electronic handheld device 102 is being held by the handheld device holder portion 404. Additionally, in response to the detection of the scanning event, the circuitry 424 may trigger the camera 420 to capture one or more images. The circuitry 424 may include other circuitry or circuit components to facilitate communication, such as an antenna (or antenna array) and / or transceiver circuitry, as non-limiting examples.
[0077] In addition, as previously mentioned, the valid entry detection apparatus 108 may include a battery (also called a holder battery) 426 disposed within the body 402 of the valid entry detection apparatus 108. The battery 426 may be configured to power the electronic components of the valid entry detection apparatus 108, such as the camera 420, the sensor 422, and the active circuit components (e.g., controller, processor, memory, etc. ) of the circuitry 424. In some implementations such as shown in FIGs. 4B and 4C, the battery 426 may be configured to power the camera 420 and / or the 422 via the circuitry 424, although other implementations where the battery 426 directly sends power to the camera 420 and / or to the sensor 422 may be possible.
[0078] Also, in some implementations as mentioned, the battery 426 may be configured to be charged by a battery 230 (FIG. 2D) of the electronic handheld device 102. As shown in FIG. 4C, when the handle 204 of the electronic handheld device 102 is in the volume 412 of the handheld device holder portion 404, the device electrical connector 224 (FIGs. 2C-2E) physically contacts and electrically connects to the holder electrical connector 414 (FIG. 4B) . Upon engagement between the device and holder electrical connectors 224, 414, an electrical circuit may form between the battery 230 of the electronic handheld device 102 and the battery 426 of the valid entry detection apparatus 108, allowing the battery 230 to charge the battery 426. As described in further detail below, a controller 250 (FIG. 2F) of the electronic handheld device 102 and / or the controller circuitry 424 may control charging, e.g., when and / or if the battery 230 charges the battery 426 when the handle 204 is in the handheld device holder portion 404.
[0079] Additionally, in some implementations as shown in FIG. 4A, the valid entry detection apparatus 108 may include a light source 428 configured to output or emit a light output. As described in further detail below, the light source 428 may be an output device of the electronic system 100. In addition or alternatively, the light output that the light source 428 outputs may have one or more properties or characteristics, such as a color and / or a flashing pattern, that conveys or indicates any of various information at different points in time, such as a valid entry status for example. Additionally, in some implementations such as in FIG. 4A, the light source 428 is integrated with the handheld device holder portion 404. However, the position of the light source 428 shown in FIG. 4A is just an example, and the light source 428 may be positioned in any of various other locations on the body 402 in any of various other implementations.
[0080] Additionally, in some implementations as shown in FIGs. 4B, 4C, the valid entry detection apparatus 108 may include at least one speaker 430 configured to output sound or other audible output. As described in further detail below, the speaker 430 may be an output device of the electronic system 100. In addition or alternatively, the audible output that the speaker 430 outputs may have a certain sound and / or carry audible words, phrases, or terms, conveying any of various information at different points in time, such as a valid entry status for example. In addition or alternatively, the speaker 430 may include or otherwise function as a buzzer. Additionally, in some implementations such as in FIGs. 4B, 4C, the at least one speaker 430 includes two speakers, one positioned above the camera 420 and another positioned below the camera 420. However, use of two speakers 430 and their positions shown in FIGs. 4B, 4C is just an example, and the number of speakers 430 may be only one or more than two, and / or may be positioned in any of various other locations on the body 402 in any of various other implementations. Various ways of integrating one or more speakers with the valid entry detection apparatus 108 are possible.
[0081] Referring back to FIG. 1, the electronic system 100 may further include a valid entry detection controller 116 configured to determine a valid entry status of an object in the container (e.g., shopping cart) 110. In particular implementations, the valid entry detection controller 116 may determine the valid entry status of an object that has been detected in the image capture region 500 by the sensor 422 and has had one or more images of it taken by the camera 420.
[0082] The valid entry detection controller 116 may be implemented in, or otherwise operatively coupled to, the electronic system 100 in any of various ways. For example, the valid entry detection controller 116 may be a component (or sub-component) of one of the other electronic components in the electronic system 100, such as the electronic handheld device 102, the device storage 104, the device authorization and assignment system 106, and / or the valid entry detection apparatus 108. In other implementations, the valid entry detection controller 116 is a standalone electronic component or at least a component that is separate from the other electronic components of the electronic system 100. For example, the valid entry detection controller 116 may be component of, or implemented in, a local server or other computing device within a building (e.g., a store) in which the other components of the electronic system 100 are being used, or may be configured remote or outside of the building or store. Various ways of configuring the valid entry detection controller 116 in the electronic system 100 may be possible. Also, for implementations where the valid entry detection controller 116 is separate from another electronic component of the electronic system 100, the valid entry detection controller 116 may be configured to establish a network connection (including a wired and / or wireless network connection) with one or more of the other electronic components in order to communicate with those other electronic components, such as by communicating (transmitting and / or receiving) signals carrying any of various data, information, and / or commands between the valid entry detection controller 116 and the other electronic components. In addition or alternatively, the valid entry detection controller 116 may be implemented in a computer system having some or all of the components of the computer system 1100 described in further detail below with respect to FIG. 11.
[0083] Additionally, in some implementations, in order for the valid entry detection controller 116 to determine valid entry statuses of objects or items in the container 110, the valid entry detection controller 116 may leverage a database 118 of one or more representative image data sets associated with representative images of representative objects or items. The representative objects in the representative images are representative of objects or items that could or have the potential to be placed in the container 110. For example, in the context of shopping, representative objects or items are representative of items that a retailer has for sale at given point in time. In some implementations, such items may include only those items available for purchase within a particular store in which a shopper is shopping. In other implementations, such items may include items a retailer has available for purchase, irrespective of a particular store, such as across multiple stores and / or available for purchase online (e.g., through the retailer’s or a third party’s website) .
[0084] The database 118 of representative image data sets may be stored in a memory 120 located or implemented in any of various locations or combinations of locations within and / or external to the electronic system 100. For example, in some implementations, the memory 120 may be a component (or sub-component) of one or more of the other electronic components of the electronic system 100, such as a component of the electronic handheld device 102, the device storage 104, the device authorization and assignment system 106, the valid entry detection apparatus 108, or a component of or part of the same computing device as the valid entry detection controller 116. In other implementations, the memory 120 may be a standalone component in the electronic system 100 or otherwise a component that is separate from the other electronic components of the electronic system 100, such as a component that is separate from the electronic handheld device 102, the device storage 104, the device authorization and assignment system 106, the valid entry detection apparatus 108, and / or the valid entry detection controller 116. In still other implementations, the memory 120 may configured in a remote location from the building or store in which the electronic system 100 is implemented. For example, the memory 120 may be part of a cloud-based storage.
[0085] Additionally, the database 118 in the memory 120 may be organized or arranged in a way that associates or links representative image data sets with object identifications (IDs) of objects. In at least some implementations, the object IDs of the objects form at least part of information that can be obtained or derived by scanning the objects, such as with the scanner 216 of the electronic handheld device 102 or with scanner using the same or similar type of technology as the scanner 216. For example, the IDs may include SKU numbers or other similar information that can be obtained from barcode or other similar types of scanning or reading.
[0086] FIG. 7 shows a schematic diagram of an example configuration of the database 118 associating object IDs and representative image data sets. In the example configuration, the database 700 may include one or more entries 702. For example, FIG. 7 shows the database 118 including a n-number of entries, including a first entry 702 (1) , a second entry 702 (2) , and extending to an nth entry 702 (n) . At any given point in time, the database 118 may include zero entries or more or more entries 702. Each entry 702 may correspond to a representative object, and include a respective object ID of the corresponding representative object and a respective representative image data set. Within a given entry 702, the object ID and the representative image data set are linked or associated with each other and associated with the same representative object.
[0087] Additionally, different objects associated with the same representative object are considered to be interchangeable or otherwise have the same properties, qualities and / or characteristics. For example, different objects associated with the same representative object may be of the same product type, have the same size and weight, have the same labeling or packaging, made by the same manufacturer, sold under the same brand, and / or have the same purchase price at a given point in time within a store, as non-limiting examples. In addition or alternatively, an object associated with the same representative object may have the same object ID as the object ID of the representative object, and correspondingly, different objects associated with the same representative object have the same object ID as each other and as the representative object. Further, different entries 702 in the database 700 may correspond to different representative objects and in turn, correspond to different object IDs.
[0088] Additionally, in general, a representative object is generally an object captured in one more representative images with which the representative object is associated. In any of various implementations, the representative object may or may not be one of the objects that can potentially be placed in a container 110. For example, the representative object may or may not be part of a retailer’s inventory or available for purchase in any of various implementations.
[0089] In addition, a given representative image data set may include one or more representative images and / or an associated feature ID. A representative image is an image that includes a representative object. Additionally, a representative image includes any of various data, such as pixel data, that a computing device may use to reproduce the representative object on a display. Also, a feature ID is a unique ID for an object that is derived based on one or more features of the object in a set of one or more images. Example features may include shape, size, and color, as non-limiting examples. Correspondingly, a feature ID may be derived based on an associated shape value, an associated size value, and / or an associated color value determined for the object in the set of one or more images. For a representative image data set, an associated feature ID may be derived from one or more representative images associated with the representative image data set. Correspondingly, the representative image data set of a given entry 702 in the database 118 stored in the memory 120 may include only the one or more representative images, only the feature ID, or a combination of the one or more representative images and the feature ID.
[0090] Referring back to FIG. 1, the electronic system 100 may include a database controller 122 configured to build, populate, modify, and / or manage the database 118. Similar to the valid entry detection controller 116, the database controller 122 may be implemented in, or otherwise operatively coupled to, the electronic system 100 in any of various ways. For example, the database controller 122 may be a component (or sub-component) of one the other electronic components in the electronic system 100, such as the electronic handheld device 102, the device storage 104, the device authorization and assignment system 106, the valid entry detection apparatus 108, the valid entry detection controller 116, and / or the memory 120. As a non-limiting example, the valid entry detection controller 116 and the database controller 122 may be implemented in the same computing device or system and / or part of the same processing unit configured to read and execute software computer code in order to perform the functions of the valid entry detection controller 116 and the database controller 122. In addition or alternatively, the database controller 122 may be implemented in the same computing device or system as the memory 120. In other implementations, the valid entry detection controller 116 is a standalone electronic component or at least a component that is separate from the other electronic components of the electronic system 100. For example, the database controller 122 may be component of, or implemented in, a local server or other computing device within a building (e.g., a store) in which the other components of the electronic system 100 are being used, or may be configured remote or outside of the building or store. Various ways of configuring the database controller 122 in the electronic system 100 may be possible. Also, for implementations where the database controller 122 is separate from another electronic component of the electronic system 100, the database controller 122 may be configured to establish a network connection (including a wired and / or wireless network connection) with one or more of the other electronic components in order to communicate with those other electronic components, such as by communicating (transmitting and / or receiving) signals carrying any of various data, information, and / or commands between the database controller 122 and the other electronic components. In addition or alternatively, the database controller 122 may be implemented in a computer system having some or all of the components of the computer system 1100 described in further detail below with respect to FIG. 11. Irrespective of the configuration, the database controller 122 may be operatively coupled to the memory 120 in order to read from and write to the memory 120 so that the database controller 122 can access, populate, modify, and / or manage the database 118 in the memory 120.
[0091] Additionally, in some implementations, the database controller 122 may populate or build the database 118 in conjunction with, or with the assistance of, scanning and imaging performed on the representative object. For example, in some implementations, a scanning device may scan or read a label or other marking of a representative object to obtain an object ID of the representative object, the scanning device may then supply the object ID to the database controller 122. If the object ID is not already in the database 118, the database controller 122 may create a new entry (e.g., a new entry 702) in the database 118 corresponding to the object ID. In other implementations, an object ID of a representative object may be received by the database controller 122 without scanning performed by a scanning device. As non-limiting examples, the object ID may be provided to the database controller 122 via an input on a keyboard, a touchscreen input on touchscreen, a mouse input on a computer mouse, or an audible input into a microphone, of a computer system in which the database controller 122 is implemented or operatively coupled.
[0092] Similarly, in some implementations, a camera may capture or take the one or more representative images of the representative object, and then supply the one or more representative images to the database controller 122. In some of these implementations, the one or more representative images, such as captured by a camera, may include a plurality, or a set of multiple, representative images, that capture at least two different features, views, angles, and / or perspectives of the representative object. For example, one or more of the representative images may capture the representative object from one or more of its sides, and / or one or more of the representative images may capture the representative object from one or more of its corners. In particular of these implementations, the plurality or set of representative images includes ten images, including six representative images capturing one or more sides of the representative object and four representative images capturing one or more corners of the representative object. Various other implementations may use a number of representative images more or less than ten to be included or otherwise used for a given representative image data set.
[0093] Additionally, in some of these implementations, the object ID may be included in the signal carrying the representative images so that the database controller 122 is able to determine the object ID to which the representative images are to be linked in the database 118. Other ways may be possible. For example, upon receipt of an object ID, the database controller 122 may infer that images received within a certain or predetermined time period of receipt belong to, or are to be linked with, the object ID. Various ways of ensuring that representative images provided to the database controller 122 are linked to the correct object ID in the database 118 may be possible. In some other implementations, one or more representative images of a representative object may be supplied to the database controller 122 without necessarily use of a camera to take the representative images of the representative object that is scanned. For example, the representative images may be of a first representative object, different than a second representative object that is scanned, where the representative images of the first representative object are taken a certain time period (such as days, weeks, or years) before the second representative object is scanned. In such an example, the representative images of the first representative object may be stored or archived in memory, which the database controller 122 may access and retrieve upon receipt of the object ID. This way, additional images of a representative object that is scanned do not need to be taken if an appropriate set of images for linking with the object ID of the representative object that is scanned is already available. In other implementations, the representative images may be generated using graphic and / or animation software tools, without imaging of physical representations of the objects.
[0094] Additionally, in some implementations where scanning and / or imaging of the representative object is performed, the electronic handheld device 102 may be used to perform the scanning and / or to capture the one or more representative images. For example, the scanner 216 may scan or read the representative object (or a label, such as a barcode label) of the representative object to obtain an object ID of the representative object. In turn, the electronic handheld device 102 may transmit the object ID to the database controller 122 via a connection (e.g., a network connection and / or a wired or wireless connection) the electronic handheld device 102 has with the database controller 122.
[0095] To further illustrate referring back to FIGs. 2A-2C, for at least some implementations, the electronic handheld device 102 may include at least one camera configured to capture images. In the configuration in FIGs. 2A-2F, the electronic handheld device 102 includes two cameras, including a front camera 220 and a back or rear camera 222. In general, the display 202 of the electronic handheld device 102 faces in a display direction. The front camera 220 faces in the display direction to capture images, while the rear camera 222 faces in a direction opposite the display direction to capture images. In this context, the terms “front” and “rear” are used relative to the display direction that the display 104 faces. In such implementations, the front camera 220 and / or the rear camera 222 may be used to capture one or more representative images of the representative object. Then, the electronic handheld device may transmit the one or more representative images to the database controller 122, such as via a connection (e.g., a network connection and / or a wired or wireless connection) that the electronic handheld device 102 has with the database controller 122. In turn, upon receipt of the one or more representative images, the database controller 122 may populate a corresponding entry in the database 118 with the one or more representative images, such as by linking an object ID associated with the corresponding entry with the one or more representative images. In this way, a link or association is established between scanning (e.g., barcode scanning) of an object and representative images of the object.
[0096] As described, each entry 702 of the database 118 may include a representative image data set that includes at least one representative image and / or a feature ID derived from the at least one representative image. For at least some implementations where the entries 702 include feature IDs, the database controller 122 may be configured to generate the feature IDs based on the representative images. For example, when one or more representative images of a representative object are made accessible to the database controller 122 (e.g., such as by being uploaded to a certain area in the memory 120) , the database controller 122 may analyze the one or more representative images to generate a feature ID for the presentative object. For example, the database controller 122 may analyze the one or more representative images to determine a shape value of a determined shape of the representative object, a size value of a determined size of the representative object, and / or a color value of a determined color of the representative object. In turn, the database controller 122 may generate a feature ID based on the shape value, the size value and / or the color value that is unique to the representative object. The database controller 122 may include the feature ID as at least part of the representative image data set in an entry associated with the representative object. In other implementations, the feature ID may be predetermined or already known, in which case the database controller 122 does not separately generate the feature ID based on the one or more representative images.
[0097] As mentioned, the valid entry detection controller 116 may use the database 118 to determine valid entry statuses of objects. When the sensor 422 senses the presence of an object in the image capture region 500, the camera 420 may be triggered to take one or more captured images of the image capture region 500 including the object, as previously described. The valid entry detection apparatus 108 may then transmit the one or more captured images to the valid entry detection controller 116. In response to receipt of the one or more captured images, the valid entry detection controller 116 may then compare the one or more captured images with a representative image data set from the database 118 in order to determine a degree of similarity between the one or more captured images and the one or more representative images.
[0098] Upon determining a degree of similarity between the one or more captured images and the one or more representative images, the valid entry detection controller 116 may determine whether the object in the captured images sufficiently matches the object in the one or more representative images. The valid entry detection controller 116 may do so by determining whether the degree of similarity satisfies a threshold criteria. In some implementations, the threshold criteria is meeting or exceeding a similarity threshold level. That is, if the degree of similarity meets or exceeds the similarity threshold, then the valid entry detection controller 116 determines that the object in the captured images sufficiently matches the object in the representative images. On the other hand, if the degree of similarity does not meet or exceed (i.e., is below) the similarity threshold, then the valid entry detection controller 116 determines that the object in the captured images does not sufficiently match the object in the representative images. In particular implementations, the similarity threshold level is 80%, although other threshold levels or values may be possible.
[0099] Additionally, if the valid entry detection controller 116 determines that the determined degree of similarity satisfies the threshold criteria-e.g., the determined degree of similarity is greater than or equal to the similarity threshold level-then the database controller 122 may determine that the valid entry status of the object is valid-i.e., the object is acceptably in the container 110. On the other hand, if the valid entry detection controller 116 determines that the determined degree of similarity does not satisfy the threshold criteria-e.g., the determined degree of similarity is less than the similarity threshold level-then the database controller 122 may determine that the valid entry status of the object is invalid-i.e., the object is unacceptably in the container 110.
[0100] As mentioned, the database 118 may include a plurality of entries corresponding to a plurality of different object IDs and associated representative images. The database controller 122 may use or leverage a scanning event performed by the scanner 216 of the electronic handheld device 102 in order to determine which set of representative images corresponding to which object ID to use to perform the image comparison. For example, in some situations, before a user (e.g., a shopper) places an object (e.g., an item for sale) in a container (e.g., the shopping cart 110) , the user may first perform a scanning event on the object with the electronic handheld device 102, in which the user scans the object with the scanner 216 of the electronic handheld device 102. If the scanning event is successful, then the electronic handheld device 102 obtains an object ID of the object from the scanning. In turn, the electronic handheld device 102 may transmit the object ID to the valid entry detection controller 116. Then, in response to receipt of the object ID, the valid entry detection controller 116 may access the database 118 with the object ID, and locate or identify the entry corresponding to the object ID. The valid entry detection controller 116 may then use a set of one or more representative images of or associated with that entry for the image comparison.
[0101] In some situations, the object that the scanner 216 scans is the same object that the sensor 422 senses and of which the camera 420 takes one or more pictures in the image capture region 500. In such situations, the valid entry detection controller 116 may determine that the degree of similarity between the images taken by the camera 420 and the representative images retrieved from the database 118 satisfies the threshold criteria, and in turn identifies the valid entry status of the object as valid. In other situations, the object that the scanner 216 scans may be different than the object a user places or drops into the container 110-i.e., different than the object that the sensor 422 senses and of which the camera 420 takes one or more pictures. In such situations, the valid entry detection controller 116 may determine that the degree of similarity between the images taken by the camera 420 and the representative images retrieved by the database 118 does not satisfy the threshold criteria, and in turn identifies the valid entry status of the object as invalid. In still other situations, the scanner 216 may not scan any object before the user places or drops an object in the container 110. In such situations, the valid entry detection controller 116 may determine that no object was scanned before an object was sensed by the sensor 422 and / or pictures of the object were taken by the camera 420, and in turn identify the valid entry status of the object as invalid.
[0102] Additionally, in some implementations, the valid entry detection controller 116 may transmit the determined valid entry status of the object to an output device of the electronic system 100, which in turn may generate an output that indicates the determined valid entry status. In general, an output device may be a component or a combination of components capable or configured to generate at least one output that can be perceived by a human (e.g., a shopper or a store employee) through any of various senses. For example, the output can be a visual output or light output, an audible output (e.g., sound) , and / or a tactile output, as non-limiting examples. As examples, an output device of the electronic system 100 may include the display 202, the light source 225 and / or the speaker 227 of the electronic handheld device 102. In addition or alternatively, an output device of the electronic system 100 may include the light source 306 of the device storage 104. In addition or alternatively, an output device of the electronic system 100 may include the light source (e.g., a LED) 428 (FIG. 4A) and / or the speaker (s) 430 (FIGs. 4B, 4C) of the valid entry detection apparatus 108. Correspondingly, if the valid entry status is valid, the display 202 may display a message such as “Scan Successful” or “Valid” , as non-limiting examples. If the valid entry status is invalid, the display 104 may display a message such as “Scan Unsuccessful” or “Invalid” , as non-limiting examples. In addition or alternatively, one or more of the light sources 225, 306, 428 of the electronic handheld device 102, the device storage 104, and the valid entry detection apparatus 108, respectively, may generate a light output that indicates the valid entry status (e.g., a green light when the status is valid and a red light when the status is invalid, as non-limiting examples) . In addition or alternatively, one or more of the speakers 227, 430 of the electronic handheld device 102 and the valid entry detection apparatus 108, respectively, may generate an audible output or sound to indicate the valid entry status. For example, if the valid entry status is invalid, one or more of the speakers 227, 430 may output audible instructions to scan the item, remove the item from the container, place the item back on the shelf, or some other warning message indicating that the item has been improperly placed in the container. Various ways to use an output device of the electronic system 100 to generate an output (e.g., audible, visual, and / or tactile output) that enables a user (e.g., a shopper or store employee) to know of the valid entry status determination may be possible.
[0103] Accordingly, the valid entry detection controller 116 uses scanning events performed by the scanner 216 of the electronic handheld device 102 to both determine which set of representative image data sets to use or access from the database 118 for a comparison, and determine that the object is validly or acceptably in the container 110 only if or when the user (e.g., shopper) successfully scans the object with the electronic handheld device 102 before putting the object in the container 110. Such an implementation may help protect against fraud, theft, loss of goods, or other improper placement of objects in containers by identifying situations where the object placed in the container 110 is not the object that is scanned and / or where no scanning is performed before the object is placed in the container 110.
[0104] Referring back to FIG. 1, in some implementations, the container (e.g., shopping cart) 110 may include a scale (e.g., an electronic scale) 115 disposed at a base or bottom of the container, such as the bottom of the basket of the shopping cart such as shown in FIG. 1. The scale 115 may be configured to measure a weight and / or changes in weight of one or more objects 117 in the container 110. In addition, the scale 115 may include internal circuitry configured to establish a connection (e.g., a network connection and / or a wired or wireless connection, such as but not limited to a Bluetooth connection) with the electronic handheld device 102, the valid entry detection apparatus 108, and / or the valid entry detection controller 116 in any of various implementations. If the scale 115 detects or measures an increase in weight, indicating that an additional or another object is placed in the container 110, the scale 115 may transmit an indication of the weight change (e.g., weight increase) to the valid entry detection controller 116, either directly or via the electronic handheld device 102 and / or the valid entry detection apparatus 108. Upon receipt of the indication of the weight change, the valid entry detection controller 116 may determine if an additional object was placed in the container 110 improperly. For example, the valid entry detection controller 116 may determine that an additional object is placed in the container 110 in response to receipt of the indication of the weight change. In addition, the valid entry detection controller 116 may determine whether a scanning event occurred within a predetermined time period of receipt of the weight change indication. If so, then the valid entry detection controller 116 may determine that the additional object was properly placed in the container 110 (i.e., the valid entry status of the additional object is valid) . If not, then the valid entry detection controller 116 may determine that the additional object was improperly placed in the container 110 (i.e., the valid entry status of the additional object is invalid) . The valid entry detection controller 116 may the transmit the determined valid entry status to the electronic handheld device 102 for display on the display 202. Referring back to FIG. 6B, in some implementations of the method 600 configuring a container for valid entry status detection, the method 600 may include positioning a scale (e.g., the scale 115) at a base or bottom of the container (e.g., at the bottom of a basket of a shopping cart 110) .
[0105] FIG. 8 is a flow chart of an example method 800 of valid entry status determination. At block 802, a sensor (e.g., the sensor 422 of the valid entry detection apparatus 108) may sense a presence of an object in an image capture region adjacent to an opening (e.g., opening 114) of a container (e.g., shopping cart 110) . As described, in some implementations, the image capture region is above the opening of the container. In addition or alternatively, in some implementations, the sensor sense the presence of the object as the object is moving (e.g., falling) toward the opening of the container. For example, in some situations, a user (e.g., shopper) may hold the object in or above the image capture region and then release the object, such that the object falls toward the container, passing through the image capture region in the process. Such positioning and / or movement of the object in the image capture region may trigger the sensor to sense the presence of the object.
[0106] At block 804, a camera (e.g., the camera 420 of the valid entry detection apparatus 108) may capture or take at least one image of the object in the image capture region in response to the sensor sensing the presence of the object in the image capture region. In some implementations, the camera takes a plurality of images, such as four or five images in response to being triggered by the sensor, although numbers of images fewer than four or greater than five are possible in other implementations.
[0107] At block 806, a valid entry detection controller (e.g., the valid entry detection controller 116) may detect or determine a valid entry status of the object in the container based on the at least one image captured at block 804 and a scanning event, such as a scanning event performed by the scanner 216 of the electronic handheld device 102. As described, in some implementations, the scanning event may provide the valid entry detection controller with an object ID of an object being scanned, which the valid entry detection controller may use to obtain a representative image data set corresponding to the object ID. The representative image data may include one or more representative images of a representative object, and / or a feature ID of the representative object, as previously described. The valid entry detection controller may then analyze and / or compare image data of the at least one image captured at block 804 with the representative image data set accessed from the database to determine a degree of similarity between the at least one captured image and the one or more representative images. For implementations where feature IDs are used, at block 806, the valid entry detection controller may also determine a feature ID of the object in the captured images. For example, as previously described, the valid entry detection controller may determine one or more feature values for one or more features (e.g., a shape, size, and / or color) , and in turn may generate a unique feature ID based on the one or more feature values for the object. The valid entry detection controller may then compare the feature ID for the object in the captured images with the feature ID of the representative object to determine a degree of similarity between the feature IDs.
[0108] The valid entry detection controller may then determine whether the degree of similarity satisfies a threshold criterion, such as meeting or exceeding a similarity threshold level or value. If the valid entry detection controller determines that the degree of similarity satisfies the threshold criteria, then the valid entry detection controller may determine that the valid entry status of the object is valid. On the other hand, if the valid entry detection controller determines that the degree of similarity does not satisfy the threshold criteria, then the valid entry detection controller may determine that the valid entry status of the object is invalid.
[0109] Also, as described, if the scanning event is performed on the same object that the sensor sensed at block 802 and that the camera captured the at least one image of at block 804, then the valid entry detection controller may determine that the valid entry status of the object is valid. However, if the scanning event is performed on a different object than the object that the sensor sensed at block 802 and / or that the camera captured the at least one image of at block 804, then the valid entry detection controller may determine that the valid entry status of the object is invalid. Moreover, if the scanning event does not occur (e.g., the user puts the object in the container without scanning) , then the valid entry detection controller may determine that the valid entry status of the object is invalid in response to the scanning event not occurring.
[0110] At block 808, an output device may output the valid entry status (e.g., a visual, audible, and / or tactile output indicating the valid entry status) of the object determined at block 806. As previously described, example output devices include the display 202, the light source 225 and / or the speaker 227 of the electronic handheld device 102, the light source 306 of the device storage 104, a light source 428 and / or a speaker 430 of the valid entry detection apparatus 108.
[0111] In some implementations, the valid entry detection controller, upon determining the valid entry status, may transmit a signal carrying the valid entry status to the output device in the electronic system 100 (e.g., the electronic handheld device 102, the device storage 104, and / or the valid entry detection apparatus 108) , such as via a network connection, and / or a wired or wireless connection (e.g., a wireless local area network (WLAN) connection) . The output device may then output the valid entry status according to the information carried in the signal via the connection.
[0112] Referring again to FIGs. 1, 2A-2F, and 4A-4E, in some implementations, the camera 420 of the valid entry detection apparatus 108 may be configured to operate in different modes, including a sleep mode and an active mode. In general, the valid entry detection apparatus is configured to consume less power when operating in the sleep mode than when operating in the active mode. For at least some of these implementations, in order to save power, the camera 420 may be configured to operate in the sleep mode before entering into an active time period during which the camera 420 operates in the active mode and takes one or more pictures. Accordingly, when the camera is to enter into an active time period, the camera 420 may transition or “wake up” from the sleep mode to the active mode. During the active mode, the camera 420 may take one or more pictures. At the end of the active time period when the camera is to no longer take pictures) , the camera 420 then transition back to the sleep mode.
[0113] Referring also to the flow chart in FIG. 8, in some implementations, before the sensor senses the presence of the object at block 802, the camera may be operating in the sleep mode. Then, at block 802 when the sensor senses the presence of the image and in response and the camera is triggered to take at least one picture of the image capture region, the camera may be triggered to transition from the sleep mode to the active mode in order to take the at least one picture. In some implementations, after the camera take the at least one picture, the camera may transition back to the sleep mode until it is again triggered to take at least one picture.
[0114] Additionally, in some implementations, a scanning event performed by the electronic handheld device 102 may trigger the camera 420 to take one or more initial pictures of the image capture region 500, which may occur with or without the object in the image capture region 500 in any of various situations or implementations. For example, when the scanner 216 of the electronic handheld device 102 is in the process of performing a scanning event, or in response to the scanner 216 performing a successful scanning event and obtaining the object ID of the object, the electronic handheld device 102 may transmit a signal indicating the scanning event to the valid entry detection apparatus 108. In response to receipt of the signal, a controller of the valid entry detection apparatus 108 may trigger the camera 420 to take the one or more initial pictures of the image capture region 500. Also, in some of these implementations, the scanning event may trigger the camera 420 to transition from the sleep mode to the active mode to take the one or more initial pictures. After taking the one or more initial pictures, the camera 420 may then transition back to the sleep mode. While it is expected that the sensor 422 will soon sense the object in the image capture region 500 shortly after the scanning event, configuring the camera 420 to transition into the sleep mode in between the scanning event and when the sensor 422 senses the presence of the object may still serve to conserve power and / or battery life, especially in situations where the user (e.g., shopper) takes a relatively long time to put the object in the container after scanning.
[0115] Additionally, in some of these implementations where the scanning event triggers the camera 420 to take one or more initial pictures, the one or more initial pictures taken by the camera 420 may be part of the set of images that the valid entry detection apparatus 108 transmits to the valid entry detection controller 116 for use in determining the valid entry status of the object. Accordingly, after taking the one or more initial pictures, the valid entry detection apparatus 108 may transmit the one or more initial pictures to the valid entry detection controller 116. The valid entry detection controller 116 may transmit the one or more initial pictures before or concurrently with the transmission of the pictures captured in response to the sensor 422 sensing the presence of the object in the image capture region 500, in any of various implementations.
[0116] FIG. 9 shows a flow chart of another example method 900 of valid entry status determination. At block 902, if a scanning event on an object (e.g., by the electronic handheld device 102) is not performed, but a sensor (e.g., the sensor 422 of the valid entry detection apparatus 108) senses a presence of an object in an image capture region adjacent to a container (e.g., above a shopping cart 110) at block 904, then at block 906, an output device (e.g., the display 202, the light source 225 and / or the speaker 227 of the electronic handheld device 102, the light source 306 of the device storage 104, and / or the light source 428 and / or the speaker 430 of the valid entry detection apparatus 108) may output a valid entry status of invalid for the object. In some implementations, an indication that the sensor sensed the presence of the object in the image capture region may be transmitted to a valid entry detection controller (e.g., the valid entry detection controller 116) . In response to receipt of the indication, the valid entry detection controller may determine whether a scanning event has occurred within a predetermined time period of the sensing of the object. If the scanning event has not occurred within the predetermined time period, then the valid entry detection controller may determine that the valid entry status of the object is invalid at block 906. In turn, the valid entry detection controller may transmit the valid entry status of invalid to the output device at block 906. Otherwise, if a scanning event is not performed at block 902 and the sensor has not sensed the presence of an object in block 904, then no further action may occur until a scanning event is performed at block 902 and / or the sensor senses the presence of the object at block 904, as indicated by the continuous loop of the flow chart in FIG. 9.
[0117] Referring back to block 902, if a scanning event is performed on an object (e.g., by the electronic handheld device 102) , then at block 908, a camera (e.g., the camera 420 of the valid entry detection apparatus 108) may capture one or more initial images of the image capture region. As previously described, in some implementations, the camera may transition from a sleep mode to an active mode in response to performance of the scanning event in order to take the one or more initial images. In some of these implementations, after taking the one or more initial images, the camera may transition back to the sleep mode. Also, in some implementations, at block 908, the one or more initial images may be transmitted (e.g., by the valid entry detection apparatus 108) to the valid entry detection controller, which may use the one or more initial images in its determination of the valid entry status of the object.
[0118] Referring back to block 902, in event that a scanning event is performed (e.g., performed successfully) , an object ID of the scanned object may be obtained by the device performing the scanning (e.g., the electronic handheld device 102) . In turn, the obtained object ID may be transmitted (e.g., by the electronic handheld device 102) to the valid entry status controller, such as via a network connection and / or a wired and / or wireless connection. In response to receipt of the object ID, the valid entry detection controller may use the object ID to retrieve from a database (e.g., database 118) a representative image data set that corresponds to the object ID. For example, the valid entry detection controller may identify the representative image data set that is associated with, or linked to, an object ID in the database that matches the object ID that the valid entry detection controller received in response to the scanning event. The valid entry detection controller may use the representative image data set for its valid entry status determination, as described in further detail below.
[0119] Additionally, in some implementations, a timer may be started when or after the scanning event is performed at block 902 and / or when or after the camera takes one or more initial pictures at block 904. In such implementations, at block 910, if the sensor has not yet sensed the presence of the scanned object in the image capture region, then in some implementations, and if the timer has not yet expired at block 912, then no further action may be performed until the timer expires at block 912 or the sensor senses the presence of the object in the image capture region at block 910. However, at block 912, if the timer expires, then it may be assumed that the scanned object will not be placed in the container and / or sensed by the sensor (block 914) , and the method 900 may proceed back to block 902 where further action occurs when a scanning event is performed at block 902 and / or the sensor senses an object in the image capture region at block 904. Other implementations of the method 900 may not utilize a timer to track an elapsed time after an object is sensed at block 902 and / or initial images of the object are captured at block 904.
[0120] Referring back to block 910, if the sensor senses the object in the image capture region, then at block 912, the camera takes at least one additional image of the image capture region including the object. As previously described, the sensor sensing the object at block 910 may trigger the camera to take the at least one additional image at block 916. Additionally, in some implementations, the sensor sensing the object may trigger the camera to transition from a sleep mode to an active mode to take the at least one additional image. In addition or alternatively, at block 916, the at least one additional image may be transmitted (e.g., by the valid entry detection apparatus 108) to the valid entry detection controller for determination of the valid entry status of the object. In some implementations, including those where the one or more initial images captured at block 908 are not yet transmitted to the valid entry detection controller, the transmission of the one or more additional images may also include the one or more initial images.
[0121] At block 918, the valid entry detection controller may analyze or compare image data associated with the one or more additional images and / or the one or more initial images captured by the camera with the representative image data set that the valid entry detection controller retrieved from the database. As previously described, in some implementations, the comparison may include a comparison of the captured images with representative images of the representative image data set. In addition or alternatively, the comparison may include a comparison of feature IDs associated with the captured images and the representative images. In such latter implementations, the valid entry detection controller may derive or determine a feature ID associated with the captured images. For example, the valid entry controller may determine one or more features values for one or more features of the object in the captured images, such as a shape value, a size value, and / or a color value, as non-limiting examples. the valid entry controller may determine the one or more feature values by analyzing the pixel data of the image data of the captured images. In turn, the valid entry controller may generate a feature ID based on the at least one feature values. The valid entry controller may then compare the feature ID corresponding to the captured images with the feature ID corresponding to the representative images.
[0122] Based on, or in response to, the comparison, the valid entry detection controller may determine a degree of similarity between an object identified or indicated in the images captured by the camera and an object identified in or indicated by the representative image data set. At block 920, the valid entry detection controller may determine whether a threshold criteria is satisfied, such as whether the degree of similarity meets or exceeds a similarity threshold level for example. If the valid entry detection controller determines that the threshold criteria is satisfied, then at block 922, the valid entry detection controller may determine that the valid entry status is valid for the object, as previously described. However, at block 920, if the valid entry detection controller determines that the threshold criteria is not satisfied, then at block 924, the valid entry detection controller may determine that the valid entry status is invalid for the object.
[0123] At block 926, an output device (e.g., the display 202, the light source 225 and / or the speaker 227 of the electronic handheld device 102, the light source 306 of the device storage 104, and / or the light source 428 and / or the speaker 430 of the valid entry detection apparatus 108) may output the valid entry status determined at block 922 or block 924 may output the valid entry status (e.g., a visual, audible, and / or tactile output indicating the valid entry status) . Additionally, in some implementations at block 926, the valid entry detection controller may transmit the valid entry status to the output device, such as via a network connection and / or a wired or wireless connection established between the valid entry detection controller and the output device.
[0124] Other implementations of the methods 600, 800, and / or 900 or other methods herein may include more or fewer blocks that the blocks shown in FIGs. 6A-6B, 8, and 9, respectively. For example, other methods may include fewer actions than those described for the methods 600, 800, and / or 900, and / or may include actions performed by any of various components as described herein, other than, or in addition to, those described for the methods 600, 800, and / or 900. In addition or alternatively, other methods may include combinations of the methods 600, 800, and / or 900. In addition or alternatively, other methods may include actions performed in different orders than those indicated in the flow charts for the method 600, 800, and / or 900. Accordingly, the flow charts for the methods 600, 800, and 900 are provided as examples, and are not meant to necessarily limit when certain actions must be performed relative to others.
[0125] FIG. 10 shows an operational schematic diagram of an example implementation of the electronic system 100. As shown in FIG. 10, the database controller 122 may populate, such as regularly or intermittently populate, the database 118 with new entries and / or update existing entries with representative image data sets associated with representative objects, as previously described. The valid entry detection apparatus 108, such as with the sensor 422, may detect the presence of an object 117. In response, the valid entry detection apparatus 108, such as with the camera 420, may capture one or more images of the object 117 in an image detection region adjacent to an opening 114 of a container (e.g., shopping cart) 110. As shown in FIG. 10, the valid entry detection apparatus 108 may transmit the captured image (s) to the valid entry detection controller 116.
[0126] As indicated in FIG. 10, in some implementations, the valid entry detection controller 116 may include a plurality of components, including a captured image queue module 1002, an image preprocessing module 1004, an object recognition module 1006, a feature extraction module 1008, a feature matching module 1010, and an image recognition results module 1012. As used herein, the term module refers to a circuit implemented in hardware or a combination of hardware and software, configured to perform one or more functions. With respect to the valid entry detection controller 116, each module 1002-1012, each module is configured to perform one or more functions associated with determining valid entry status of objects in images captured by the camera 420 of the valid entry detection apparatus 108. Correspondingly, each module may include its own designated hardware, and / or two or more modules may be part of the same hardware (e.g., the same processor) that performs the functions of the two or modules, such as by executing computer software. Various ways of implementing the modules 1002-1012 of the valid entry detection controller 116 may be possible.
[0127] In further detail, the captured image queue module 1002 may be configured to queue or store (e.g., temporarily store) the image data of the captured images received from the valid entry detection apparatus 108, such as until the captured image data is to be processed or analyzed. The image preprocessing module 1004 may be configured retrieve captured images from the captured image queue module 1002, and perform preprocessing of the captured image data. For example, the image preprocessing module 1004 may perform segmentation or other conditioning of the captured image data for image recognition.
[0128] After the image preprocessing module 1004 preprocesses the captured image data, the object recognition module 1006 may perform image data analysis on the preprocessed captured image data to recognize the part of the captured image data corresponding to the object 117 placed in the container 110. For example, the object recognition module 1006 may analyze pixel data to differentiate between the pixel data corresponding to the object 117 that was placed in the container 110 and other pixel data corresponding to parts of the captured image data other than the object 117, such as pixel data of the container 110, background images such as parts of the store, etc.
[0129] Upon the object recognition module 1006 recognizing the pixel data corresponding to the object 117, the feature extraction module 1008 may analyze the pixel data corresponding to the object 117 to determine one or more feature values for one or more features of the object 117, such as size, shape, and color, as previously described. The feature extraction module 1008 may then use the one or more feature value to generate a feature ID or code for the object 117.
[0130] Upon the feature extraction module 1008 generating the one or more feature values and / or the feature ID, the feature matching module 1010 may compare the feature ID for the object 117 with a feature ID of a selected representative image data set, as previously described. For example, in some implementations, the feature matching module 1010 may receive an object ID (e.g., a SKU number) of an object that is scanned by the electronic handheld device 102. The feature matching module 1010 may then use the received object ID as an index to determine an entry including a representative image data set associated with that object ID. The feature matching module 1010 may then determine the feature ID of the representative image data set and compare it with the feature ID corresponding to the captured image data. Based on the comparison, the feature matching module 1010 may determine a degree of similarity between the feature IDs.
[0131] The image recognition results module 1012 may obtain the degree of similarity, and in turn, determine whether the degree of similarity satisfies a threshold criteria, such as whether the degree of similarity meets or exceeds a similarity threshold level, as previously described. If the image recognition results module 1012 determines that the degree of similarity satisfies the threshold criteria, then the image recognition results module 1012 may determine that the object 117 has a valid entry status of valid. On the other hand, if the image recognition results module 1012 determines that the degree of similarity does not satisfy the threshold criteria, then the image recognition results module 1012 may determine that the object 117 has a valid entry status of invalid. Upon determining the valid entry status for the object 117, the image recognition results module 1012 may provide the valid entry status to the electronic handheld device 102, such as for display by the display 202 of the electronic handheld device 102.
[0132] Additionally, in some implementations as shown in FIG. 10, the scale 115 may transmit weight sensing data to the electronic handheld device 102, such as via a wireless connection. The weight sensing data may indicate a sensed or measured weight change, such as if an object is added to or removed from the container 110 to cause a weight change as sensed by the scale 115. Upon receipt of the weight sensing data, the electronic handheld device 102, such as with the controller 250 (FIG. 2F) , may determine whether the weight change is indicative of an improper placement of an object in the container 110. For example, the controller 250 may determine if the scanner 216 scanned an object within a predetermined time period before receipt of the weight change indication from the scale 115. If not, then the controller 250 may determine that an object was improperly placed in the container 110, and in turn cause the display 202 to output a warning or other message that an object was improperly placed in the cart without scanning.
[0133] Referring to FIGs. 2C and 2D, as previously mentioned, the electronic handheld device 102 may include a device electrical connector 224 configured to engage with, such as by physically and electrically connecting to, an electrical connector 308 of the device storage 104 when the handle 204 of the electronic handheld device 102 is in a receptacle 304 holding the electrical connector 308, and / or a holder electrical connector 414 of the valid entry detection apparatus 108 when the handle 204 is being held by the handheld device holder portion 404. As shown in FIGs. 2C and 2D, the device electrical connector 224 is integrated with the handle 204 and / or the handle portion 210 of the housing 206 at the second end 214. In some implementations such as shown in FIGs. 2C and 2D, the device electrical connector 224 may include one or more conductive contacts or leads 226 configured to physically or mechanically engage with a corresponding one or more pins (e.g., pogo pins) of the device storage electrical connector 308 and / or the holder electrical connector 414. The conductive contacts 226 may be mounted on a support (or support structure) 228 made of a non-conductive material that electrically insulates the conductive contacts 226 from each other.
[0134] Referring particularly to FIG. 2D, the electronic handheld device 102 may also include a battery (e.g., a rechargeable battery) 230 that powers the electronic components of the electronic handheld device 102. In some implementations, the battery 230 is a Lithium ion battery with a charge capacity of about 6, 800 milliampere-hour (mAh) ) , although other battery technology and / or other charging capacities for any of various other implementations of the battery 230 may be possible. Also, in some implementations such as shown in FIG. 2D, the battery 230 is disposed within the handle portion 210 of housing 206. Additionally, the device electrical connector 224 and the battery 230 may be electrically connected to each other. In particular implementations, such as shown in FIG. 2D, the device electrical connector 224 and the battery 230 are electrically connected to each other via a circuit board 232 to which various circuit or electronic components may be mounted, such as but not limited to an internal controller (e.g., including a processor) that communicates with and controls the various other electronic or circuit components of the electronic handheld device 102, such as the display 202, the scanner 216, the front camera 220, and the rear camera 222, in addition or alternatively to other electronic components described with reference to FIG. 2E. Further, as shown in FIG. 2D, one or more first electrical conductors (e.g., wires) 234 may be electrically connected to the conductive contacts 226 and extend from the conductive contacts 226 to the circuit board 232. As shown in FIG. 2D, the first conductors 234 may extend alongside the battery 230. Additionally, one or more second electrical conductors (e.g., wires) 236 may be electrically connected to the battery 230 and extend from the battery 230 to the circuit board 232.
[0135] When the handle 204 is positioned in a receptacle 304 of the device storage 104, the device storage 104 may supply charge to the electronic handheld device 102 to charge the battery 230. For example, electrical current may be supplied from the device storage 104 to the electronic handheld device 102 via the electrical connector 308 and the device electrical connector 224 (which are physically in contact and electrically connected to each other) . From the device electrical connector 224, electrical current flows through the first conductor 234 to the circuit board 232. The electrical current may then flow from the circuit board 232 through the second conductor 236 to the battery 230 to charge the battery 230. In some implementations, the controller of the electronic handheld device 102 may regulate or control the flow of the current to the battery 230.
[0136] In addition, as described, when the handle 204 is positioned in the handheld device holder portion 404 (FIG. 4C) , and the device electrical connector 224 and the holder electrical connector 414 are connected or engaged with each other, the battery 230 may charge the battery 426 of the valid entry detection apparatus 108. For example, electrical current may flow from the battery 230 through the second conductors 236 to the circuit board 232, and then through the first conductors 234 to the conductive contacts 226. The electrical current may then flow to the electrical connector 414, and to the battery of the valid entry detection apparatus 108.
[0137] In some implementations, a controller 250 (FIG. 2F) (e.g., a power management unit of the controller 250) of the electronic handheld device 102 may be configured to control the charging of the battery 426 of the valid entry detection apparatus 108 using the battery 230 of the electronic handheld device 102. For example, the controller 250 may be configured to detect a charge level of the battery of the valid entry detection apparatus 108 and a charge level of the battery 230. If the controller 250 detects that the battery 426 of the valid entry detection apparatus 108 is below an associated threshold level, and that the battery 230 of the electronic handheld device 102 is above an associated threshold level, then the controller 250 may facilitate the charging of the battery 426 of the valid entry detection apparatus 108 with the battery 230.
[0138] Referring to FIG. 2D and also to FIG. 2E, for at least some implementations of the electronic handheld device 102, the device electrical connector 224 may be removably coupled to the housing 206 via at least one coupling member. By being removably coupled, the device electrical connector 224 can be detached from the electronic handheld device 102, such as shown in FIG. 2E. In this regard, the device electrical connector 224 may be a modular component of the electronic handheld device 102 that can be attached to and detached from the electronic handheld device 102. Moreover, through the removable coupling and / or modular configuration, the device electrical connector 224 may be a replaceable component. For example, one device electrical connector 224 may be removed or detached from the electronic handheld device 102, and may be replaced by a new or at least different device electrical connector 224, which may be attached or coupled to, or installed onto, the electronic handheld device 102 in the same way that first device electrical connector 224 was. Also, as used herein, a coupling member is some component or apparatus that keeps the device electrical connector 224 coupled or attached to the electronic handheld device 102, and that is manipulated or has a configuration changed in order for the device electrical connector to be uncoupled, detached or removed from electronic handheld device 102.
[0139] As shown in FIG. 2D, the at least one coupling member may include a removable cover 238 of the housing 206 that may form at least a portion of the handle portion 210 of the housing 206. In order to detach the device electrical connector 224 from the electronic handheld device 102, the removable cover 238 may be removed and / or detached from the rest of the housing 206, such as shown in FIG. 2D. In some implementations, the removable cover 238 is secured to the housing 206 with a fastener (e.g., a screw) . In order to detach the removable cover 238 from the rest of the housing 206, the faster is manipulated (e.g., unscrewed) to be detached from the housing 206, allowing the removable cover 238 to be detached. In this context, the fastener (e.g., screw) may also be part of the at least one coupling mechanism. Other implementations may not use a faster (e.g., screw) , and instead use other mechanisms, such as tabs or clips or other similar structures to provide a removable coupling between the removable cover 238 and the rest of the housing 206. In still other implementations, the at least one coupling member may not include a removable cover. For example, the housing 206 of the electronic handheld device 102 may not have a removable cover. As another example, the electronic handheld device 102 have a removable cover, such as to gain access to the battery 230, but may not require such a removable cover to be removed in order to detach the device electrical connector 224 from the electronic handheld device 102.
[0140] Referring to FIG. 2E, in some implementations, the device electrical connector 224, including the support 228, may be removably disposed in a receptacle 240 of and / or within the housing 206. In addition or alternatively, in some implementations, the at least one coupling member couples or secures the support 228 to the housing 206, such as within an inner surface 242 of the housing 206 and / or within the receptacle 240. Referring also to FIG. 2D, in some of these implementations, the at least one coupling member includes a fastener (e.g., a screw) 244 that couples or secures the support 228 to the housing 206, such as to the inner surface 242 of the housing 206 and / or within the receptacle 240. The fastener (e.g., screw) 244 is detached or uncoupled from the housing 206 in order to remove or detach the device electrical connector 224 from the housing 206. For example, as shown in FIG. 2E, the screw 244 is unscrewed or threadingly disengaged from a corresponding threaded bore 246 to allow the device electrical connector 224 to be removed from the receptacle 240. Other implementations may be possible. For example, the at least one coupling mechanism may include a clip or other similar mechanism and secures the device electrical connector 224 to the housing 206 and is manipulated in order to detach or remove the device electrical connector 224 from the handle 204. In other implementations, the at least one coupling member may not fixedly or securely attach the device electrical connector 224 to the inner surface 242 of the housing 206. For example, the support 228 may be removably disposed within the receptacle 240 without being securely attached to the inner surface 242 of the housing 206 using a removable coupling member such as a screw.
[0141] Still referring to FIG. 2E, in some implementations, the first conductors 234 may be removably coupled to the circuit board 232. For at least some of these implementations, the first conductors 234 may be considered part of the device electrical connector 224 that is removably coupled to the electronic handheld device 102. In addition or alternatively, in some of these implementations such as shown in FIGs. 2D and 2E, the at least one coupling member may include one or more wire-to-board connectors 248 that provide a removable coupling between the first conductors 234 and the circuit board 232. As shown in FIG. 2E, the first conductors 234 may include wire-end portions 248a of the wire-to-board connectors 248, and the circuit board 232 may include corresponding board-end portions 248b of the wire-to-board connectors 248. To physically and electrically connect the first conductors 234 to the circuit board 232, the wire-end portions 248a may be coupled or attached to the board-end portions 248b. In addition, to physically and electrically disconnect the first conductors 234 from the circuit board 232, the wire-end portions 248a may be uncoupled or detached from the board-end portions 248b. In this context, the one or more wire-to-board connectors 248 may form at least a part of the at least one coupling member in that the wire-end portions 248a are decoupled or detached from the board-end portions 248b in order to uncouple or detach the device electrical connector 224 from the electronic handheld device 102. In other implementations, the first conductors 234 are fixedly attached to the circuit board 232, and may be removably coupled to the conductive contacts 226 and / or the support 228.
[0142] The removable coupling configurations of the device electrical connector 224 described herein may allow or facilitate the replacement of the device electrical connector 224 without having to replace the entire electronic handheld device 102. In addition or alternatively, in the implementations described herein, the removable coupling configuration may allow end users or owners of the electronic handheld device 102 to be able to replace the device electrical connector 224 themselves, such as onsite (e.g., within the retailer store by an employee of the retailer) , instead of having a manufacturer or seller of the electronic handheld device 102, or another third party, perform the replacement, such as by the end user or owner sending the electronic handheld device 102 to the manufacture, seller, or third party for serving or repair. This, in turn, may provide substantial reduction in downtime of the electronic handheld device 102 and / or provide substantial cost saving to the owner of the electronic handheld device 102, especially in situations where the owner (e.g. a retailer or store owner) owns tens, hundreds, or even thousands of such electronic handheld devices 102 at a single time.
[0143] FIG. 2F shows a block diagram of at some electronic components of the electronic handheld device 102. As previously described, the electronic handheld device 102 may include a display 202 (e.g., a touchscreen) . In addition, the electronic handheld device 102 may include a battery (e.g., a rechargeable battery) 230 that is configured to power the other electronic components of the electronic handheld device 102. Also, as mentioned, the electronic handheld device 102 may include a controller 250 configured to control or facilitate operation of the electronic handheld device 102 and / or control operation of one or more of the other electronic components of the electronic handheld device 102. The controller 250 may include circuitry implemented as hardware only or a combination of hardware and software and / or firmware. For example, the controller 250 may include digital logic circuitry, an integrated circuit (IC) , a field programmable gate array (FPGA) , an application specific integrate circuit (ASIC) , a processor (e.g., a central processing unit (CPU) ) configured to execute software and / or firmware, such as in the form of computer readable instructions, in order to perform one or more functions or operations, or any of various combinations thereof. In at least some implementations, the controller 250 includes memory, including volatile memory (e.g., random access memory RAM) ) , non-volatile memory (e.g., read-only memory (ROM) ) , or any of various combination of volatile memory and non-volatile memory. The memory may store at least the computer readable instructions executable by the processor. Various ways of implementing the controller 250 in hardware or a combination of hardware and software may be possible. In addition, as previously described, the controller may include one or more data capture components 252, such as the scanner 216, the front camera 220, and / or the rear camera 222. As previously described, the front camera 220 and / or the rear camera 222 may be used to capture one or more representative images of the representative object. In addition, in some implementations, the front camera 220 may track the location (e.g., geo-location) of the electronic handheld device 102, such as by using an associated image-capturable map positioned on a ceiling of a building (e.g., store) in which the electronic handheld device 102 is being operated. In addition or alternatively, the front camera 220 and / or the rear camera 222 may be may be used for facial or other types of recognition.
[0144] Also, for a least some implementations, the electronic handheld device 102 may include one or more sensor components 254, including one or more of a scanner proximity sensor 256, a magnetometer 258, a gyroscope 260, and / or a barometer 262. The scanner proximity sensor 256 may operate in conjunction with the scanner 216 and detect the presence of an object within a predetermined distance of the electronic handheld device 102 or the scanner 216 in particular. The scanner proximity sensor 256 detecting the presence of an object may trigger the scanner 216 to start scanning. The inclusion of the scanner proximity sensor 256 may provide hands-free scanning, at least in that a user (e.g., shopper) can place the electronic handheld device 102 in the handheld device holder portion 404 of the valid entry detection apparatus 108, and then position an object within the predetermined distance in front of the scanner 216 to cause the scanner to automatically scan the object.
[0145] The magnetometer 258 may be configured to test a strength and / or direction of magnetic fields and locate an orientation of the electronic handheld device 102. The gyroscope 260 may be configured to measure, detect, and / or maintain directional stability. The barometer 262 may be configured to measure atmospheric pressure of a surrounding environment in which the electronic handheld device 102 is positioned and / or operating. In some implementations, the barometer may be set for a particular floor, particular for implementations where the building (e.g., store) has several floors.
[0146] In addition, the electronic handheld device 102 may include transceiver circuitry 264, such as an antenna (e.g., an antenna array) and associated transmitter and / or receiver front-end circuitry, to enable the electronic handheld device 102 to wirelessly communicate (transmit and receive) . In addition, the electronic handheld device 102 may include network circuitry 266 to enable the electronic handheld device 102 to connect to a network and / or establish a network connection using any of various wired and / or wireless technology, such as a local area network (LAN) , a wireless local area network (WLAN) , a wide area network (WAN) , a personal area network (PAN) , a wireless personal area network (PAN) , a Bluetooth network, or a virtual private area network, as non-limiting examples. In addition or alternatively, the electronic handheld device 102 may include near-field communication (NFC) circuitry 268 configured to enable contactless payment. For example, for implementations where the electronic handheld device 102 is used in retail shopping, when a shopper is finished shopping and wants to pay for his / her items, the shopper can position a payment card (e.g., a credit card, a debit card, a loyalty card, a gift card, etc. ) a predetermined distance above the display, which may cause the NFC circuitry 268 to detect and / or read the payment card, and in turn effect payment of the items.
[0147] In addition or alternatively, the electronic handheld device 102 may include a radio frequency identification (RFID) tag 270 configured to provide a location of the electronic handheld device 102, such as within a building (e.g., a store) or other area of intended use. In some embodiments, the RFID tab 272 may be embedded in an inner surface of the housing 206. The RFID tag 270 may be used in conjunction with a RFID device (e.g., a RFID gate or a RFID reader) , which may be operated in a building (e.g., store) in which the electronic handheld device 102 is used, and / or by an employee of a retailer in the shopping context for example. Such a RFID device may emit an RF (e.g., an ultra high frequency (UHF) ) signal, such as in the range of 800-900 Megahertz (MHz) for example, causing the RFID tag 270 to respond by emitting a response signal indicating a location of the electronic handheld device 102. In turn, the RFID device may receive the emitted signal from the RFID tag 270 and locate the electronic handheld device 102, such as within the building or store. In some implementations, the RFID tag 270 may function to locate lost or misplaced electronic handheld devices 102, especially in situations where the battery 230 has died and as a result, the electronic handheld device 102 cannot, or will likely not, be located through other electronic means. In addition or alternatively, in some implementations, in event a user attempts to move the electronic handheld device 102 outside or beyond an authorized area (e.g., outside of a store) , a RFID device may activate or excite the RFID tag 270, which in turn may activate an alarm, such as within the building.
[0148] FIG. 11 shows a computer system 1100, some or all of which may be representative of a computer system used for, implemented in, and / or used to implement the device authorization and assignment system 106, the valid entry detection controller 116, and / or the database controller 122. The components shown in FIG. 11 for computer system 1100 are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. Neither should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of a computer system 1100.
[0149] The computer system 1100 may include certain human interface input devices. Such a human interface input device may be responsive to input by one or more human users through, for example, tactile input (such as: keystrokes, swipes, data glove movements) , audio input (such as: voice, clapping) , visual input (such as:gestures) , olfactory input (not depicted) . The human interface devices can also be used to capture certain media not necessarily directly related to conscious input by a human, such as audio (such as: speech, music, ambient sound) , images (such as: scanned images, photographic images obtain from a still image camera) , video (such as two-dimensional video, three-dimensional video including stereoscopic video) .
[0150] Additionally, input human interface devices may include one or more of (only one of each depicted) : keyboard 1101, mouse 1102, trackpad 1103, touch screen 1110, data-glove (not shown) , joystick 1105, microphone 1106, scanner 1107, camera 1108.
[0151] The computer system 1100 may also include certain human interface output devices. Such human interface output devices may be stimulating the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human interface output devices may include tactile output devices (for example tactile feedback by the touch-screen 1110, data-glove (not shown) , or joystick 1105, but there can also be tactile feedback devices that do not serve as input devices) , audio output devices (such as: speakers 1109, headphones (not depicted) ) , visual output devices (such as screens 1110 to include CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch-screen input capability, each with or without tactile feedback capability-some of which may be capable to output two dimensional visual output or more than three dimensional output through means such as stereographic output; virtual-reality glasses (not depicted) , holographic displays and smoke tanks (not depicted) ) , and printers (not depicted) .
[0152] The computer system 1100 can also include human accessible storage devices and their associated media such as optical media including CD / DVD ROM / RW 1120 with CD / DVD or the like media 1121, thumb-drive 1122, removable hard drive or solid state drive 1123, legacy magnetic media such as tape and floppy disc (not depicted) , specialized ROM / ASIC / PLD based devices such as security dongles (not depicted) , and the like.
[0153] Those skilled in the art should also understand that the term “computer readable media” as used in connection with the presently disclosed subject matter does not encompass transmission media, carrier waves, or other transitory signals.
[0154] The computer system 1100 can also include an interface 1154 to one or more communication networks 1155. Networks can for example be wireless, wireline, optical. Networks can further be local, wide-area, metropolitan, vehicular and industrial, real-time, delay-tolerant, and so on. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks to include GSM, 3G, 4G, 5G, LTE and the like, TV wireline or wireless wide area digital networks to include cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial to include CAN bus, and so forth. Certain networks commonly require external network interface adapters that attached to certain general-purpose data ports or peripheral buses 1149 (such as, for example USB ports of the computer system 1100) ; others are commonly integrated into the core of the computer system 1100 by attachment to a system bus as described below (for example Ethernet interface into a PC computer system or cellular network interface into a smartphone computer system) . Using any of these networks, computer system (1100) can communicate with other entities. Such communication can be uni-directional, receive only (for example, broadcast TV) , uni-directional send-only (for example CANbus to certain CANbus devices) , or bi-directional, for example to other computer systems using local or wide area digital networks. Certain protocols and protocol stacks can be used on each of those networks and network interfaces as described above.
[0155] Aforementioned human interface devices, human-accessible storage devices, and network interfaces can be attached to a core 1140 of the computer system 1100.
[0156] The core 1140 can include one or more Central Processing Units (CPU) 1141, Graphics Processing Units (GPU) 1142, specialized programmable processing units in the form of Field Programmable Gate Areas (FPGA) 1143, hardware accelerators for certain tasks 1144, graphics adapters 1150, and so forth. These devices, along with Read-only memory (ROM) 1145, Random-access memory 1146, internal mass storage such as internal non-user accessible hard drives, SSDs, and the like 1147, may be connected through a system bus 1148. In some computer systems, the system bus 1148 can be accessible in the form of one or more physical plugs to enable extensions by additional CPUs, GPU, and the like. The peripheral devices can be attached either directly to the core’s system bus 1148, or through a peripheral bus 1149. In an example, the screen 1110 can be connected to the graphics adapter 1150. Architectures for a peripheral bus include PCI, USB, and the like.
[0157] CPUs 1141, GPUs 1142, FPGAs 1143, and accelerators 1144 can execute certain instructions that, in combination, can make up the aforementioned computer code. That computer code can be stored in ROM 1145 or RAM 1146. Transitional data can also be stored in RAM 1146, whereas permanent data can be stored for example, in the internal mass storage 1147. Fast storage and retrieve to any of the memory devices can be enabled through the use of cache memory, that can be closely associated with one or more CPU 1141, GPU 1142, mass storage 1147, ROM 1145, RAM 1146, and the like.
[0158] The computer readable media can have computer code thereon for performing various computer-implemented operations. The media and computer code can be those specially designed and constructed for the purposes of the present disclosure, or they can be of the kind well known and available to those having skill in the computer software arts.
[0159] As a non-limiting example, the computer system having architecture 1100, and specifically the core 1140 can provide functionality as a result of processor (s) (including CPUs, GPUs, FPGA, accelerators, and the like) executing software embodied in one or more tangible, computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage as introduced above, as well as certain storage of the core 1140 that are of non-transitory nature, such as core-internal mass storage 1147 or ROM 1145. The software implementing various embodiments of the present disclosure can be stored in such devices and executed by core 1140. A computer-readable medium can include one or more memory devices or chips, according to particular needs. The software can cause the core 1140 and specifically the processors therein (including CPU, GPU, FPGA, and the like) to execute particular processes or particular parts of particular processes described herein, including defining data structures stored in RAM 1146 and modifying such data structures according to the processes defined by the software. In addition, or as an alternative, the computer system can provide functionality as a result of logic hardwired or otherwise embodied in a circuit (for example: accelerator 1144) , which can operate in place of or together with software to execute particular processes or particular parts of particular processes described herein. Reference to software can encompass logic, and vice versa, where appropriate. Reference to a computer-readable media can encompass a circuit (such as an integrated circuit (IC) ) storing software for execution, a circuit embodying logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware and software.
[0160] The subject matter of the present description may also relate, among others, to the following aspects:
[0161] A first aspect includes an apparatus for valid entry detection, the apparatus comprising: a body comprising: a container attachment portion configured to attach to a corresponding attachment area of a container, the container attachment portion comprising an inner surface configured to engage with a corresponding surface of the attachment area to establish a secure attachment to the attachment area of the container; a handheld device holder portion configured to receive and hold an electronic handheld device, the handheld device holder portion comprising a wall having an inner surface defining a volume for receipt of the electronic handheld device, the inner surface comprising a holder electrical connector configured to physically and electrically connect to a corresponding device electrical connector upon receipt of the electronic handheld device within the volume; and a camera disposed within the body, the camera oriented relative to the container attachment portion such that when the container attachment portion establishes the secure attachment to the attachment area of the container, the camera is configured to establish an image capture region adjacent to an opening of the container.
[0162] A second aspect includes the first aspect, and further includes: a holder battery configured to be charged by a battery of the handheld device via the holder electrical connector connected to the device electrical connector.
[0163] A third aspect includes any of the first or second aspects, and further includes : a circuit configured to establish a network connection with the electronic handheld device.
[0164] A fourth aspect includes the third aspect, and further includes wherein the network connection comprises a wireless network connection.
[0165] A fifth aspect includes any of the first through fourth aspects, and further includes wherein the camera is configured to take at least one initial image of the image capture area in response to a scanning event performed by the electronic handheld device.
[0166] A sixth aspect includes the fifth aspect, and further includes wherein the camera is configured to transition from a sleep mode to an active mode to take the at least one initial image in response to the scanning event.
[0167] A seventh aspect includes the sixth aspect, and further includes wherein the camera is configured to transition from the active mode to the sleep mode after taking the at least one initial image.
[0168] An eighth aspect includes any of the fifth through seventh aspects, and further includes: a circuit configured to: establish a network connection with electronic the handheld device; detect the scanning event performed by the electronic handheld device via the network connection; and trigger the camera to take the at least one initial image of the image capture area in response to detection of the scanning event.
[0169] A ninth aspect includes any of the first through eighth aspects, and further includes: a sensor disposed within the body and operatively coupled to the camera, wherein the sensor is configured to: sense a presence of an object in the image capture area; and in response to sensing the presence of the object, trigger the camera to take at least one image of the image capture area including the object.
[0170] A tenth aspect includes any of the first through ninth aspects, and further includes wherein when the secure connection is established, the image capture area comprises a lower boundary that is substantially parallel with the opening of the container.
[0171] An eleventh aspect includes any of the first through tenth aspects, and further includes wherein the container attachment portion comprises a clamp.
[0172] A twelfth aspect includes any of the first through eleventh aspects, and further includes wherein the container attachment portion is configured to relatively move about the attachment area to vertically adjust the image capture region.
[0173] A thirteenth aspect includes any of the first through twelfth aspects, and further includes wherein the camera is configured to adjust an angle of view to horizontally adjust the image capture region.
[0174] A fourteenth aspect includes any of the first through thirteenth aspects, and further includes wherein the holder electrical connector is removably coupled to the handheld device holder portion.
[0175] A fifteenth aspect includes any of the first through fourteenth aspects, and further includes a through hole extending through the wall of the handheld device holder portion, the through hole configured to drain liquid in the volume.
[0176] A sixteenth aspect includes a system comprising: an electronic handheld device comprising: a display, a scanner, and a handle; and a valid entry detection apparatus for a container, the valid entry detection apparatus comprising: a body comprising: a container attachment portion configured to attach to an attachment area of the container; and a handheld device holder portion configured to removably receive and hold the handle of the electronic handheld device; and a camera disposed in the body, the camera configured to establish an image capture region adjacent to an opening of the container when the valid entry detection apparatus is securely attached to the attachment area of the container.
[0177] A seventeenth aspect includes the sixteenth aspect, and further includes wherein the camera is configured to capture at least one image of the image capture area in response to the scanner performing a scanning event on an object.
[0178] An eighteenth aspect includes the seventeenth aspect, and further includes wherein the electronic handheld device and the valid entry detection apparatus are configured to establish a network connection with each other, and wherein the electronic handheld device is configured to communicate the scanning event to the valid entry detection apparatus via the network connection.
[0179] A nineteenth aspect includes the eighteenth aspect, and further includes wherein the network connection comprises a wireless network connection.
[0180] A twentieth aspect includes any of the eighteenth or nineteenth aspects, and further includes wherein the electronic handheld device is configured to communicate the scanning event via the network connection when the handle is outside of the handheld device holder portion of the valid entry detection apparatus.
[0181] A twenty-first aspect includes any of the sixteenth through twentieth aspects, and further includes wherein the valid entry detection apparatus further comprises a sensor configured to sense a presence of an object in the image capture area, wherein the camera is configured to capture at least one image of the image capture area with the object in response to detection of the presence of the object by the sensor, and wherein the display of the electronic handheld device is configured to display a valid entry status of the object based on the at least one image of the image capture with the object and a scanning event performed by the scanner of the electronic handheld device.
[0182] A twenty-second aspect includes a system comprising: a container configured to hold one or more items, the container comprising an attachment area; and a valid entry detection apparatus comprising: a body comprising: a container attachment portion attached to the attachment area of the container; and a handheld device holder portion configured to removably receive and hold an electronic handheld device; and a camera disposed in the body, the camera configured to establish an image capture region adjacent to an opening of the container when the valid entry detection apparatus is attached to the attachment area of the container.
[0183] A twenty-third aspect includes the twenty-second aspect, and further includes wherein the container comprises a shopping cart.
[0184] A twenty-fourth aspect includes the twenty-third aspect, and further includes wherein the attachment area comprises a handlebar of the shopping cart.
[0185] A twenty-fifth aspect includes the twenty-fourth aspect, and further includes wherein the container attachment portion comprises a clamp that is clamped to the handlebar, the clamp being rotatable about the handlebar to adjust a vertical position of the image capture region.
[0186] A twenty-sixth aspect includes any of the twenty-second through twenty-fifth aspects, and further includes wherein a lower boundary of the image capture region is substantially parallel with an opening of the container.
[0187] A twenty-seventh aspect includes any of the twenty-second through twenty-sixth aspects, and further includes wherein the camera is configured to adjust a horizontal angle of view relative to the opening of the container.
[0188] A twenty-eighth aspect includes any of the twenty-second through twenty-seventh aspects, and further includes: a scale disposed at a base of the container, the scale configured to measure a weight of one or more objects in the container.
[0189] A twenty-ninth aspect includes the twenty-eighth aspect, and further includes wherein the scale and the valid entry detection apparatus form a network connection with each other.
[0190] A thirtieth aspect includes the twenty-ninth aspect, and further includes wherein the network connection comprises a Bluetooth connection.
[0191] A thirty-first aspect includes a method of configuring a container for valid entry status detection: attaching a valid detection status apparatus to an attachment area of a container, wherein the valid entry status apparatus comprises a camera; and establishing an image capture region adjacent to an opening of the container.
[0192] A thirty-second aspect includes the thirty-first aspect, and further includes wherein the container comprises a shopping cart, and the image capture area is above an opening of the shopping cart.
[0193] A thirty-third aspect includes the thirty-second aspect, and further includes wherein the attachment area comprises a handlebar of the shopping cart.
[0194] A thirty-fourth aspect includes any of the thirty-first or thirty-second aspects, and further includes wherein attaching the valid entry status apparatus to the attachment area comprises clamping the valid entry status apparatus to the attachment area.
[0195] A thirty-fifth aspect includes any of the thirty-first through thirty-fourth aspects, and further includes wherein establishing the image capture region comprises adjusting a rotatable position of the valid entry status apparatus about the attachment area until a lower boundary of the image capture region is substantially parallel with the opening of the container.
[0196] A thirty-sixth aspect includes any of the thirty-first through thirty-fifth aspects, and further includes wherein establishing the image capture region comprises adjusting a horizontal angle of view of the camera relative to the opening of the container.
[0197] A thirty-seventh aspect includes any of the thirty-first through thirty-sixth aspects, and further includes positioning a scale with a base of a basket of the shopping cart.
[0198] A thirty-eighth aspect includes a container valid entry detection method, the method comprising: sensing, by a sensor, a presence of an object in an image capture region adjacent to an opening of a container; capturing, by a camera, at least one image of the object in the image capture region in response to the sensor sensing the presence of the object in the image capture region as the object is moving to enter into the container; detecting, by a valid entry detection controller, a valid entry status of the object in the container based on the at least one image and a scanning event; outputting, by an output device, the valid entry status of the object.
[0199] A thirty-ninth aspect includes the thirty-eighth aspect, and further includes wherein the object comprises a first object, the method further comprising: performing, by a scanner of the electronic handheld device, the scanning event on a second object before the first object is sensed by the sensor; in response to the scanning event, obtaining, by the valid entry detection controller, a representative image data set associated with the second object from a database; and comparing, by the valid entry detection controller, the at least one image captured by the camera with a representative image data set associated with the second object that is scanned, wherein detecting the valid entry status of the first object is based on the comparing.
[0200] A fortieth aspect includes the thirty-ninth aspect, and further includes wherein detecting the valid entry status of the first object comprises detecting that the valid entry status is valid in response to the comparison indicating that the first object in the at least one image captured by the camera sufficiently matches a representative object of the representative image data set.
[0201] A forty-first aspect includes the fortieth aspect, and further includes wherein the first object that is sensed by the sensor is the same as the second object that is scanned by the scanner.
[0202] A forty-second aspect includes any of the fortieth or forty-first aspects, and further includes wherein detecting the valid entry status of the first object comprises detecting that the valid entry status is invalid in response to the comparison indicating that the first object in the at least one image captured by the camera insufficiently matches a representative object of the representative image data set.
[0203] A forty-third aspect includes the forty-second aspect, and further includes wherein the first object that is sensed by the sensor is different than the second object that is scanned by the scanner.
[0204] A forty-fourth aspect includes any of the thirty-ninth through forty-third aspects, and further includes wherein the representative image data set comprises at least one representative image associated with the second object, and wherein the comparing comprises comparing the at least one image captured by the camera with the at least one representative image.
[0205] A forty-fifth aspect includes any of the thirty-ninth through forth-fourth aspects, and further includes wherein the representative image data set comprises a representative feature identification (ID) derived from one or more features of at least one representative image associated with the second object, and wherein the comparing comprises comparing a feature ID code derived from one or more features of the first object in the at least one image captured by the camera with the representative feature ID.
[0206] A forty-sixth aspect includes any of the thirty-ninth through forty-fifth aspects, and further includes: in response to the comparing, determining a degree of similarity between the first object in the at least one image captured by the camera and at least one representative object associated with the representative image data set.
[0207] A forty-seventh aspect includes the forty-sixth aspect, and further includes: determining whether the degree of similarity satisfies a threshold criteria, wherein detecting the valid entry status is based on whether the degree of similarity satisfies the threshold criteria.
[0208] A forty-eighth aspect includes any of the thirty-ninth through forty-seventh aspects, and further includes: in response to the scanning event, receiving, by the valid entry controller, an object identification (ID) of the second object derived from performing the scanning event on the second object, wherein obtaining the representative image data set comprises obtaining the representative image data set in response to the representative image data set being linked to the object ID.
[0209] A forty-ninth aspect includes any of the thirty-ninth through forty-eighth aspects, and further includes: capturing, by the camera, at least one initial image in response to the scanner performing the scanning event before the camera captures the at least one image of the first object in response to the sensor sensing the object in the image capture region, wherein the set of images further comprises the at least one initial image.
[0210] A fiftieth aspect includes any of the thirty-eighth through forty-ninth aspects, and further includes: receiving, by a valid entry detection controller, an indication that the object is in the sensing region; in response to receiving the indication, determining, by the valid entry detection controller, that no scanning event has occurred within a predetermined time period before receiving the indication, wherein detecting the valid entry status of the object comprises detecting that the valid entry status is invalid in response to determining that the scanning event has not occurred within the predetermined time period before receiving the indication.
[0211] A fifty-first aspect includes any of the thirty-eighth through fifty-fiftieth aspects, and further includes wherein the output device comprises a display, and wherein outputting the valid entry status comprises displaying the valid entry status of the object on the display.
[0212] A fifty-second aspect includes any of the thirty-eighth through fifty-first aspects, and further includes wherein the output device comprises a light source, and wherein outputting the valid entry status comprises emitting a light from the light source to indicate the valid entry status.
[0213] A fifty-third aspect includes any of the thirty-eighth through fifty-second aspects, and further includes wherein the output device comprises a speaker, and wherein outputting the valid entry status comprises outputting a sound from the speaker to indicate the valid entry status.
[0214] A fifty-fourth aspect includes a system comprising: a memory storing a plurality of instructions; and a processor configured to execute the plurality of instructions, and upon execution of the plurality of instructions, is configured to: receive a captured image data set of a first object in an image capture region adjacent to an opening of a container; receive a representative image data set associated with a second object in response to a scanning event performed by a scanner on the second object; compare the captured image data set with the representative image data set to determine a degree of similarity between the first object and the second object; determine a valid entry status of the second object in a container based on whether the degree of similarity satisfies a threshold criteria; and output the valid entry status to an output device.
[0215] A fifty-fifth aspect includes the fifty-fourth aspect, and further includes wherein the captured image data set comprises at least one captured image of the first object in the capture region, wherein the representative image data set comprises at least one representative image associated with the second object, and wherein in order to compare the captured image data set with the representative image data set, the processor is configured to compare the at least one captured image with the at least one representative image.
[0216] A fifty-sixth aspect includes any of the fifty-fourth through fifty-fifth aspects, and further includes wherein the captured image data set comprises a feature identification (ID) of the first object, wherein the representative image data set comprises a feature ID associated with the second object, and wherein in order to compare the captured image data set with the representative image data set, the processor is configured to compare the feature ID of the first object with the feature ID associated with the second object.
[0217] A fifty-seventh aspect includes the fifty-sixth aspect, and further includes wherein the processor, upon execution of the plurality of instructions, is further configured to: analyze at least one image in the image capture region captured by a camera to determine one or more features of the first object; and derive the feature ID of the first object based on the one or more features.
[0218] A fifty-eighth aspect includes any of the fifty-fourth through fifty-seventh aspects, and further includes wherein the memory further stores: a database comprising a plurality of entries, wherein each of the plurality entries links an object identification (ID) of a respective representative object with a respective representative data set.
[0219] A fifty-ninth aspect includes the fifty-eighth aspect, and further includes wherein the processor, upon execution of the plurality of instructions, is further configured to: identify an object ID of the second object from the scanning event; identify the representative image data set by identifying that the representative image data set is linked to the object ID of the second object in the database.
[0220] A sixtieth aspect includes an electronic handheld device comprising: a display; an electronic scanner; a housing comprising: a display portion holding the display; a scanner portion holding the electronic scanner; an elongate handle portion extending from a first end to a second end, wherein the first end is connected to the display portion, and the second end is opposite the first end; a device electrical connector integrated with the elongate handle portion, the electrical connector comprising at least one conductive contact exposed from within the housing, wherein the electrical connector is removably coupled to the housing via at least one coupling member.
[0221] A sixty-first aspect includes the sixtieth aspect, and further includes wherein the electrical connector further comprises: a support on which the at least one conductive contact is mounted, wherein the support is removably disposed within a receptacle of the housing within the elongate handle portion at the second end.
[0222] A sixty-second aspect includes the sixty-first aspect, and further includes wherein the at least one coupling member secures the support to an inner surface of the housing when the support is in the receptacle.
[0223] A sixty-third aspect includes any of the sixtieth through sixty-second aspects, and further includes wherein the electrical connector further comprises: at least one conductor electrically connected to the at least one conductive contact and electrically connected to a circuit board disposed within the housing, wherein the at least one conductor is removably coupled to the circuit board via a wire-to-board connector of the least one coupling member.
[0224] A sixty-fourth aspect includes the sixty-third aspect, and further includes a battery disposed within the elongate handle portion of the housing, wherein the at least one conductive contact is electrically connected to the battery via the at least one conductor and the circuit board.
[0225] A sixty-fifth aspect includes the sixty-fourth aspect, and further includes wherein the at least one conductor extends from a support of the least one contact and alongside the battery to the circuit board.
[0226] A sixty-sixth aspect includes any of the sixtieth through sixty-fifth aspects, and further includes wherein the housing further comprises a removable cover that forms a portion of the elongate handle portion, and wherein the at least one coupling member comprises the cover.
[0227] A sixty-seventh aspect includes any of the sixtieth through sixty-sixth aspects, and further includes: a proximity sensor configured to: detect an object within a predetermined range of the proximity sensor; and trigger the electronic scanner to scan the object in response to detection of the object.
[0228] A sixty-eighth aspect includes any of the sixtieth through sixth-seventh aspects, and further includes a radio frequency (RF) identification (ID) tag.
[0229] The foregoing description of various embodiments of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. Numerous modifications or variations are possible in light of the above teachings. The embodiments discussed were chosen and described to provide the best illustration of the principles of the invention and its practical application to thereby enable one of ordinary skill in the art to utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated. All such modifications and variations are within the scope of the invention as determined by the appended claims when interpreted in accordance with the breadth to which they are fairly, legally, and equitably entitled.
Claims
1.An apparatus for valid entry detection, the apparatus comprising:a body comprising:a container attachment portion configured to attach to a corresponding attachment area of a container, the container attachment portion comprising an inner surface configured to engage with a corresponding surface of the attachment area to establish a secure attachment to the attachment area of the container;a handheld device holder portion configured to receive and hold an electronic handheld device, the handheld device holder portion comprising a wall having an inner surface defining a volume for receipt of the electronic handheld device, the inner surface comprising a holder electrical connector configured to physically and electrically connect to a corresponding device electrical connector upon receipt of the electronic handheld device within the volume; anda camera disposed within the body, the camera oriented relative to the container attachment portion such that when the container attachment portion establishes the secure attachment to the attachment area of the container, the camera is configured to establish an image capture region adjacent to an opening of the container.2.The apparatus of claim 1, further comprising:a holder battery configured to be charged by a battery of the handheld device via the holder electrical connector connected to the device electrical connector.3.The apparatus of any of claim 1 or 2, further comprising:a circuit configured to establish a network connection with the electronic handheld device.4.The apparatus of claim 3, wherein the network connection comprises a wireless network connection.5.The apparatus of any of claims 1 to 4, wherein the camera is configured to take at least one initial image of the image capture area in response to a scanning event performed by the electronic handheld device.6.The apparatus of claim 5, wherein the camera is configured to transition from a sleep mode to an active mode to take the at least one initial image in response to the scanning event.7.The apparatus of claim 6, wherein the camera is configured to transition from the active mode to the sleep mode after taking the at least one initial image.8.The apparatus of any of claims 5 to 7, further comprising:a circuit configured to:establish a network connection with electronic the handheld device; anddetect the scanning event performed by the electronic handheld device via the network connection; andtrigger the camera to take the at least one initial image of the image capture area in response to detection of the scanning event.9.The apparatus of any of claims 1 to 8, further comprising:a sensor disposed within the body and operatively coupled to the camera, wherein the sensor is configured to:sense a presence of an object in the image capture area; andin response to sensing the presence of the object, trigger the camera to take at least one image of the image capture area including the object.10.The apparatus of any of claims 1 to 9, wherein when the secure connection is established, the image capture area comprises a lower boundary that is substantially parallel with the opening of the container.11.The apparatus of any of claims 1 to 10, wherein the container attachment portion comprises a clamp.12.The apparatus of any of claims 1 to 11, wherein the container attachment portion is configured to relatively move about the attachment area to vertically adjust the image capture region.13.The apparatus of any of claims 1 to 12, wherein the camera is configured to adjust an angle of view to horizontally adjust the image capture region.14.The apparatus of any of claims 1 to 13, wherein the holder electrical connector is removably coupled to the handheld device holder portion.15.The apparatus of any of claims 1 to 14, further comprising a through hole extending through the wall of the handheld device holder portion, the through hole configured to drain liquid in the volume.16.A system comprising:an electronic handheld device comprising: a display, a scanner, and a handle; anda valid entry detection apparatus for a container, the valid entry detection apparatus comprising:a body comprising:a container attachment portion configured to attach to an attachment area of the container; anda handheld device holder portion configured to removably receive and hold the handle of the electronic handheld device; anda camera disposed in the body, the camera configured to establish an image capture region adjacent to an opening of the container when the valid entry detection apparatus is securely attached to the attachment area of the container.17.The system of claim 16, wherein the camera is configured to capture at least one image of the image capture area in response to the scanner performing a scanning event on an object.18.The system of claim 17, wherein the electronic handheld device and the valid entry detection apparatus are configured to establish a network connection with each other, and wherein the electronic handheld device is configured to communicate the scanning event to the valid entry detection apparatus via the network connection.19.The system of claim 18, wherein the network connection comprises a wireless network connection.20.The system of any of claim 18 or 19, wherein the electronic handheld device is configured to communicate the scanning event via the network connection when the handle is outside of the handheld device holder portion of the valid entry detection apparatus.21.The system of any of claims 16 to 20, wherein the valid entry detection apparatus further comprises a sensor configured to sense a presence of an object in the image capture area,wherein the camera is configured to capture at least one image of the image capture area with the object in response to detection of the presence of the object by the sensor, andwherein the display of the electronic handheld device is configured to display a valid entry status of the object based on the at least one image of the image capture with the object and a scanning event performed by the scanner of the electronic handheld device.22.A system comprising:a container configured to hold one or more items, the container comprising an attachment area; anda valid entry detection apparatus comprising:a body comprising:a container attachment portion attached to the attachment area of the container; anda handheld device holder portion configured to removably receive and hold an electronic handheld device; anda camera disposed in the body, the camera configured to establish an image capture region adjacent to an opening of the container when the valid entry detection apparatus is attached to the attachment area of the container.23.The system of claim 22, wherein the container comprises a shopping cart.24.The system of claim 23, wherein the attachment area comprises a handlebar of the shopping cart.25.The system of claim 24, wherein the container attachment portion comprises a clamp that is clamped to the handlebar, the clamp being rotatable about the handlebar to adjust a vertical position of the image capture region.26.The system of any of claims 22 to 25, wherein a lower boundary of the image capture region is substantially parallel with an opening of the container.27.The system of any of claims 22 to 26, wherein the camera is configured to adjust a horizontal angle of view relative to the opening of the container.28.The system of any of claims 22 to 27, further comprising:a scale disposed at a base of the container, the scale configured to measure a weight of one or more objects in the container.29.The system of claim 28, wherein the scale and the valid entry detection apparatus form a network connection with each other.30.The system of claim 29, wherein the network connection comprises a Bluetooth connection.31.A method of configuring a container for valid entry status detection, the method comprising:attaching a valid detection status apparatus to an attachment area of a container, wherein the valid entry status apparatus comprises a camera; andestablishing an image capture region adjacent to an opening of the container.32.The method of claim 31, wherein the container comprises a shopping cart, and the image capture area is above an opening of the shopping cart.33.The method of claim 32, wherein the attachment area comprises a handlebar of the shopping cart.34.The method of any of claims 31 to 33, wherein attaching the valid entry status apparatus to the attachment area comprises clamping the valid entry status apparatus to the attachment area.35.The method of any of claims 31 to 34, wherein establishing the image capture region comprises adjusting a rotatable position of the valid entry status apparatus about the attachment area until a lower boundary of the image capture region is substantially parallel with the opening of the container.36.The method of any of claims 31 to 35, wherein establishing the image capture region comprises adjusting a horizontal angle of view of the camera relative to the opening of the container.37.The method of any of claims 31 to 36, further comprising: positioning a scale with a base of a basket of the shopping cart.38.A container valid entry detection method, the method comprising:sensing, by a sensor, a presence of an object in an image capture region adjacent to an opening of a container;capturing, by a camera, at least one image of the object in the image capture region in response to the sensor sensing the presence of the object in the image capture region as the object is moving to enter into the container;detecting, by a valid entry detection controller, a valid entry status of the object in the container based on the at least one image and a scanning event;outputting, by an output device, the valid entry status of the object.39.The method of claim 38, wherein the object comprises a first object, the method further comprising:performing, by a scanner of the electronic handheld device, the scanning event on a second object before the first object is sensed by the sensor;in response to the scanning event, obtaining, by the valid entry detection controller, a representative image data set associated with the second object from a database; andcomparing, by the valid entry detection controller, the at least one image captured by the camera with a representative image data set associated with the second object that is scanned,wherein detecting the valid entry status of the first object is based on the comparing.40.The method of claim 39, wherein detecting the valid entry status of the first object comprises detecting that the valid entry status is valid in response to the comparison indicating that the first object in the at least one image captured by the camera sufficiently matches a representative object of the representative image data set.41.The method of claim 40, wherein the first object that is sensed by the sensor is the same as the second object that is scanned by the scanner.42.The method of any of claims 39 to 41, wherein detecting the valid entry status of the first object comprises detecting that the valid entry status is invalid in response to the comparison indicating that the first object in the at least one image captured by the camera insufficiently matches a representative object of the representative image data set.43.The method of claim 42, wherein the first object that is sensed by the sensor is different than the second object that is scanned by the scanner.44.The method of any of claims 39 to 43, wherein the representative image data set comprises at least one representative image associated with the second object, and wherein the comparing comprises comparing the at least one image captured by the camera with the at least one representative image.45.The method of any of claims 39 to 44, wherein the representative image data set comprises a representative feature identification (ID) derived from one or more features of at least one representative image associated with the second object, and wherein the comparing comprises comparing a feature ID code derived from one or more features of the first object in the at least one image captured by the camera with the representative feature ID.46.The method of any of claims 39 to 45, further comprising:in response to the comparing, determining a degree of similarity between the first object in the at least one image captured by the camera and at least one representative object associated with the representative image data set.47.The method of claim 46, further comprising:determining whether the degree of similarity satisfies a threshold criteria,wherein detecting the valid entry status is based on whether the degree of similarity satisfies the threshold criteria.48.The method of any of claims 39 to 47, further comprising:in response to the scanning event, receiving, by the valid entry controller, an object identification (ID) of the second object derived from performing the scanning event on the second object,wherein obtaining the representative image data set comprises obtaining the representative image data set in response to the representative image data set being linked to the object ID.49.The method of any of claims 39 to 48, further comprising:capturing, by the camera, at least one initial image in response to the scanner performing the scanning event before the camera captures the at least one image of the first object in response to the sensor sensing the object in the image capture region,wherein the set of images further comprises the at least one initial image.50.The method of any of claims 38 to 49, further comprising:receiving, by a valid entry detection controller, an indication that the object is in the sensing region;in response to receiving the indication, determining, by the valid entry detection controller, that no scanning event has occurred within a predetermined time period before receiving the indication,wherein detecting the valid entry status of the object comprises detecting that the valid entry status is invalid in response to determining that the scanning event has not occurred within the predetermined time period before receiving the indication.51.The method of any of claims 38 to 50, wherein the output device comprises a display, and wherein outputting the valid entry status comprises displaying the valid entry status of the object on the display.52.The method of any of claims 38 to 51, wherein the output device comprises a light source, and wherein outputting the valid entry status comprises emitting a light from the light source to indicate the valid entry status.53.The method of any of claims 38 to 52, wherein the output device comprises a speaker, and wherein outputting the valid entry status comprises outputting a sound from the speaker to indicate the valid entry status.54.A system comprising:a memory storing a plurality of instructions; anda processor configured to execute the plurality of instructions, and upon execution of the plurality of instructions, is configured to:receive a captured image data set of a first object in an image capture region adjacent to an opening of a container;receive a representative image data set associated with a second object in response to a scanning event performed by a scanner on the second object;compare the captured image data set with the representative image data set to determine a degree of similarity between the first object and the second object;determine a valid entry status of the second object in a container based on whether the degree of similarity satisfies a threshold criteria; andoutput the valid entry status to an output device.55.The system of claim 54, wherein the captured image data set comprises at least one captured image of the first object in the capture region, wherein the representative image data set comprises at least one representative image associated with the second object, and wherein in order to compare the captured image data set with the representative image data set, the processor is configured to compare the at least one captured image with the at least one representative image.56.The system of any of claim 54 or 55, wherein the captured image data set comprises a feature identification (ID) of the first object, wherein the representative image data set comprises a feature ID associated with the second object, and wherein in order to compare the captured image data set with the representative image data set, the processor is configured to compare the feature ID of the first object with the feature ID associated with the second object.57.The system of claim 56, wherein the processor, upon execution of the plurality of instructions, is further configured to:analyze at least one image in the image capture region captured by a camera to determine one or more features of the first object; andderive the feature ID of the first object based on the one or more features.58.The system of any of claims 54 to 57, wherein the memory further stores:a database comprising a plurality of entries, wherein each of the plurality entries links an object identification (ID) of a respective representative object with a respective representative data set.59.The system of claim 58, wherein the processor, upon execution of the plurality of instructions, is further configured to:identify an object ID of the second object from the scanning event;identify the representative image data set by identifying that the representative image data set is linked to the object ID of the second object in the database.60.An electronic handheld device comprising:a display;an electronic scanner;a housing comprising:a display portion holding the display;a scanner portion holding the electronic scanner;an elongate handle portion extending from a first end to a second end, wherein the first end is connected to the display portion, and the second end is opposite the first end;a device electrical connector integrated with the elongate handle portion, the electrical connector comprising at least one conductive contact exposed from within the housing,wherein the electrical connector is removably coupled to the housing via at least one coupling member.61.The electronic handheld device of claim 60, wherein the electrical connector further comprises:a support on which the at least one conductive contact is mounted,wherein the support is removably disposed within a receptacle of the housing within the elongate handle portion at the second end.62.The electronic handheld device of claim 61, wherein the at least one coupling member secures the support to an inner surface of the housing when the support is in the receptacle.63.The electronic handheld device of any of claims 60 to 62, wherein the electrical connector further comprises:at least one conductor electrically connected to the at least one conductive contact and electrically connected to a circuit board disposed within the housing,wherein the at least one conductor is removably coupled to the circuit board via a wire-to-board connector of the least one coupling member.64.The electronic handheld device of claim 63, further comprising a battery disposed within the elongate handle portion of the housing,wherein the at least one conductive contact is electrically connected to the battery via the at least one conductor and the circuit board.65.The electronic handheld device of claim 64, wherein the at least one conductor extends from a support of the least one contact and alongside the battery to the circuit board.66.The electronic handheld device of any of claims 60 to 65, wherein the housing further comprises a removable cover that forms a portion of the elongate handle portion, and wherein the at least one coupling member comprises the cover.67.The electronic handheld device of any of claims 60 to 66, further comprising:a proximity sensor configured to:detect an object within a predetermined range of the proximity sensor; andtrigger the electronic scanner to scan the object in response to detection of the object.68.The electronic handheld device of any of claims 60 to 67, further comprising an radio frequency (RF) identification (ID) tag.
Citation Information
Patent Citations
Handheld communication shopping device
CN104299338A
Novel handheld laser reading device based on OCR recognition technology
CN107169391A
RFID-based smart shopping cart
CN108537997A
Shopping cart and shopping system and method
CN109741552A
Intelligent shopping cart, shopping assistance method and device, equipment and readable storage medium
CN110148042A