Wall-mounted self-checkout system

Abstract

A wall-mounted self-checkout system will be provided. [Solution] The wall-mounted self-checkout system also includes a computing system configured to perform an operation that includes determining, at least partially, that the tilt angle of the wall-mounted self-checkout system has reached a predetermined tilt threshold, or that the inclination angle of the wall-mounted self-checkout system has reached a predetermined tilt threshold, and performing a control action in response to the determination that the tilt angle has reached a predetermined tilt threshold, or that the inclination angle has reached a predetermined tilt threshold.

Description

[Background technology]

[0001]

[0001] Many retailers offer customers the option to purchase goods at self-service kiosks. Self-service kiosks have become desirable for both customers and retailers. For customers, kiosks reduce waiting times compared to using checkout lanes. Retailers can benefit from improved checkout efficiency. During a checkout transaction, customers can scan the product barcode of each product and place those products on the platform for weighing and / or monitoring during the transaction. A display screen can provide customers with useful information, such as the cost of the scanned items, whether the items are on sale or discounted, and the weight of the items. In some cases, kiosks may be affected by events that may prevent them from operating optimally. [Brief explanation of the drawing]

[0002] [Figure 1]

[0002] A perspective view of a wall-mounted self-checkout system according to one or more embodiments of the present disclosure. [Figure 2]

[0003] Figure 1 is a front view of the wall-mounted self-checkout system, showing the system oriented without tilt or inclination. [Figure 3]

[0004] Schematic diagram of the wall-mounted self-checkout system shown in Figure 1. [Figure 4]

[0005] A flowchart for implementing an operation to detect and address tilt and / or tilt events associated with a wall-mounted self-checkout system, according to one or more embodiments of the present disclosure. [Figure 5]

[0006] Figure 1 is a front view of the wall-mounted self-checkout system, showing the system in a tilted position. [Figure 6]

[0007] FIG. 1 is a side view of a wall-mounted self-checkout system, showing the wall-mounted self-checkout system oriented with or without tilt or inclination. [Figure 7]

[0008] FIG. 1 is a side view of a wall-mounted self-checkout system, showing the wall-mounted self-checkout system in a tilted state. [Figure 8]

[0009] A flowchart for implementing operations for detecting and handling tilt events associated with shelves of a wall-mounted self-checkout system according to one or more aspects of the present disclosure. [Figure 9]

[0010] FIG. 1 is a front view of a wall-mounted self-checkout system, showing the input shelf of the wall-mounted self-checkout system in a tilted state. [Figure 10]

[0011] FIG. 1 is a front view of a wall-mounted self-checkout system, showing the output shelf of the wall-mounted self-checkout system in a tilted state.

DETAILED DESCRIPTION OF THE INVENTION

[0003]

[0012] A wall-mounted self-checkout system is capable of being attached to a wall, for example, being cantilevered from the wall. In some cases, the wall-mounted self-checkout system may be affected by significant forces. For example, a shopping cart full of groceries may hit the unit and apply a relatively large force to the unit. As another example, a person may jump or sit on the shelf of the wall-mounted self-checkout system. Such wall-mounted self-checkout systems are designed and implemented to withstand significant forces, but in some cases, the wall-mounted self-checkout system may be tilted or inclined. The tilting and / or inclination of the wall-mounted self-checkout system may affect the sensors and measurement units of the system and potentially present safety issues.

[0004]

[0013] The wall-mounted self-checkout systems disclosed herein may include features for detecting and addressing such tilt and / or tilt events. In one or more examples, the wall-mounted self-checkout system may include a housing, a counter having a horizontal panel and a vertical panel connected thereto, a shelf mounted on the vertical panel, a tower extending upward from the counter and having one or more mounts for mounting the wall-mounted self-checkout system to a wall, and a display screen mounted on the tower. The wall-mounted self-checkout system may also include a computing system calibrated to perform an operation that includes an operation for detecting and addressing tilt and tilt events. In at least one example, the operation may include determining that the tilt angle of the wall-mounted self-checkout system has reached a predetermined tilt threshold and / or that the tilt angle of the wall-mounted self-checkout system has reached a predetermined tilt threshold. Captured image data, sensor data, etc., may be used to determine the tilt and / or tilt angle of the wall-mounted self-checkout system. The operation may further include taking a control action in response to the determination that the tilt angle has reached a predetermined tilt threshold and / or that the tilt angle has reached a predetermined tilt threshold. Exemplary control actions may include, but are not limited to, shutting down lanes using lane blockers, communicating tilt and / or tilt events to an operator (e.g., a salesperson), changing the mode of a wall-mounted self-checkout system from normal mode to standby mode or shutdown mode, and automatically performing a recalibration process for one or more load cells. Thus, such a wall-mounted self-checkout system may be configured to detect and address tilt and / or tilt events.

[0005]

[0014] In some further embodiments, tilt and / or tilt events can be monitored and addressed for one or more components of a wall-mounted self-checkout system, such as an input shelf that can be cantilevered from the rest of the unit.

[0006]

[0015] As used herein, “angle of inclination,” “inclination,” or “inclination” of an object is measured relative to the angle or orientation of the wall-mounted self-checkout system with respect to a horizontal reference plane perpendicular to the wall on which the wall-mounted self-checkout system is installed. “Angle of inclination,” “inclination,” or “inclination” of an object refers to the angle or orientation of the wall-mounted self-checkout system with respect to a vertical reference plane parallel to or coplanar with the wall on which the wall-mounted self-checkout system is installed.

[0007]

[0016] Referring here to Figures 1 and 2, Figure 1 shows a perspective view of a wall-mounted self-checkout system 100 according to one or more embodiments of the present disclosure. Figure 2 shows a front view of the wall-mounted self-checkout system 100. The wall-mounted self-checkout system 100 may also be referred to as a self-service kiosk or checkout terminal. For reference, the wall-mounted self-checkout system 100 defines the X, Y, and Z directions, which are perpendicular to each other. In one or more examples, the X direction is longitudinal, the Y direction is transverse, and the Z direction is vertical. The wall-mounted self-checkout system 100 may be mounted on a wall 101. In Figure 1, the wall 101 extends in a plane perpendicular to the X direction, or in other words, in the YZ plane. In one or more examples, the wall-mounted self-checkout system 100 does not physically touch the ground.

[0008]

[0017] The wall-mounted self-checkout system 100 has a front side 102 and a rear side 104, a first side 106 and a second side 108, and an upper side 110 and a lower side 112. The wall-mounted self-checkout system 100 includes a housing 114. The housing 114 has a base 116 and a plurality of side walls 118 extending upward from the base 116. The housing 114 defines an interior where various components, such as a product scanner 120 and a printer, can be arranged. The housing 114 includes a printer drawer 122 that is movable between a retracted position (shown in Figure 1) and an extended position. In the extended position, the printer drawer 122 is moved to open, for example along the X direction, to allow access to the printer engine of the printer, for example, to replenish paper. The printer drawer 122 has a receipt dispenser 124 which can eject printed receipts and, in at least one example, can be illuminated to inform the user that a receipt has been printed.

[0009]

[0018] The wall-mounted self-checkout system 100 also includes a counter 126 having a horizontal panel 128 and a vertical panel 130 connected to the horizontal panel 128 by, for example, a curved transition section. In this respect, the counter 126 has a “waterfall” configuration. In the example shown in Figure 1, the horizontal panel 128 is positioned in the XY plane, while the vertical panel 130 is positioned in the YZ plane. The horizontal panel 128 is mounted on a housing 114, and the vertical panel 130 surrounds the sides of the housing 114. The horizontal panel 128 includes a plurality of guide lights 132, which can be illuminated to guide the user to place items in the “purchase zone” so that the items can be scanned by an item scanner 120, which may be located within the housing 114, as described above. In one or more examples, the item scanner 120 may include one or more load cells 134 (Figure 3) to measure the weight of items placed in the purchase zone, for example.

[0010]

[0019] In one or more examples, the input shelf 136 is cantilevered from the vertical panel 130, for example, as shown in Figure 1 (see also Figure 2). The input shelf 136 may be attached to the vertical panel 130, for example, by tabs and one or more fasteners. The tabs of the input shelf 136 can be inserted into recesses in the vertical panel 130 and bolted to the recesses or to the housing 114, such as inside the housing 114. In addition, or instead, the input shelf 136 may be attached to the vertical panel 130 by other adhesive means, for example, by adhesive. In yet another example, the input shelf 136 may be formed integrally with the counter 126, so that the counter 126 and the input shelf 136 constitute a single monolithic component. Such a component may be formed by additive manufacturing techniques such as 3D printing. The input shelf 136 can provide a place for the user to place items before registering or scanning them for purchase. In at least one example, the vertical panel 130 is positioned at least partially below the bottom surface of the input shelf 136 (see Figure 2). In at least one example, the input shelf 136 is positioned at the same height as the base 116, for example, along the Z direction (see Figure 2).

[0011]

[0020] In addition to the input shelf 136, at least a portion of the horizontal panel 128 is cantilevered from the rest of the horizontal panel 128 to form the output shelf 138. The output shelf 138 can provide a place for the user to bag or set goods after purchase. As shown in Figure 1 (see also Figure 2), the housing 114 does not extend below the output shelf 138.

[0012]

[0021] The wall-mounted self-checkout system 100 further includes a tower 140 extending upward from the counter 126. The tower 140 may be mounted on the counter 126, on the housing 114, or a combination thereof. The tower 140 has one or more mounts 142 (Figures 6 and 7) for mounting the wall-mounted self-checkout system 100 to the wall 101. The mounts 142 may include hooks, rods, fasteners, mounting plates, and / or other mounting devices. A display screen 144 is mounted on the tower 140. The display screen 144 can present the user with useful information, such as the cost of scanned items, whether items are on sale or discounted, and the weight of items. The tower 140 may also include lane lights 146 positioned above it. The lane lights 146 may be controlled to indicate the status of the wall-mounted self-checkout system 100, for example, green when ready / open, red when closed, yellow when in use, etc. Camera 148 is mounted on the tower 140, for example, below the display screen 144. In at least one example, camera 148 can capture images of the wall-mounted self-checkout system 100, images of goods, such as within or near the purchase zone. Camera 148 may also be used to capture images of users present at the wall-mounted self-checkout system 100, for example, to capture biometric data. A payment terminal 150 may be mounted on the side wall of the tower 140. The payment terminal 150 may include a display, keypad, card reader, near-field communication (NFC) beacon, etc., to facilitate payment processing during a transaction.

[0013]

[0022] Figure 3 is a schematic diagram of a wall-mounted self-checkout system 100. As illustrated in Figure 3, the wall-mounted self-checkout system 100 may include a computing system 160. The computing system 160 may include one or more computing devices, such as computing device 161. Computing device 161 may include one or more processors 162 and one or more memory devices 163 that store one or more programs 164, which, when executed by any combination of one or more processors 162, cause one or more processors 162 to perform actions including actions to detect and respond to tilt and / or tilt events. One or more memory devices 163 may also store data 165. The data 165 may, among other things, include a library 166. The library 166 may include baseline images 167.

[0014]

[0023] In some examples, the library 166 may be stored locally on the computing device 161, for example, on one or more non-temporary memory devices 163 of the computing device 161. In other embodiments, the library 166 may be stored outside the wall-mounted self-checkout system 100, for example, on a data store 170 as shown in Figure 3. The computing device 161 can access the library 166 via a network 172, such as the Internet. The computing device 161 may include a communication interface 168 that enables communication with devices via the network 172 and also local communication with other devices of the wall-mounted self-checkout system 100 via a communication bus 169. The communication interface 168 may include a transmitter circuit configured to send communication signals and a receiver circuit configured to receive communication signals. In this regard, the communication interface 168 may include transmitters, receivers, transceivers, etc., for communicating via the network 172 and / or the communication bus 169. In yet another embodiment, the library 166 may be stored partially locally and partially remotely.

[0015]

[0024] The computing device 161 is communicatively coupled to other devices / components of the wall-mounted self-checkout system 100 via a communication bus 169, for example, by one or more wired and / or wireless communication links. As shown in Figure 3, the computing device 161 may be communicatively coupled to the product scanner 120, the guidance light 132, the display screen 144, the payment terminal 150, the lane light 146, the camera 148, the load cell 134, the printer, the accelerometer 174 (which may be embedded in the display screen 144), the accelerometer 180 (which may be embedded in the shelf of the wall-mounted self-checkout system), and the lane blocker 176. The computing device 161 can be located in any suitable position, such as behind the display screen 144. The computing device 161 may also be communicatively coupled to other devices, such as one or more speakers, a user input device, other light sources, and off-board devices (e.g., off-board cameras, computing devices, sensors, etc.).

[0016] Detection of tilt and / or tilt events

[0025] In one or more examples, the wall-mounted self-checkout system 100 may be configured to detect and respond to tilt and / or tilt events. Referring here to Figures 1 to 7, the operation for detecting and responding to tilt and / or tilt events may be implemented by the wall-mounted self-checkout system 100 according to process 200 shown in the flowchart of Figure 4. In at least one example, a computing system 160 may implement the operation at least partially. As described above, the computing system 160 may include one or more processors and one or more memory devices for storing a program, which, when executed, causes one or more processors to perform an operation, for example, an operation for detecting and responding to tilt and / or tilt events associated with the wall-mounted self-checkout system 100, individually or collectively.

[0017]

[0026] At 202, when operating, one or more processors can determine at least one of that the tilt angle of the wall-mounted self-checkout system 100 has reached a predetermined tilt threshold or that the inclination angle of the wall-mounted self-checkout system has reached a predetermined inclination threshold, based at least in part on the received data.

[0018]

[0027] In one or more examples, as shown in FIG. 5, the wall-mounted self-checkout system 100 experiences a tilt event when it is angled with respect to a horizontal reference plane RP perpendicular to the wall 101 to which the wall-mounted self-checkout system 100 is attached. The horizontal reference plane RP H extends in a plane perpendicular to the Z direction or rather, in the XY plane. As shown in the example of FIG. 5, the wall-mounted self-checkout system 100 is tilted, and as a result, has a non-zero tilt angle θ H with respect to the horizontal reference plane RP. T In comparison, in FIG. 2, the wall-mounted self-checkout system 100 is not tilted and thus has a tilt angle θ T of 0 degrees (0°). A predetermined tilt threshold T T can be set to a non-zero tilt angle, for example 3 degrees (3°). In FIG. 5, the tilt threshold T T is represented by a dashed line. Thus, at 202, one or more processors can determine that the tilt angle θ T of the wall-mounted self-checkout system 100 has reached the predetermined tilt threshold T T when the tilt angle θ T of the wall-mounted self-checkout system 100 exceeds the predetermined tilt threshold T T based at least in part on the received data (e.g., image data, accelerometer data, etc., as further explained below).

[0019]

[0028] In one or more examples, as shown in Figure 7, the wall-mounted self-checkout system 100 is positioned on a vertical reference plane RP parallel to the wall 101 on which the wall-mounted self-checkout system 100 is installed. V If an angle is applied to it, a tilt event may occur. Vertical reference plane RP V It extends in a plane perpendicular to the X direction, or rather, in the YZ plane. As shown in the example in Figure 7, the wall-mounted self-checkout system 100 is inclined, and as a result, the vertical reference plane RP V Regarding the non-zero tilt angle θ S In comparison, in Figure 6, the wall-mounted self-checkout system 100 is not inclined (or tilted), and therefore has an inclination angle θ of 0 degrees (0°). S It has a predetermined gradient threshold T. S The slope threshold T can be set to a non-zero slope angle, for example, 3 degrees (3°). In Figure 7, the slope threshold T S This is represented by a dashed line. Therefore, in 202, one or more processors determine the tilt angle θ of the wall-mounted self-checkout system 100, at least in part, based on the received data (e.g., image data, accelerometer data, etc.). S The predetermined integral threshold T S Since it exceeds the inclination angle θ of the wall-mounted self-checkout system 100 S The predetermined integral threshold T S It can be determined that it has reached that point.

[0020]

[0029] In one or more examples, the received data used to determine whether the tilt angle has reached a predetermined tilt threshold and / or whether the inclination angle has reached a predetermined tilt threshold may be image data captured by one or more cameras of the wall-mounted self-checkout system 100, such as camera 148. In such examples, when determining, at least in part, that the tilt angle has reached a predetermined tilt threshold and / or that the inclination angle has reached a predetermined tilt threshold, one or more processors may receive a current image 178 (Figure 3) of the wall-mounted self-checkout system 100 captured by one or more cameras (e.g., by camera 148). The current image 178 may be included in the data received by one or more processors. Furthermore, one or more processors may receive a baseline image 167 (Figure 3) of the wall-mounted self-checkout system 100 captured by one or more cameras (e.g., by camera 148). The baseline image 167 may be accessed from library 166. The baseline image 167 is the reference tilt angle or horizontal reference plane RP of the wall-mounted self-checkout system 100. H When it is known that the tilt angle of the wall-mounted self-checkout system 100 is within a predetermined range (for example, at a tilt angle of 0 degrees (0°)), and the tilt angle of the wall-mounted self-checkout system 100 is within a reference tilt angle or vertical reference plane RP V It can be captured if it is known to be within a predetermined range (for example, at an inclination angle of zero degrees (0°)). The baseline image 167 may be included in the data received by one or more processors.

[0021]

[0030] One or more processors can determine at least one of the tilt angle by comparing the current image 178 with the baseline image 167, or the inclination angle by comparing the current image 178 with the baseline image 167. When performing an image comparison, which may be implemented by running one or more computer vision algorithms or machine learning models (e.g., convolutional neural networks (CNNs)), one or more processors can detect or determine the tilt and / or inclination angle of the wall-mounted self-checkout system 100. Thus, with the tilt angle and / or inclination angle determined, one or more processors can determine whether the tilt angle has reached a predetermined tilt threshold and / or whether the inclination angle has reached a predetermined inclination threshold.

[0022]

[0031] In one or more examples, the display screen 144 has an accelerometer 174 attached to or embedded in the display screen 144 as described above. In such examples, when determining, at least in part, based on received data, that the tilt angle has reached a predetermined tilt threshold and / or the inclination angle has reached a predetermined tilt threshold, one or more processors can receive inputs from the accelerometer 174 indicating the tilt angle and / or the inclination angle. This input may be included in the data received by one or more processors. Thus, with the tilt angle and / or the inclination angle received from the accelerometer 174, one or more processors can determine whether the tilt angle has reached a predetermined tilt threshold and / or the inclination angle has reached a predetermined tilt threshold.

[0023]

[0032] In at least one example, data from the accelerometer 174 and data received from the camera 148 and library 166 may be used to determine whether the tilt angle and / or inclination angle have reached their respective thresholds. In other examples, the tilt angle and / or inclination angle may be determined by other suitable means, such as an offboard camera (comparison of current to baseline), a gradient sensor, tilt sensor, and / or inclination sensor mounted on the wall-mounted self-checkout system 100.

[0024]

[0033] In at least one example, the operation in 202 may be initiated periodically (for example, based on a predetermined time interval) or based on the fulfillment of a trigger condition.

[0025]

[0034] In one example, determining that the tilt angle has reached a predetermined tilt threshold and / or that the inclination angle has reached a predetermined tilt threshold may be done in response to a force applied to the wall-mounted self-checkout system 100 reaching a force threshold. The applied force may be measured by an accelerometer 174 mounted on or embedded in the display screen 144. For example, if a shopping cart hits the wall-mounted self-checkout system 100, the accelerometer 174 can measure the applied force, and if the applied force reaches a force threshold (e.g., equal to or exceeding the force threshold), an action may be initiated to check the tilt or inclination of the wall-mounted self-checkout system 100.

[0026]

[0035] In another example, determining in 202 that the tilt angle has reached a predetermined tilt threshold and / or that the inclination angle has reached a predetermined inclination threshold can be done in response to a force applied to the wall-mounted self-checkout system 100 reaching a force threshold, the applied force being measured by one or more load cells 134 of the product scanner 120 attached to or embedded in the counter 126. For example, if a relatively heavy object is placed on the product scanner 120 or its platform and the applied force reaches a force threshold (e.g., equal to or exceeding the force threshold), the operation in 202 may be initiated to check the tilt or inclination of the wall-mounted self-checkout system 100.

[0027]

[0036] In a further example, determining that the tilt angle has reached a predetermined tilt threshold and / or that the tilt angle has reached a predetermined tilt threshold may be done in response to one or more cameras capturing a predefined user gesture. Exemplary predefined user gestures may include, but are not limited to, a user sitting on a shelf of the wall-mounted self-checkout system 100, a user slamming goods against a shelf or counter 126, or a user bumping into a shelf with a shopping cart. Camera 148 may capture such user gestures. In addition, or alternatively, one or more off-board cameras may capture such user gestures and report them to the wall-mounted self-checkout system 100.

[0028]

[0037] In further examples, the determination that the inclination angle has reached a predetermined inclination threshold, and / or that the inclination angle has reached a predetermined inclination threshold, may be made after each user transaction.

[0029]

[0038] In 204, when performing an operation, one or more processors may take control actions in response to a determination that the tilt angle has reached a predetermined tilt threshold and / or that the tilt angle has reached a predetermined tilt threshold.

[0030]

[0039] In one or more examples, the control action may include physically blocking access to the wall-mounted self-checkout system 100. In at least one example, the control action may include moving a lane blocker 176 to physically close the wall-mounted self-checkout system 100. The lane blocker 176 may be coupled, for example, to an input shelf 136 and may be movable between a retracted position (shown in Figure 2) and an extended position where the lane blocker 176 physically closes the wall-mounted self-checkout system 100. In at least one example, the lane blocker 176 may include a pole or bar coupled to an actuator (e.g., an electrically controlled actuator). The actuator may be controlled to selectively move the pole or bar, for example, outward from the input shelf 136 along the X direction and inward toward the input shelf 136 along the X direction. In this regard, the lane blocker 176, or its pole or bar, can be movable between a retracted position and an extended position in which the lane blocker 176 physically closes the wall-mounted self-checkout system 100. In the retracted position, the lane blocker 176 can allow physical access to the wall-mounted self-checkout system 100 and can be retracted so as to be hidden from view. For example, the pole or bar may be retracted into a recess defined by the input shelf 136 (Figure 2), or it may be located below the input shelf 136.

[0031]

[0040] If tilting and / or tilting events occur, one or more components of the wall-mounted self-checkout system 100 may be damaged or otherwise unsafe for the user. Therefore, the lane blocker 176 may be selectively deployed to prevent or force users not to move near the wall-mounted self-checkout system 100. In other examples, the lane blocker 176 may be located at a distance, for example, at the entrance to the lane or waiting area associated with the wall-mounted self-checkout system 100. In yet another example, in addition to or instead of the lane blocker coupled to the input shelf 136, the wall-mounted self-checkout system 100 may include a lane blocker coupled to the output shelf 138.

[0032]

[0041] In some cases, the lane blocker 176 may be deployed, for example, when it is confirmed that a user is not in the wall-mounted self-checkout system 100, to prevent the lane blocker 176 from hitting a user. In at least one example, the computing system 160 may "hold" the movement of the lane blocker 176 until the user has left the area. In at least one example, the speaker of the wall-mounted self-checkout system 100 may generate a sound to warn the user that the lane blocker 176 is being deployed or is about to be deployed, thereby enhancing safety. In at least one example, the lane blocker 176 may be deployed based on the magnitude of the deviation of the tilt angle to a given tilt threshold and / or based on the magnitude of the deviation of the tilt angle to a given inclination angle. For example, the lane blocker 176 may be deployed when it is determined that the tilt angle and / or inclination angle have reached a relatively extreme level that exceeds the aforementioned threshold.

[0033]

[0042] In one or more examples, in addition to or instead of any of the control actions described above or below, a control action may include switching the wall-mounted self-checkout system 100 from normal mode to some other mode, such as shutdown mode or standby mode. In at least one example, performing a control action may include changing the wall-mounted self-checkout system 100 from normal mode to standby mode, in which the functionality of the wall-mounted self-checkout system 100 is reduced compared to normal mode, but it is still operational. For example, in standby mode, the wall-mounted self-checkout system 100 may allow the user to continue scanning items for purchase, but may withhold allowing the user to complete the payment in order to keep the user at the wall-mounted self-checkout system 100, for example, so that an operator can inspect the wall-mounted self-checkout system 100 before the user leaves. In at least one example, performing a control action may include changing the wall-mounted self-checkout system 100 from normal mode to shutdown mode, in which the functionality of the wall-mounted self-checkout system 100 is turned off.

[0034]

[0043] In at least one example, the wall-mounted self-checkout system 100 switches from normal mode to either standby mode or shutdown mode based on the magnitude of the deviation of the tilt angle and / or inclination angle from their respective thresholds. For example, if the tilt angle and / or inclination angle are within a first range, the wall-mounted self-checkout system 100 can switch from normal mode to standby mode, and if the tilt angle and / or inclination angle are within a second range, the wall-mounted self-checkout system 100 can switch from normal mode to shutdown mode. The first range may be associated with a deviation lower than the second range.

[0035]

[0044] In one or more examples, performing a control action in addition to or instead of any of the control actions described above or below may include automatically recalibrating the load cell 134 of the product scanner 120. In such an example, during the recalibration of the load cell 134, performing a control action may include changing the operating mode of the wall-mounted self-checkout system 100 to prevent the user from scanning products whose price is determined by weight. In such an example, the wall-mounted self-checkout system 100 may switch to an operating mode (e.g., normal mode) that allows the user to scan products whose price is determined by weight once the recalibration process is successfully completed.

[0036]

[0045] In one or more examples, performing a control action in addition to or instead of any of the control actions described above may include automatically communicating to the operator that the wall-mounted self-checkout system 100 is experiencing a tilt event and / or tilt event. The communication provided to the operator may be an audible communication (e.g., output from the speaker of the wall-mounted self-checkout system 100), an optical signal (e.g., the lane lights 146 and / or the guidance lights 132 may flash in a specific color and / or a predetermined frequency), a digital signal (e.g., transmitted to the operator's system), or a combination thereof. In at least one example, the operator may be a salesperson. In at least one example, the operator may be a maintenance worker. The maintenance worker may receive a communication indicating that assistance is needed to verify or secure the mounting of the wall-mounted self-checkout system 100 to the wall 101.

[0037] Shelf tilt event detection

[0046] In one or more examples, the wall-mounted self-checkout system 100 may be configured to detect and address tilt and / or damage events associated with the shelves of the wall-mounted self-checkout system 100. Referring here to Figures 1 to 3 and Figures 8 to 10, the operation for detecting and addressing tilt events associated with the shelves of the wall-mounted self-checkout system 100 is implemented by the wall-mounted self-checkout system 100 according to process 300 shown in the flowchart of Figure 8. In at least one example, a computing system 160 may implement this operation at least partially. As described above, the computing system 160 may include one or more processors and one or more memory devices for storing a program, which, when executed, causes one or more processors to perform an operation, for example, an operation for detecting and addressing tilt events associated with the shelves of the wall-mounted self-checkout system 100, individually or collectively.

[0038]

[0047] In 302, when performing an operation, one or more processors may determine, at least in part, that the tilt angle of the shelf has reached a predetermined tilt threshold based on the received data.

[0039]

[0048] In at least one example, an input shelf 136 cantilevered from a vertical panel 130 may experience a tilt event. As shown in Figure 9, the input shelf 136 is located on a horizontal reference plane RP H An angle is set with respect to the horizontal reference plane RP. H It is perpendicular to the wall 101 on which the wall-mounted self-checkout system 100 is installed. Therefore, the input shelf 136 is inclined and is relative to the horizontal reference plane RP. H Non-zero inclination angle θ T It has. In comparison, in Figure 2, the input shelf 136 is not tilted and therefore has a tilt angle θ of 0 degrees (0°). T It has a predetermined slope threshold T. TThis can be set to a non-zero slope angle, for example, 5 degrees (5°). In Figure 9, the slope threshold T T This is represented by a dashed line. Therefore, in 302, one or more processors determine the inclination angle θ of the input shelf 136 at least in part based on the received data (e.g., image data, accelerometer data, etc., as further described below). T is a predetermined slope threshold T T Since it exceeded the limit, the inclination angle θ of input shelf 136 T is a predetermined slope threshold T T It can be determined that it has reached the inclination angle θ of the input shelf 136 in at least one example. T is a predetermined slope threshold T T Reaching this point may indicate that input tray 136 is broken, damaged, or otherwise unsuitable for service.

[0040]

[0049] In at least one example, an output shelf 138, which is part of a cantilevered horizontal panel 128, may experience a tilting event. The output shelf 138 is part of a horizontal panel 128 to which the housing 114 is not located. As shown in Figure 10, the output shelf 138 is located on the horizontal reference plane RP H An angle is set with respect to the horizontal reference plane RP. H It is perpendicular to the wall 101 on which the wall-mounted self-checkout system 100 is installed. Therefore, the output shelf 138 is tilted and is relative to the horizontal reference plane RP. H Non-zero inclination angle θ T It has. In comparison, in Figure 2, the output shelf 138 is not tilted and therefore has a tilt angle θ of 0 degrees (0°). T It has a predetermined slope threshold T. T This can be set to a non-zero slope angle, for example, 5 degrees (5°). In Figure 10, the slope threshold T T This is represented by a dashed line. Therefore, in 302, one or more processors determine the tilt angle θ of the output shelf 138 at least in part based on the received data (e.g., image data, accelerometer data, etc.). T is a predetermined slope threshold T TSince it exceeded that, the tilt angle θ of output shelf 138 T is a predetermined slope threshold T T It can be determined that it has reached the inclination angle θ of the output shelf 138 in at least one example. T is a predetermined slope threshold T T Reaching this level may indicate that output shelf 138 is broken, damaged, or otherwise unsuitable for service.

[0041]

[0050] In one or more examples, the received data used to determine whether the tilt angle has reached a predetermined tilt threshold may be image data captured by one or more cameras of the wall-mounted self-checkout system 100, such as camera 148. In such an example, when determining, at least in part, that the tilt angle has reached a predetermined tilt threshold, one or more processors may receive a current image 178 (Figure 3) of the shelf captured by one or more cameras (e.g., by camera 148). The current image 178 may be included in the data received by one or more processors. Furthermore, one or more processors may receive a baseline image 167 (Figure 3) of the shelf captured by one or more cameras (e.g., by camera 148). The baseline image 167 may be accessed from library 166. The baseline image 167 is set so that the tilt angle of the shelf is the reference tilt angle or the horizontal reference plane RP H It can be captured if it is known to be within a predetermined range (for example, at an inclination angle of zero degrees (0°)). The baseline image 167 may be included in the data received by one or more processors.

[0042]

[0051] One or more processors can determine the tilt angle by comparing the current image 178 with the baseline image 167. When performing this image comparison, which can be implemented by running one or more computer vision algorithms or machine learning models (e.g., CNNs), one or more processors can detect or determine the tilt angle of the shelf. Thus, with the tilt angle determined, one or more processors can determine whether the tilt angle reaches a predetermined tilt threshold.

[0043]

[0052] In one or more examples, a shelf may have a tilt sensor attached to or embedded in the shelf. For example, in the example in Figure 9, the input shelf 136 has an accelerometer 180 attached to the input shelf 136. In such an example, determining that the tilt angle of the shelf has reached a predetermined tilt threshold, at least in part, based on the received data, may include receiving an input from the accelerometer 180 indicating the tilt angle of the shelf (e.g., the input shelf 136). This input may be included in the received data. Thus, with the tilt angle received from the accelerometer 180, one or more processors can determine whether the tilt angle has reached a predetermined tilt threshold. In another example, the output shelf 138 may have an accelerometer attached to or embedded in the output shelf 138.

[0044]

[0053] In at least one example, data from the accelerometer 180, as well as data received from the camera 148 and library 166, may be used to determine whether the tilt angle has reached a threshold. In other examples, the tilt angle may be determined by other suitable methods, such as an offboard camera (comparison of current to baseline), or other sensors mounted on the wall-mounted self-checkout system 100.

[0045]

[0054] In at least one example, the operation in 302 may be initiated periodically (for example, based on a predetermined time interval) or based on the fulfillment of a trigger condition, such as one of the trigger conditions described above with respect to 202. For brevity, therefore, exemplary trigger conditions are not described here.

[0046]

[0055] In 304, when performing an operation, one or more processors may take a control action in response to a determination that the tilt angle has reached a predetermined tilt threshold. One or more of the control actions described above with respect to 204 may be implemented for 304 to address tilt events associated with the shelf, including any combination of tilt events. For brevity, therefore, exemplary control actions are not described here. Furthermore, in one or more examples, the wall-mounted self-checkout system 100 may monitor tilt events associated with one or more of its shelves.

[0047]

[0056] In one or more further examples, a non-temporary computer-readable medium may be provided. The non-temporary computer-readable medium may have computer-readable program code embodied therein, which can be executed by one or more processors of a wall-mounted self-checkout system to determine, at least in part, that the tilt angle of the wall-mounted self-checkout system or its components has reached a predetermined tilt threshold, or that the inclination angle of the wall-mounted self-checkout system or its components has reached a predetermined tilt threshold, and to perform a control action in response to the determination that the tilt angle has reached a predetermined tilt threshold or that the inclination angle has reached a predetermined tilt threshold. The received data may include image data (e.g., current and baseline images for image comparison purposes), data from tilt sensors or accelerometers, etc. When executing the program code, the tilt and / or inclination of the wall-mounted self-checkout system may be taken into consideration, or its components may be monitored for tilt and / or inclination. For example, shelves in a wall-mounted self-checkout system may be taken into consideration. If the tilt and / or inclination of the wall-mounted self-checkout system or its components reaches a threshold level, control actions (e.g., any of the control actions described herein, and any combination thereof) may be taken to address the tilt and / or inclination of the unit or component.

[0048]

[0057] The descriptions of various embodiments are presented for illustrative purposes only and are not intended to be exhaustive or limitful to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein have been selected to best describe the principles of the embodiments, their practical applications or technical improvements to the technology found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0049]

[0058] References to embodiments presented in this disclosure are made below. However, the scope of this disclosure is not limited to the embodiments described. Rather, any combination of the following features and elements is intended to implement and practice the imagined embodiments, whether related to different embodiments or not. Furthermore, the embodiments disclosed herein may achieve advantages over other possible solutions or the prior art, but whether or not an advantage is achieved by a given embodiment does not limit the scope of this disclosure. Accordingly, the following aspects, features, embodiments and advantages are merely illustrative and should not be considered as elements or limitations of the appended claims unless expressly stated in the claims. Similarly, references to “this disclosure” should not be interpreted as a generalization of any inventive subject matter disclosed herein and should not be considered as elements or limitations of the appended claims unless expressly stated in the claims.

[0050]

[0059] The embodiments described may take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments that combine software and hardware embodiments, which are generally referred to herein as “circuits,” “modules,” or “systems.”

[0051]

[0060] One or more of the embodiments described may be a system, method, and / or a computer program product. A computer program product may include one (or more) computer-readable storage media having computer-readable program instructions for causing a processor to perform an aspect of the embodiment.

[0052]

[0061] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction-executing device. A computer-readable storage medium may, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of those described above. A non-exhaustive list of examples of computer-readable storage mediums includes portable computer diskettes, hard disks, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random-access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or raised structures in grooves on which instructions are recorded, and any suitable combination of those described above. Computer-readable storage media as used herein should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmitting media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0053]

[0062] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmitters, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and transfers those computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0054]

[0063] The computer-readable program instructions for performing the operations of the described embodiments may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++, and conventional procedural programming languages ​​such as the C programming language or similar programming languages. The computer-readable program instructions may run entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or remote server. In the latter scenario, the remote computer may connect to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it may connect to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions by utilizing state information of computer-readable program instructions for personalizing the electronic circuit in order to perform the embodiments described.

[0055]

[0064] The aspects of the embodiments described herein are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to the embodiments. It will be understood that each block in the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0056]

[0065] These computer-readable program instructions are provided to the processor of a general-purpose computer, a dedicated computer, or other programmable data processing device, so as to generate a machine such that instructions executed via the processor of the computer or other programmable data processing device create means for implementing functions / operations specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct computers, programmable data processing devices, and / or other devices to function in the described manner, thereby comprising a product in which the computer-readable storage medium storing the instructions includes instructions that implement modes of functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0057]

[0066] Computer-readable program instructions are loaded into a computer, other programmable data processing device, or other device, and a series of operational steps are performed on the computer, other programmable device, or other device to generate a computer implementation process, thereby enabling the instructions executed on the computer, other programmable device, or other device to implement the functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0058]

[0067] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products in various embodiments. In this regard, each block in a flowchart or block diagram may represent a module, segment, or part of an instruction containing one or more executable instructions for implementing a specified(multiple) logical function. In some alternative implementations, the functions described in a block may occur in a different order than that shown in the figure. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in a block diagram and / or flowchart, as well as combinations of blocks in a block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs a specified function or operation, or a combination of dedicated hardware and computer instructions.

[0059]

[0068] The foregoing is directed toward one or more embodiments, but other embodiments and further embodiments may be conceivable without departing from their basic scope, the scope of which will be determined by the following claims.

Claims

1. It is a wall-mounted self-checkout system, Housing and A counter having a horizontal panel installed on the housing and a vertical panel connected to the horizontal panel, A tower extending upward from the counter and having one or more mounts for mounting the wall-mounted self-checkout system to the wall, A computing system having one or more processors and one or more memory devices for storing programs. Equipped with, The program is configured on one or more processors. Based on the received data, it is determined that at least one of the following has occurred: the tilt angle of the wall-mounted self-checkout system has reached a predetermined tilt threshold, or the inclination angle of the wall-mounted self-checkout system has reached a predetermined tilt threshold. A control action is performed in response to the determination that the inclination angle has reached the predetermined inclination threshold, or that at least one of the following has been determined: A wall-mounted self-checkout system that enables users to perform this task.

2. Equipped with one or more additional cameras, Based on the received data, determining at least one of the following is necessary: ​​that the inclination angle of the wall-mounted self-checkout system has reached a predetermined inclination threshold, or that the inclination angle of the wall-mounted self-checkout system has reached a predetermined inclination threshold. The current image of the wall-mounted self-checkout system, captured by one or more cameras, is received from the received data. If it is known that the inclination angle of the wall-mounted self-checkout system is within a predetermined range of the reference inclination angle, and if it is known that the tilt angle of the wall-mounted self-checkout system is within a predetermined range of the reference inclination angle, a baseline image of the wall-mounted self-checkout system captured by one or more cameras is received from the received data. Determining at least one of the following: the tilt angle obtained by comparing the current image with the baseline image, or the inclination angle obtained by comparing the current image with the baseline image. A wall-mounted self-checkout system according to claim 1, comprising the features described above.

3. The display screen further comprises an accelerometer mounted on the tower and attached to or embedded in the display screen, Based on the received data, determining at least one of the following is necessary: ​​that the inclination angle of the wall-mounted self-checkout system has reached a predetermined inclination threshold, or that the inclination angle of the wall-mounted self-checkout system has reached a predetermined inclination threshold. Receiving an input from the accelerometer that indicates at least one of the tilt angle or the inclination angle, Equipped with, The wall-mounted self-checkout system according to claim 1, wherein at least one of the following is performed: the tilt angle received from the accelerometer is compared with the predetermined tilt threshold, or the inclination angle received from the accelerometer is compared with the predetermined tilt threshold.

4. The wall-mounted self-checkout system according to claim 1, wherein determining that the inclination angle of the wall-mounted self-checkout system has reached the predetermined inclination threshold, or determining that the inclination angle of the wall-mounted self-checkout system has reached the predetermined inclination threshold, is performed according to at least one of a predetermined time interval or after each user transaction.

5. The wall-mounted self-checkout system according to claim 1, wherein determining that the inclination angle of the wall-mounted self-checkout system has reached a predetermined inclination threshold, or determining that the inclination angle of the wall-mounted self-checkout system has reached a predetermined inclination threshold, is performed in response to a force applied to the wall-mounted self-checkout system reaching a force threshold, the applied force being measured by an accelerometer attached to or embedded in a display screen mounted on the tower.

6. The wall-mounted self-checkout system according to claim 1, wherein determining that the inclination angle of the wall-mounted self-checkout system has reached a predetermined inclination threshold, or determining that at least one of the inclination angles of the wall-mounted self-checkout system has reached a predetermined inclination threshold, is performed in response to a force applied to the wall-mounted self-checkout system reaching a force threshold, the applied force being measured by one or more load cells of a product scanner attached to or embedded in the counter.

7. The wall-mounted self-checkout system according to claim 1, wherein determining that the inclination angle of the wall-mounted self-checkout system has reached a predetermined inclination threshold, or determining that the inclination angle of the wall-mounted self-checkout system has reached a predetermined inclination threshold, is performed in response to one or more cameras capturing a predefined user gesture.

8. The wall-mounted self-checkout system is the wall-mounted self-checkout system according to claim 1, wherein the wall-mounted self-checkout system does not physically touch the ground.

9. Performing the aforementioned control action involves moving a lane blocker to physically close the wall-mounted self-checkout system. The wall-mounted self-checkout system according to claim 1, wherein the lane blocker is coupled to a shelf attached to the vertical panel and is movable between a retracted position and an extended position in which the lane blocker physically closes the wall-mounted self-checkout system.

10. The wall-mounted self-checkout system according to claim 1, wherein performing the control action includes changing the wall-mounted self-checkout system from a normal mode to a standby mode in which the functionality of the wall-mounted self-checkout system is reduced compared to the normal mode, or to a shutdown mode in which the functionality of the wall-mounted self-checkout system is stopped.

11. The wall-mounted self-checkout system according to claim 10, wherein the wall-mounted self-checkout system is changed to the standby mode or the shutdown mode based on at least one of the magnitude of the deviation of the tilt angle with respect to a predetermined tilt threshold, or the magnitude of the deviation of the tilt angle with respect to a predetermined tilt threshold.

12. Further equipped with a product scanner that has a load cell, The wall-mounted self-checkout system according to claim 1, wherein the control action includes automatically recalibrating the load cell of the product scanner.

13. The wall-mounted self-checkout system according to claim 12, further comprising performing the control action during the recalibration of the load cell to change the operating mode of the wall-mounted self-checkout system to prevent the user from scanning items whose price is determined by weight.

14. The wall-mounted self-checkout system according to claim 1, wherein performing the control action includes automatically communicating to the operator that the wall-mounted self-checkout system has experienced at least one of a tilt event or a tilt event.

15. It is a wall-mounted self-checkout system, Housing and A counter having a horizontal panel installed on the housing and a vertical panel connected to the horizontal panel and surrounding the side of the housing, Shelves and, A computing system comprising one or more processors and one or more memory devices for storing programs, The program is configured on one or more processors. Based on the received data, it is determined that the tilt angle of the shelf has reached a predetermined tilt threshold. In response to the determination that the tilt angle has reached the predetermined tilt threshold, a control action is performed. A wall-mounted self-checkout system that enables users to perform this task.

16. The wall-mounted self-checkout system according to claim 15, further comprising an input shelf cantilevered from the vertical panel, wherein the shelf is the input shelf.

17. The wall-mounted self-checkout system according to claim 15, wherein at least a portion of the horizontal panel is cantilevered to form an output shelf, and the shelf is the output shelf.

18. Equipped with one or more additional cameras, Based on the received data, it is determined that the tilt angle of the shelf has reached the predetermined tilt threshold. The current image of the shelf captured by the one or more cameras is received from the received data. If it is known that the tilt angle of the shelf is within a predetermined range of the reference tilt angle, a baseline image of the shelf captured by one or more cameras is received from the received data. The tilt angle is determined by comparing the current image with the baseline image. A wall-mounted self-checkout system according to claim 15, comprising the features described above.

19. The shelf has an accelerometer attached to or embedded within the shelf. Based on the received data, it is determined that the tilt angle of the shelf has reached the predetermined tilt threshold. The system includes receiving an input from the accelerometer indicating the tilt angle of the shelf from the received data, The wall-mounted self-checkout system according to claim 15, wherein the tilt angle received from the accelerometer is compared with the predetermined tilt threshold.

20. A non-temporary computer-readable medium having computer-readable program code embodied in the non-temporary computer-readable medium, the computer-readable program code being processed by one or more processors of a wall-mounted self-checkout system. Based on the received data, it is determined that at least one of the following has occurred: the tilt angle of the wall-mounted self-checkout system or its components has reached a predetermined tilt threshold, or the inclination angle of the wall-mounted self-checkout system or its components has reached a predetermined tilt threshold. A control action is performed in response to the determination that the tilt angle has reached the predetermined tilt threshold, or that at least one of the following has been determined: the tilt angle has reached the predetermined tilt threshold. A non-temporary, computer-readable medium that is executable in such a way.

Images