Detection system and method for retail environments

Abstract

A detection method for a retail environment comprises providing a thermal camera in a fixed position relative to a monitoring area within a retail environment, such that a part of a field of view of the thermal camera includes a part of the monitoring area; capturing a series of images of a person in a known duration of time using the thermal camera; identifying, in each image of the series of images, the position of the person using a temperature difference to detect an edge of the person; identifying, in each image of the series of images, at least two of a head, a body, an arm and a hand of the person by detecting temperature gradients within an area bounded by the edge of the person; determining, for each image of the series of images, the position of said person's hand and / or arm relative to the monitoring area; and registering an event when said person's hand and / or arm is within the monitoring area.

Description

[0001] Detection System and Method for Retail Environments

[0002] FIELD OF THE INVENTION

[0003] This invention relates to a system and method for detecting the movement of a person in a retail environment. In particular, the system and method of the present invention may be used to monitor a person’s movements near goods displayed in the retail environment to detect possible theft of those goods. The method and system of the present invention may be used to detect repeated removal of objects from a shelf or retail display indicative of possible theft of those objects.

[0004] BACKGROUND TO THE INVENTION

[0005] Numerous products, systems and methods exist to deter or prevent theft from a retail environment such as a shop or store. For example, security tags may be attached to high value items to deter or prevent theft of those items. However, it is not commercially feasible to attach security tags to all items within a shop or store. Firstly, it would be costly to purchase the high number of tags required. Further, security tags must be detached or deactivated at the point of sale. If it was necessary to detach or deactivate a tag associated with every item, this would be incredibly inconvenient for the store and the customer.

[0006] Accordingly, lower priced items in a shop or store are not individually tagged and may be removed from the store without detection. In many cases the theft of a single low value item from a store may not be sufficiently commercially important for a store owner to pursue. However, the theft of multiple items, including low value items, can be commercially significant.

[0007] One known method used in organised retail crime is shelf sweeping. This involves a thief removing multiple items from a shelf at one time. To deter this mode of theft, a number of solutions are available that restrict access to products on a shelf so that it is difficult to remove more than one product at a time. These solutions include, for example, guards or raised lips at shelf edges, or dividers that only permit one item to be dispensed at a time. However, these solutions also make it more difficult for legitimate shoppers to access the products.

[0008] Another solution involves monitoring a retail environment using CCTV (closed circuit television) or similar image capture equipment. It is then necessary, however, to process or monitor the captured images to detect a theft event.

[0009] It is advantageous for a shop manager or store owner to be alerted to a possible theft while the theft is occurring to be able to prevent the theft. There is therefore a drive to use image processing to detect movement or patterns in captured images, and to alert a user when a predetermined movement or pattern is detected that is indicative of a possible theft. However, the data processing capabilities required to reliably detect an event through processing of CCTV or video images is extremely high and is prohibitively expensive for most retail establishments.

[0010] Another known technology utilises a curtain or line of infrared (IR) radiation that is aligned with the edge of a shelf. A monitoring system may then be used to detect an object crossing the IR curtain multiple times, which may indicate a possible theft of the items on the shelf. However, these systems are known to be unreliable, with many false alarms. To minimise the false alarms significant adjustment is needed, which adds to the complexity and cost of these systems. In most cases, therefore, these systems are not commercially viable.

[0011] It is an aim of the present invention to provide an improved system and method for detection of an event in a retail environment that is indicative of a possible theft. It is an aim of the present invention to provide an improved detection system and method that overcomes at least one problem associated with prior art detection systems and methods, whether referred to herein or otherwise.

[0012] SUMMARY OF THE INVENTION

[0013] A first aspect of the invention provides a detection method for a retail environment comprising: providing a thermal camera in a fixed position relative to a monitoring area within a retail environment, such that a part of a field of view of the thermal camera includes a part of the monitoring area; capturing a series of images of a person in a known duration of time using the thermal camera; identifying, in each image of the series of images, the position of the person using a temperature difference to detect an edge of the person; identifying, in each image of the series of images, at least two of a head, a body, an arm and a hand of the person by detecting temperature gradients within an area bounded by the edge of the person; determining, for each image of the series of images, the position of said person’s hand and / or arm relative to the monitoring area; and registering an event when said person’s hand and / or arm is within the monitoring area.

[0014] In preferred embodiments the head of the person is identified by detecting a temperature gradient between the head and the body of the person.

[0015] In preferred embodiments the arm of the person is identified by detecting a temperature gradient along a length of the arm.

[0016] In some embodiments the detection method further comprises the steps of: determining, for the series of images, the number of times the person’s hand and / or arm enters the monitoring area; comparing said determined number of times with a threshold number of times; and generating an alert signal when the number of times the person’s hand and / or arm enters the monitoring area exceeds the threshold number of times.

[0017] In some embodiments the detection method further comprises the steps of: determining, in each image of the series of images, the position of the person’s hand; measuring, in each image of the series of images, a distance between the person’s hand and an edge of the monitoring area; and tagging said image if the person’s hand is within the monitoring area and said measured distance is greater than a threshold distance.

[0018] In these embodiments the detection method may further comprise the steps of: determining the number of tagged images with the series of images; comparing said number of tagged images with a threshold tagging number; and generating an alert signal when the number of tagged images exceeds the threshold tagging number.

[0019] In other embodiments the detection method may further comprise the steps of: determining a duration of time for which the person’s hand remains within the monitoring area; comparing said duration of time with a threshold time; and generating an alert signal when the duration of time exceeds the threshold time.

[0020] In yet further embodiments the detection method may further comprise the steps of: detecting, in each image of the series of images, whether there is an object in the person’s hand; identifying, in the series of images, a removal event corresponding to an object being present in the person’s hand crossing an edge of the monitoring area from a region within the monitoring area to a region outside the monitoring area; and generating an alert signal when the number of identified removal events exceeds a threshold number of events.

[0021] In these embodiments the detection method preferably further comprises the steps of: identifying, in the series of images, a replacement event corresponding to an object being present in the person's hand crossing an edge of the monitoring area from a region outside the monitoring area to a region within the monitoring area; calculating a net removal event number by subtracting the number of replacement events from the number of identified removal events; and generating an alert signal when the net removal event number exceeds a threshold number.

[0022] A second aspect of the invention provides a detection system for a retail environment comprising: a thermal imaging camera mounted within a retail environment such that a part of a monitoring area is within a field of view of the thermal imaging camera; and a processor configured to carry out the method steps of: identifying, in each image of a series of images captured by the thermal imaging camera, a position of a person using a temperature difference to detect an edge of the person; identifying, in each image of the series of images, at least two of a head, a body, an arm and a hand of the person by detecting temperature gradients within an area bounded by the edge of the person; and determining, for each image of the series of images, the position of said person’s hand and / or arm relative to the monitoring area; and registering an event when said person’s hand and / or arm is within the monitoring area.

[0023] Preferably the processor is configured to identify the head of the person by detecting a temperature gradient between the head and the body of the person. Preferably the processor is configured to identify the arm of the person by detecting a temperature gradient along a length of the arm.

[0024] In some preferred embodiments the processor is further configured to: determine, for the series of images, the number of times the person’s hand and / or arm enters the monitoring area; compare said determined number of times with a threshold number of times; and generate an alert signal when the number of times the person’s hand and / or arm enters the monitoring area exceeds the threshold number of times. In other embodiments the processor is further configured to: determine, in each image of the series of images, the position of the person’s hand; measure, in each image of the series of images, a distance between the person’s hand and an edge of the monitoring area; and tag said image if the person’s hand is within the monitoring area and said measured distance is greater than a threshold distance.

[0025] In these embodiments the processor may be further configured to: determine the number of tagged images with the series of images; compare said number of tagged images with a threshold tagging number; and generate an alert signal when the number of tagged images exceeds the threshold tagging number.

[0026] In yet further embodiments the processor is further configured to: determine a duration of time for which the person’s hand remains within the monitoring area; compare said duration of time with a threshold time; and generate an alert signal when the duration of time exceeds the threshold time.

[0027] In some embodiments the processor is further configured to: detect, in each image of the series of images, whether there is an object in the person’s hand; identify, in the series of images, a removal event corresponding to an object being present in the person’s hand crossing an edge of the monitoring area from a region within the monitoring area to a region outside the monitoring area; and generate an alert signal when the number of identified removal events exceeds a threshold number of events.

[0028] In these embodiments the processor may be further configured to: identify, in the series of images, a replacement event corresponding to an object being present in the person’s hand crossing an edge of the monitoring area from a region outside the monitoring area to a region within the monitoring area; calculate a net removal event number by subtracting the number of replacement events from the number of identified removal events; and generate an alert signal when the net removal event number exceeds a threshold number.

[0029] In a detection method according to the first aspect of the invention or in a detection system according to the second aspect of the invention, the monitoring area may comprise a shelf, a storage container, or a hanging rail in a retail environment.

[0030] A third aspect of the invention provides a computer readable medium storing computer implementable instructions to cause a programmable computer to perform a method comprising the steps of: identifying, in each image of a series of images captured by a thermal imaging camera, a position of a person using a temperature difference to detect an edge of the person; identifying, in each image of the series of images, at least two of a head, a body, an arm and a hand of the person by detecting temperature gradients within an area bounded by the edge of the person; and determining, for each image of the series of images, the position of said person’s hand and / or arm relative to a predetermined monitoring area within said image; and registering an event when said person’s hand and / or arm is within the monitoring area.

[0031] Preferred and / or optional features of each aspect and embodiment described above may also be used, alone or in appropriate combination, in the other aspects and embodiments also.

[0032] BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The invention will now be further described by way of example only and with reference to the accompanying drawings, in which like reference signs are used for like features, and in which:

[0034] Figure 1 is an example of an image obtained from a thermal imaging camera, the image including a person;

[0035] Figure 2 is an example of an image obtained from a thermal imaging camera, the image including a person, an edge of the person being highlighted following edge detection, and showing a line indicating a boundary of a monitoring area according to the present invention;

[0036] Figure 3 shows the image of Figure 2, following detection of a head, a body and an arm of the person;

[0037] Figure 4 shows the image of Figure 2, with arrows indicating dimensions of a wrist and a hand of the person and highlighting a second movement boundary of the monitoring area

[0038] Figure 5 illustrates an outline of a person’s hand holding a first object; and

[0039] Figure 6 illustrates an outline of a person’s hand holding a second object.

[0040] DETAILED DESCRIPTION

[0041] The present invention provides a method and system for detecting possible theft of items from a retail environment such as a shop or store. In one arrangement, the system is configured to detect movement of a person near a shelf, or other retail display stand, associated with the repeated removal of items from the shelf, indicative of shelf sweeping.

[0042] The system comprises a thermal imaging camera that is mounted in a fixed position to image an area within a retail environment. The retail environment may be a shop, a store, a cafe, a restaurant, an opticians, a salon, a pharmacy, or any other premises in which items are displayed for purchase by a customer. It will be understood that references to shop in the following description refer equally to any other such retail environment.

[0043] The thermal imaging camera operates in a normal manner, capturing images based on intensity of infrared (IR) radiation within a field of view. Preferably the field of view of the camera is fixed. In other embodiments the field of view may be adjusted by adjusting a zoom or focal length of the camera. The images are preferably grey scale images, but in some embodiments the images may be colour images.

[0044] The thermal imaging camera is mounted so that the field of view includes at least a part of a monitoring area. The monitoring area may be any region of the shop which includes items on display for purchase by a customer. The monitoring area may be, for example, a shelf, a storage container such as a box, basket or bin, a hanging rail, a counter, a table, or a display stand. The field of view of the camera preferably includes at least an edge region of the monitoring area that is customer facing. That is to say, an edge region of the monitoring area over which a customer’s hand will typically extend to reach an item on display and to pick up the item for potential purchase. The thermal imaging camera may, for example, be mounted so that the field of view includes the front edge of a shelf.

[0045] The thermal imaging camera is preferably positioned such that a centre line of the field of view of the camera is vertical and the field of view is below the camera.

[0046] The thermal imaging camera is configured to capture a series of images at predetermined time intervals over a period of time, i.e. within a known duration of time. The thermal imaging camera is preferably a video camera. The frame rate of the video camera is preferably constant so that the camera captures a predetermined number of images in a series of images in a known duration of time.

[0047] The images captured by the thermal imaging camera may be stored directly in a memory, or they may be processed (as described below) before being saved in a memory. In some embodiments the images may not be saved in a memory; however, an advantage of storing the images is that they may be retrieved and viewed at a later date.

[0048] The system further comprises a processor configured to perform image analysis on each of the images captured by the camera. The first step of the image analysis is the identification of a hot body within the image that may be a person using a suitable object detection algorithm. This identification may be based on the absolute measured temperature of the hot body or a temperature difference between the hot body and its surroundings, together with a size of the hot body.

[0049] An example of a grayscale image 10 captured by a thermal imaging camera is shown in Figure 1. A hot body 12 may be identified towards the left-hand edge of the image 10.

[0050] An edge detection algorithm is used to identify the edge of the person in the image. The edge detection algorithm may identify large temperature gradients within the image or large differences in intensity between neighbouring pixels in the image to locate an edge of the hot body. The image processing method may include the step of highlighting the edge 14 of the hot body 12 as illustrated in Figure 2.

[0051] To confirm that the hot body corresponds to a person the processor may be configured to identify a head and a torso of the hot body. Temperature gradients within the region bounded by the edge of the person may be used to differentiate the head 16 and the torso 18. The shapes of the head and torso, together with their relative sizes, may be used to confirm that the hot body is a person.

[0052] As described above, the thermal imaging camera is preferably positioned so that its field of view is vertically downwards. In this way, the camera views a person from above. This positioning of the camera may permit more accurate image processing to detect and identify a person’s head and torso, and to enable object tracking as described below. The fixed positioning of the thermal imaging camera minimises the problems often associated with object tracking in images, including cluttered background (redundant information), variation in illumination, occlusion, scale variation and changes in shape of the object. This enables reasonably accurate object detection and tracking even with relatively low resolution thermal images.

[0053] The detection method of the present invention is configured to identify and track a person’s hand as it enters and leaves the monitoring area. Repeated movement of a person’s hand into and out of the monitoring area may be indicative of the theft of multiple items from the monitoring area. Similarly, movement of a person’s hand into the monitoring area and the hand remaining in the monitoring area for a prolonged period of time may be indicative of tampering with one or more items within the monitoring area.

[0054] The processor is preferably configured to identify a boundary line corresponding to the edge of the monitoring area. The processor may be configured to display this boundary line 20 on the image, as illustrated in Figures 2, 3 and 4. In the present example, the monitoring area 22 is disposed to the right-hand side of the boundary line 20.

[0055] The processor may then identify events corresponding to a part of a person crossing the boundary line into and out of the monitoring area. To more accurately determine whether the movement of a person across the boundary line may relate to a potential theft, the method preferably includes the step of identifying the arm and / or hand of the person and determining the movement of the person’s arm or hand relative to the boundary line.

[0056] To identify an arm of the person the processor is preferably configured to identify an elongate region of the hot body that is connected to and extends from the torso of the hot body. Again, it will be appreciated that with the camera mounted above the person, and imaging vertically downwards, when a person stretches their arm out to reach into the monitoring area, the shape of the arm of the person will be identifiable by a suitable imaging processing algorithm.

[0057] The processor may be configured to confirm the identity of an arm 24 of the person by detecting temperature gradients along a length of the arm. The processor may be configured to distinguish parts of the arm, and in particular to identify a hand at the end of the arm. Identification of a hand may be achieved by measuring dimensions of the arm. In particular, the processor may be configured to measure a width of the arm at a first region 26 proximate a distal end of the arm (corresponding to the likely position of a hand 28) and a minimum width 30 at a distance from the distal end (corresponding to a wrist 32 of the arm). If the processor confirms the presence of an elongate hot body section having dimensions within ranges expected for an arm, wrist and hand, then the processor may be configured to identify the position of the person’s hand 28 within the image.

[0058] The processor is configured to determine the position of the hand 28 of the hot body relative to the boundary line 20. The processor is preferably configured to tag an image if the hand 28 is identified as being within the monitoring area 22.

[0059] The image analysis described above may be performed for each of the images in the series of images captured by the thermal imaging camera. It will be understood that, once a hot body identified as a person has been identified in one image using the object detection algorithm, an object tracking algorithm may be used to track the hot body in subsequent sequential images. In particular a single object tracking algorithm may be used to track the position of the person’s hand 28 relative to the boundary line 20 and monitoring area 22.

[0060] It will be appreciated that a person reaching their hand into the monitoring area, picking up an item and then removing the item and hand from the monitoring area is most likely to be associated with a genuine purchase of that item. Accordingly, the processor and image processing algorithms are configured to identify a pattern of object movement associated with a potential theft event.

[0061] In a first embodiment the algorithm tracks the hand of the person in a series of images and determines whether the hand crosses the boundary line multiple times in a predetermined duration of time.

[0062] Preferably a threshold number of times that a person’s hand enters the monitoring area within a predetermined duration of time is set. This threshold number may be, for example, 4 times in 10 seconds. The threshold number should be set at a level that may be associated with a shelf sweep theft in which a person removes multiple items from the shelf in a short period of time. However, the threshold number should not be set too low so that the shop does not experience a high number of false alarms. For example, a genuine customer may take at item off the shelf to view it, replace the item while they consider the purchase, and then remove the item again when they decide to continue with the purchase.

[0063] In this embodiment the processor is configured to compare a determined number of times that a person’s hand crosses the boundary line and enters the monitoring area with the threshold number. If the determined number of times is greater than the threshold number then the processor is preferably configured to generate an alert signal. The alert signal may generate a visual alert such as an icon on a screen or may generate an audible alert such as an audible alarm or buzzer.

[0064] In another embodiment the image analysis algorithm may determine a distance between the person’s hand and the boundary line. The processor may be configured to tag an image only if the determined distance between the person’s hand and the boundary line exceeds a threshold distance. The processor may then determine whether the number of tagged images in a series of images exceeds a threshold number.

[0065] In this way, in some embodiments, in addition to tracking the movement of a person’s hand into the monitoring area, the processor may only tag an image if the person’s hand moves far enough into the monitoring area. This may avoid false alarms being generated from a person moving around near the edge of a shelf, for example pointing at a number of different items on a shelf when considering a purchase.

[0066] In these embodiments the processor may also be configured to identify a second boundary line 34 at a fixed distance from the first boundary line 20 (the first boundary line corresponding to the edge of the monitoring area). The processor may be configured to display this second boundary line 34 on the image, as illustrated in Figure 4. The second boundary line 34 is preferably within the monitoring area.

[0067] The processor may be configured to identify if the person’s hand crosses one or both of the first and second boundary lines 20, 34. The processor may be configured to tag an image only if the hand 28 is identified as having crossed both the first and second boundary lines 20, 34 such that the hand is located sufficiently far inside the monitoring area 22.

[0068] In a further embodiment, rather than detecting multiple crossings of the boundary line to detect a possible theft event, the processor is configured to detect if a person’s hand remains in the monitoring area for more than a predetermined length of time. This may be indicative of a person tampering with an item in the monitoring area.

[0069] In this embodiment the processor is configured to detect the position of the hand 28 of the hot body relative to the boundary line 20. The processor is preferably configured to tag an image if the hand 28 is identified as being within the monitoring area 22. The processor is preferably configured to calculate a duration of time for which the hand remains in the monitoring area. This may be calculated based on the frame rate of the video captured by the thermal imaging camera. The processor, in this embodiment, is then configured to compare the calculated duration of time with a threshold length of time. If the calculated duration of time exceeds the threshold length of time, then the processor is preferably configured to generate an alert signal.

[0070] One particular example of a method of object detection and tracking is as follows.

[0071] Procedure

[0072] After acquiring a live video feed from a thermal camera, an algorithm performs the following steps on each frame for activity recognition:

[0073] (i) identify all contours (connected regions) having a temperature in a range of full human body temperature provided by a user; (ii) for an appropriately sized contour that may represent a human, detect the contours with a temperature range indicative of a head as provided by the user and use this value to calculate or look up a temperature associated with a hand;

[0074] (iii) confirming that the contours corresponding to the head are within the contours identified as a human body to confirm presence of valid human body;

[0075] (iv) proceed with subsequent steps of the algorithm only if valid human body confirmed;

[0076] (v) identify the extremity (hand) of the detected human body proximate or closest to the boundary line;

[0077] (vi) track the identified extremity using a centroid tracking algorithm centred around a point and compare it with the position of the boundary line using a point and line comparison algorithm to detect if the identified extremity is on one side of the boundary line or the other;

[0078] (vii) between consecutive frames, compare if the position of the extremity is the same with respect to the boundary line or if it has moved and, if the position has changed, log that the extremity has crossed the boundary line;

[0079] (vii) identify if the crossing of the boundary line is indicative of the extremity (hand) passing into or out of the monitoring area;

[0080] (viii) if the crossing direction is into the monitoring area, log an IN direction with a time corresponding to that crossing, and if the crossing direction is out of the monitoring area, log an OUT direction with a time corresponding to that crossing and check for a previous logged IN direction;

[0081] (ix) if the crossing direction is out of the monitoring area, calculate a time between the time corresponding to the OUT direction and the time corresponding to the IN direction and compare the calculated time with a predetermined time window provided by the user;

[0082] (x) if the calculated time is within the predetermined time window, log an event for that human body for later analysis and report.

[0083] It will be appreciated that the above methods of object detection and tracking to determine a possible theft event may be utilised with a relatively low resolution thermal imaging camera. This allows this method to be executed using low cost equipment. In some instances it may be beneficial to obtain a greater amount of detail from the image analysis to permit more precise alerts to be generated. To achieve this it is desirable to use a higher resolution thermal imaging camera. In the above embodiments it was only necessary to have sufficient image resolution to identify and track a hand of a person into and across a defined area. In preferred embodiments it would be desirable to identify if a person is holding an object or not, and possibly obtain some information about the size of the object that the person is holding. To achieve this, the resolution of the images obtained from the thermal imaging camera must be sufficient to determine a shape of a person’s hand.

[0084] As illustrated in Figures 5 and 6, with higher resolution images it is possible to determine, from the shape of a person’s hand 28 if the person is holding an object 36, and an approximate size and shape of the object. As the images are obtained using a thermal imaging camera, it would not be possible to determine what the object is, but, importantly, image analysis may be used to determine: whether a person picks up and object and removes it from the monitoring area; whether a person returns an object to the monitoring area; and whether the removed object and the returned object are the same size and shape.

[0085] Accordingly, in some embodiments of the invention, a thermal imaging camera of sufficient resolution to determine a shape of a hand is positioned to image a monitoring area.

[0086] A processor is configured to identify, in each image captured by the camera, a person’s hand and any object held by the hand. The processor is further configured to track both the hand and the object. In particular, a multiple object tracking algorithm may be used to track the position of both the person’s hand and the object relative to the boundary line and the monitoring area.

[0087] A removal event may be logged against a series of images if the processor determines that a hand holding an object has moved across the boundary line and out of the monitoring area. The processor may be configured to determine a size and shape of the object being removed.

[0088] Similarly, a return event may be logged against a series of images if the processor determines that a hand holding an object has moved across the boundary line and into the monitoring area. The processor may be configured to determine a size and shape of the object being removed.

[0089] The processor may be configured to determine if the size and shape of the object in the return event is the same as or substantially the same as the size and shape of the object in the removal event. This may be used to determine if a person has picked up an object and then returned the same object to the shelf. A removal event followed by a return event, in which the processor determines that the same object has been removed and then returned to the shelf, may cancel the logging of both the removal event and the return event. Alternatively the combination of the removal event and the return event may be replaced by the logging of a browsing event.

[0090] The processor may be configured to generate an alert signal if a number of removal events are logged in a time period that exceeds a threshold number of removal events. This would be seen by the imaging tracking as a hand repeatedly (i) crossing the boundary line and entering the monitoring area not holding an object, (ii) the hand crossing the boundary line out of the monitoring area holding an object, (iii) the hand crossing the boundary line and returning to the monitoring area not holding an object, and (iv) the hand crossing the boundary line out of the monitoring area holding an object.

[0091] It will be appreciated that in many instances a person may which to purchase more than one of the same item, and may therefore be removing several items from the shelf. The threshold number of removal events may, therefore, be set to a number such as six or eight removal events. If the thermal imaging camera is monitoring an area containing a high value item that is usually purchased as a single item, such as an item of electrical equipment, the threshold number of removal events may be set to a lower number such as two or three removal events.

[0092] An image analysis algorithm may be configured to identify a shape of the person’s hand by, for example, identifying digits of the hand and measuring a distance between digits when it is holding an object. This distance may be compared with a reference distance relating to the size of the items in the monitoring area. This allows the processor to determine whether a person’s hand is holding an object larger than the expected size of the item in the monitoring area. This may be indicative of a person grabbing several items at one time to remove multiple items quickly from the shelf.

[0093] The present invention therefore utilises object detection and object tracking algorithms to analyse the movement of a person’s hand into and out of a monitoring area corresponding to a retail display such as a shelf. The present invention utilises image analysis to determine if the removal of items from the monitoring area is indicative of a possible theft event. Importantly, the image analysis utilises images obtained from a thermal imaging camera. The thermal imaging camera is preferably mounted above the field of view and is preferably mounted to view directly vertically downwards. The thermal imaging camera may obtain relatively low resolution images meaning that the cost of the system is minimised.

[0094] Other modifications and variations not explicitly disclosed above may also be contemplated without departing from the scope of the invention as defined in the appended claims.

Claims

CLAIMS1 . A detection method for a retail environment comprising: providing a thermal camera in a fixed position relative to a monitoring area within a retail environment, such that a part of a field of view of the thermal camera includes a part of the monitoring area; capturing a series of images of a person in a known duration of time using the thermal camera; identifying, in each image of the series of images, the position of the person using a temperature difference to detect an edge of the person; identifying, in each image of the series of images, at least two of a head, a body, an arm and a hand of the person by detecting temperature gradients within an area bounded by the edge of the person; determining, for each image of the series of images, the position of said person’s hand and / or arm relative to the monitoring area; and registering an event when said person’s hand and / or arm is within the monitoring area.

2. A detection method according to Claim 1 , in which the head of the person is identified by detecting a temperature gradient between the head and the body of the person.

3. A detection method according to Claim 1 or Claim 2, in which the arm of the person is identified by detecting a temperature gradient along a length of the arm.

4. A detection method according to any preceding claim, further comprising the steps of: determining, for the series of images, the number of times the person’s hand and / or arm enters the monitoring area; comparing said determined number of times with a threshold number of times; and generating an alert signal when the number of times the person’s hand and / or arm enters the monitoring area exceeds the threshold number of times.

5. A detection method according to any preceding claim, further comprising the steps of: determining, in each image of the series of images, the position of the person’s hand; measuring, in each image of the series of images, a distance between the person’s hand and an edge of the monitoring area; and tagging said image if the person’s hand is within the monitoring area and said measured distance is greater than a threshold distance.

6. A detection method according to Claim 5, further comprising the steps of: determining the number of tagged images with the series of images; comparing said number of tagged images with a threshold tagging number; and generating an alert signal when the number of tagged images exceeds the threshold tagging number.

7. A detection method according to any one of Claims 1 to 3, further comprising the steps of: determining a duration of time for which the person’s hand remains within the monitoring area; comparing said duration of time with a threshold time; and generating an alert signal when the duration of time exceeds the threshold time.

8. A detection method according to any one of Claims 1 to 3, further comprising the steps of: detecting, in each image of the series of images, whether there is an object in the person’s hand; identifying, in the series of images, a removal event corresponding to an object being present in the person's hand crossing an edge of the monitoring area from a region within the monitoring area to a region outside the monitoring area; and generating an alert signal when the number of identified removal eventsexceeds a threshold number of events.

9. A detection method according to Claim 8, further comprising the steps of: identifying, in the series of images, a replacement event corresponding to an object being present in the person’s hand crossing an edge of the monitoring area from a region outside the monitoring area to a region within the monitoring area; calculating a net removal event number by subtracting the number of replacement events from the number of identified removal events; and generating an alert signal when the net removal event number exceeds a threshold number.

10. A detection system for a retail environment comprising: a thermal imaging camera mounted within a retail environment such that a part of a monitoring area is within a field of view of the thermal imaging camera; and a processor configured to carry out the method steps of: identifying, in each image of a series of images captured by the thermal imaging camera, a position of a person using a temperature difference to detect an edge of the person; identifying, in each image of the series of images, at least two of a head, a body, an arm and a hand of the person by detecting temperature gradients within an area bounded by the edge of the person; and determining, for each image of the series of images, the position of said person’s hand and / or arm relative to the monitoring area; and registering an event when said person’s hand and / or arm is within the monitoring area.

11. A detection system according to Claim 10, in which the processor is configured to identify the head of the person by detecting a temperature gradient between the head and the body of the person, and is configured to identify the arm of the person by detecting a temperature gradient along a length of the arm.

12. A detection system according to Claim 10 or Claim 11 , in which the processor is further configured to:determine, for the series of images, the number of times the person’s hand and / or arm enters the monitoring area; compare said determined number of times with a threshold number of times; and generate an alert signal when the number of times the person’s hand and / or arm enters the monitoring area exceeds the threshold number of times.

13. A detection system according to Claim 10 or Claim 11 , in which the processor is further configured to: determine, in each image of the series of images, the position of the person’s hand; measure, in each image of the series of images, a distance between the person’s hand and an edge of the monitoring area; and tag said image if the person’s hand is within the monitoring area and said measured distance is greater than a threshold distance.

14. A detection system according to Claim 13, in which the processor is further configured to: determine the number of tagged images with the series of images; compare said number of tagged images with a threshold tagging number; and generate an alert signal when the number of tagged images exceeds the threshold tagging number.

15. A detection system according to Claim 10 or Claim 11 , in which the processor is further configured to: determine a duration of time for which the person’s hand remains within the monitoring area; compare said duration of time with a threshold time; and generate an alert signal when the duration of time exceeds the threshold time.

16. A detection system according to Claim 10 or Claim 11 , in which the processoris further configured to: detect, in each image of the series of images, whether there is an object in the person’s hand; identify, in the series of images, a removal event corresponding to an object being present in the person’s hand crossing an edge of the monitoring area from a region within the monitoring area to a region outside the monitoring area; and generate an alert signal when the number of identified removal events exceeds a threshold number of events.

17. A detection system according to Claim 16, in which the processor is further configured to: identify, in the series of images, a replacement event corresponding to an object being present in the person’s hand crossing an edge of the monitoring area from a region outside the monitoring area to a region within the monitoring area; calculate a net removal event number by subtracting the number of replacement events from the number of identified removal events; and generate an alert signal when the net removal event number exceeds a threshold number.

18. A detection method according to any one of Claims 1 to 9 or a detection system according to any one of Claims 10 to 17, in which the monitoring area comprises a shelf, a storage container, or a hanging rail in a retail environment.

19. A computer readable medium storing computer implementable instructions to cause a programmable computer to perform a method comprising the steps of: identifying, in each image of a series of images captured by a thermal imaging camera, a position of a person using a temperature difference to detect an edge of the person; identifying, in each image of the series of images, at least two of a head, a body, an arm and a hand of the person by detecting temperature gradients within an area bounded by the edge of the person; and determining, for each image of the series of images, the position of said person’s hand and / or arm relative to a predetermined monitoring area within saidimage; and registering an event when said person’s hand and / or arm is within the monitoring area.

Images