Behavioral event recognition system

Abstract

The present invention discloses a behavior event recognition system, including a smart shelf, a radar and a pressure sensor. Based on two behavior detection modules, the system can accurately detect and generate behavior events of an operator approaching and moving away from a smart shelf, and then perform auxiliary judgment in combination with a pressure change event of a smart shelf. The system can accurately judge whether the operator moves away from the smart shelf after taking and / or putting operations on the smart shelf. The system is easy to operate and promote, and has strong use value.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a behavior event recognition system. Background Art

[0002] Currently, one-dimensional and two-dimensional barcode labels are the most commonly used identification technologies for warehouse item management, and they are also the most widely used technologies. Commonly used labels use visible light scanning for identification. Therefore, if the label is damaged or folded during use, it will not be recognized. In addition, barcode acquisition requires light to be aligned with the barcode; acquisition outside of the visible light is impossible. Therefore, barcode technology is generally inefficient and difficult to read in batches.

[0003] UHF RFID tags operate in the UHF band, with a typical read / write range of 3-8 meters. They can read any RFID tag within the antenna's signal range, and can read hundreds of tags at once. This feature significantly improves efficiency in material management, making it extremely convenient for operations such as warehousing, outbound delivery, and inventory checks. Compared to conventional tags, RFID tags offer significant advantages in read / write distance, non-line-of-sight read / write, and single-use batch read capability. Furthermore, RFID tags can be read / written repeatedly and reused.

[0004] Combining UHF RFID technology with shelves creates a smart RFID shelf that can automatically identify item tags within the shelf using an UHF card reader terminal. Compared to traditional shelves, the advantages of smart RFID shelves are as follows:

[0005] (1) Support free shelving (return): When a user places an item on a certain layer of the smart RFID shelf, the UHF card reader will automatically recognize the item placement event, identify and generate the tag information of the newly added item, and then upload the item tag information, shelf number, and layer number information to the warehouse management system. The management system will update the inventory information and record the shelving operation information of the corresponding item under the user's account. There is no need for additional registration and confirmation procedures. The item can be placed and returned at any time, realizing free shelving (return) operations.

[0006] (2) Support free removal (retrieval): The user selects the attributes of the item to be removed from the shelf at the warehouse management system operation terminal, and the smart RFID shelf will automatically light up the indicator light of the layer where the item is stored. The user takes the item off the shelf, and the UHF card reader will automatically recognize the item removal event, identify and generate the tag information of the removed item, and then upload the item tag information and shelf number and layer number information to the warehouse management system. The management system will update the inventory information and record the removal operation information of the corresponding item under the user's account. There is no need for additional registration and confirmation procedures, and the item can be taken away at any time, realizing free removal (retrieval) operation.

[0007] (3) Supports free shelving and de-shelving (returning and taking out) at the same time: users can put in and take out any number of items from a certain layer of the smart RFID shelf. The UHF card reader will automatically identify the item putting in and taking out events, identify and generate label information for the newly added and reduced items, and then upload the item label information and shelf number and layer number information to the warehouse management system. The management system will update the inventory information and record the shelving and de-shelving operation information of the corresponding items under the user's account. No additional registration and confirmation process is required to complete the free shelving and de-shelving operations.

[0008] (4) Support remote inventory: The warehouse management system operation terminal can control any smart RFID shelf to perform automatic inventory, and automatically upload the inventory results to the warehouse management system after the inventory is completed. The management system can support item inventory balance alarms with the help of relevant data uploaded by the smart RFID shelf, support recording and analysis of the consumption cycle of various items, and facilitate users to carry out targeted pre-replenishment and adjust operation strategies.

[0009] Although the above-mentioned smart RFID shelves have many advantages, they still have the technical difficulty of being unable to accurately detect the behavior of the operator, and then determine whether an item picking and placing operation has occurred on a certain layer of a smart RFID shelf and when the operator has completed the item picking and placing operation.

[0010] There are currently two common solutions to the above problems, as follows:

[0011] Solution 1: Detection based on infrared sensor signals. When the sensor is triggered, the system identifies the shelf as someone operating it and controls the relevant shelf to enter tag recognition mode. When the sensor is deactivated, the system identifies the shelf as unoccupied and disables tag recognition mode for the relevant shelf.

[0012] advantage:

[0013] (1) It has the function of automatically turning on and off tag recognition, and its power consumption is relatively low.

[0014] (2) To a certain extent, relevant shelves can be designated for tag identification to avoid tag identification on multiple shelves at the same time, thus reducing the impact of RF signal interference and crosstalk between shelves.

[0015] (3) It supports users to freely take and put items without any feeling, and the overall operation process is relatively simple.

[0016] shortcoming:

[0017] The system cannot identify when an item has been placed or removed. Once a person enters the sensor's detection range, the shelf triggers tag recognition. As items move, the tag's status may fluctuate between being placed, removed, and placed again, causing the system to process a large amount of invalid intermediate data. Furthermore, when a user carries multiple items and wishes to place some on the current shelf, all of the items in their possession may be recognized by the shelf, resulting in the system recording erroneous data.

[0018] Solution 2: Using a motion detection camera: When a motion detection camera detects a person entering the preset monitoring screen boundary, the system identifies the person as approaching the shelf and controls the shelf for tag recognition. When the person moves away from the monitoring screen boundary, the system identifies the person as leaving the shelf and controls the shelf for tag recognition.

[0019] advantage:

[0020] (1) Accurate recognition: it can reliably identify registered faces and trigger an alarm when unregistered people approach the shelf.

[0021] (2) To a certain extent, relevant shelves can be designated for tag identification to avoid tag identification on multiple shelves at the same time, thus reducing the impact of RF signal interference and crosstalk between shelves.

[0022] (3) It supports users to freely take and put items without any feeling, and the overall operation process is relatively simple.

[0023] shortcoming:

[0024] The hardware and software costs are high, and there are high requirements for the computing power of the hardware equipment.

[0025] Comprehensive analysis shows that the two technical solutions of the above-mentioned prior art cannot solve the technical problems existing in the background technology. Summary of the Invention

[0026] The purpose of the present invention is to overcome the various problems mentioned above and provide a behavior event recognition system based on millimeter-wave radar, which can accurately detect the behavior of shelf operators, determine whether an item picking and placing operation has occurred on a specific layer of a shelf, and when the operator has completed the item picking and placing operation.

[0027] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0028] The behavioral event recognition system includes a shelf having at least one layer, each layer having a spaced area for placing items, and a pressure sensor disposed below each spaced area. The pressure sensor is configured to detect in real time whether the weight of items in the corresponding area changes, and if so, generates a third pressure change event.

[0029] A radar is installed on the top of the shelf to detect the entry and exit behavior of operators in its irradiation area in real time and send the real-time detected distance data to the behavior event recognition system.

[0030] For a single shelf scenario, without considering the interference of multiple radar detection ranges, the shelf will form a fan-shaped area in front of the shelf with the radar's farthest illumination distance. That is, the fan-shaped area in front of the shelf formed by the radar with its farthest illumination distance as the radius is the boundary area.

[0031] The behavior event recognition system generates a first behavior detection module based on characteristic calculation of distance data changes during the entire process of the operator entering the area within the boundary into the area outside the boundary.

[0032] The first behavior detection module is used to determine whether the operator enters the area outside the boundary from the inner boundary area, and if so, generate a first behavior event;

[0033] When both the first behavior event and the third pressure change event occur, it means that the operator has taken and / or placed an item away from the shelf.

[0034] Furthermore, when there are multiple shelves coexisting, a boundary distance is set at the front end of each shelf, and the boundary distance divides the radar illumination space at the front end of the shelf into two sector-shaped areas, namely: the sector-shaped area at the front end of the shelf formed by the radar irradiating with the boundary distance as a radius is the inner boundary area, and the sector-shaped area at the front end of the shelf from the boundary distance to the farthest irradiation distance of the radar is the outer boundary area;

[0035] The boundary distance is set to cover as large a sector-shaped area as possible at the front end of the shelf without overlapping the boundary areas of adjacent shelves;

[0036] The behavior event recognition system optimizes the first behavior detection module based on the characteristics of the distance data change during the entire process of the operator entering the area inside the boundary into the area outside the boundary and generates a first behavior detection module′;

[0037] The first behavior detection module' is used to determine whether the operator enters the area outside the boundary from the inner boundary area, and if so, generate a first behavior event';

[0038] When both the first behavior event' and the pressure change event occur, it means that the operator has taken and / or placed an item away from the shelf.

[0039] Furthermore, when the area within the boundary does not completely cover the effective area at the front end of the shelf, that is, there are first edge side areas and / or second edge side areas on both sides of the front edge of the shelf that are not covered by the area within the boundary; if the operator moves in the first edge side area or the second edge side area, then within a certain detection period, there will be a distance difference between the straight-line distances of the operator detected by the radar before and after, and when the straight-line distance detected after is greater than the straight-line distance detected before, and the distance difference is greater than the distance difference threshold, the behavior event recognition system calculates and generates a second behavior detection module M2 based on the change characteristics of the above distance data;

[0040] The distance difference threshold is set based on the average moving speed of the operator during the detection period, and is used to filter out a set of distance data that meets the conditions from a large amount of real-time data, that is, to assist in determining whether the operator is away from the first edge side area or the second edge side area during the detection period;

[0041] The second behavior detection module operates only within the first edge area and / or the second edge area to determine whether the operator meets another behavior characteristic of approaching the shelf and then moving away from the shelf. If so, it indicates that the operator has approached the shelf and then moved away from the shelf, and a second behavior event is generated.

[0042] When both the second behavior event and the pressure change event occur, it means that the operator has taken and / or placed an item away from the shelf.

[0043] Furthermore, the effective range of the first behavior detection module' is much larger than the effective range of the second behavior detection module covering the front end of the shelf;

[0044] The priority of the first behavior detection module' is higher than that of the second behavior detection module, and the priority is allocated as follows:

[0045] (1) The first behavior detection module is triggered and generates a first behavior event. Within a certain period, the second behavior detection module is triggered and generates a second behavior event. At this time, the second behavior event becomes invalid, and the first behavior event is finally output as the triggering event.

[0046] (2) First, the second behavior detection module is triggered and a second behavior event is generated. Within a certain period, the first behavior detection module is triggered and a first behavior event is generated. At this time, the second behavior event becomes invalid, and the first behavior event is finally output as the triggering event.

[0047] (3) The second behavior detection module is triggered first and a second behavior event is generated. Within a certain period, the first behavior detection module is not triggered. At this time, the second behavior event is finally output as the triggering event.

[0048] (4) After the first behavior detection module 'is triggered and the first behavior event 'is generated, if the first behavior detection module 'is triggered again and the first behavior event 'is generated again within a certain period, the previous first behavior event 'will become invalid, and the last first behavior event 'will be output as the triggering event;

[0049] (5) After the second behavior detection module is triggered and the second behavior event is generated, if the second behavior detection module is triggered again and the second behavior event is generated again within a certain period, the previous second behavior event will be invalid, and the last second behavior event will be output as the triggering event.

[0050] Beneficial effects of the present invention:

[0051] 1. The present invention is based on two detection modules, the first behavior detection module M1′ and the second behavior detection module M2, which can accurately detect and generate behavior events of operators approaching and moving away from a certain smart shelf. Combined with the pressure change event of a certain smart shelf for auxiliary judgment, it can accurately judge whether the operator has moved away from the smart shelf after taking and / or putting operations on the smart shelf. It is easy to operate and promote, and has strong use value.

[0052] 2. The operator behavior recognition function can be realized at a low hardware and software cost. The tag recognition function can be automatically turned on and off according to the behavior recognition results, and the power consumption is relatively low.

[0053] 3. It can accurately determine whether the operator has completed the item picking and placing action on a certain shelf. After the operator completes the item picking and placing action, the shelf will be automatically controlled to perform tag recognition, so that there will be no simultaneous tag recognition on multiple shelves, and there will be no problem of radio frequency signal interference and crosstalk between shelves.

[0054] 4. After the operator completes the item placement action, the shelf can be automatically controlled to perform label recognition, and there will be no fluctuations in the item label being recognized as being placed in, taken out, placed in, etc. during the item placement process.

[0055] 5. It supports users to freely take and put items without any feeling, and the overall operation process is relatively simple. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 : Schematic diagram of the smart shelf in Example 1.

[0057] Figure 2 : Schematic diagram of a top view of a smart shelf in Example 1.

[0058] Figure 3 : Schematic diagram of the smart shelf and radar radiation area in Example 2

[0059] Figure 4 : Schematic diagram of a top view of multiple smart shelves in Example 2.

[0060] Figure 5 : Schematic diagram of another top view of multiple smart shelves in embodiment 2.

[0061] Figure 6 : Schematic diagram of another top view of multiple smart shelves in embodiment 2.

[0062] Figure 7 : A top view of the smart shelf in Example 3 and a schematic diagram of the radar radiation area.

[0063] Figure 8 : Schematic diagram of straight-line distance difference in Example 3.

[0064] Description of labels:

[0065] 1: Smart shelves; 2: Radar. DETAILED DESCRIPTION

[0066] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0067] Example 1:

[0068] A behavioral event recognition system, such as Figure 1 As shown, a smart shelf 1 is provided. The smart shelf 1 has four layers, each with a shelf for placing items. Pressure sensors are installed below the shelf to detect changes in the weight of items on the corresponding shelf in real time. When the pressure sensor data drops or increases suddenly, it indicates that the weight of the items on the shelf has changed, and an operation of placing or removing items may have occurred. At this time, a pressure change event E3 is generated. It should be noted that the smart shelf 1, the shelf, how to install the pressure sensors below the shelf, and how the pressure sensors detect changes in the weight of items on the shelf are all prior art and will not be described in detail here.

[0069] A radar 2 is installed at the top middle position of the smart shelf 1 for detecting the entry and exit behavior of operators in its irradiation area in real time and sending real-time distance data to the behavior event recognition system.

[0070] like Figure 2As shown, the radar 2 is model HLK-LD2410B, with an illumination angle of 120° and a maximum illumination distance L1 of 6 meters. For a scenario with only one smart shelf 1, there is no need to consider the problem of interference in the detection ranges of multiple radars. The smart shelf 1 will form a fan-shaped area at the front end of the smart shelf 1 with the farthest illumination distance of the radar 2, that is, the fan-shaped area at the front end of the smart shelf 1 formed by the radar 2 with the farthest illumination distance L1 as the radius is the boundary area Z1;

[0071] Simply put, for a single smart shelf 1 scenario, the entire process of an operator expecting to pick up and place an item in front of smart shelf 1 typically involves the following steps:

[0072] (1) Start;

[0073] (2) Entering from outside the boundary zone Z1 into the boundary zone Z1;

[0074] (3) Move within the boundary area Z1 at the front end of the smart shelf 1 to pick up and place items;

[0075] (4) leaving from the boundary zone Z1;

[0076] (5) End;

[0077] From the perspective of the radar 2 illumination area, the above steps are Figure 2 The process from point P0 to point P1 and then to point P2.

[0078] Considering the complexity of the actual scenario, if the operator is already in the boundary zone Z1 before the radar 2 is powered on, the above (2) process of entering the boundary zone Z1 from outside the boundary zone Z1 may not be satisfied, that is, Figure 2 The process from point P0 to point P1 is shown. Considering the actual situation of picking up and placing items, the operator is allowed to move freely within the boundary area Z1 at the front of the smart shelf 1. However, as long as the operator eventually leaves the smart shelf 1, the process of leaving the boundary area Z1 mentioned above (4) will be satisfied, that is, Figure 2 The process from point P1 to point P2 is shown, which is a behavioral feature of the operator approaching the smart shelf 1 and then moving away from the smart shelf 1.

[0079] Therefore, the behavioral event recognition system is based on Figure 2 The process of the operator moving from point P1 to point P2 is shown, i.e., based on the change characteristics of leaving the inner boundary area Z1 in step (4), the behavior detection model M1 is generated. The behavior detection model M1 is used to determine whether the operator has approached the smart shelf 1 and then moved away from the smart shelf 1 to outside the inner boundary area Z1. If so, a first behavior event E1 is generated.

[0080] When both the first behavior event E1 and the pressure change event E3 occur, it means that the operator has taken and / or placed items away from the smart shelf 1. In this way, the behavior event recognition system will trigger the ultra-high frequency card reader to perform corresponding operations based on the behavior events that occurred, such as inventory operations.

[0081] Of course, in this embodiment, the behavior event recognition system triggering the UHF card reader to perform corresponding operations is not a technical problem to be solved by the present invention, so it will not be described in detail this time.

[0082] This embodiment uses the behavior detection model M1 to accurately detect and generate behavior events of an operator approaching and then moving away from a smart shelf, thereby accurately determining whether the operator has moved away from the smart shelf after performing a take and / or put operation on the smart shelf. Combined with the pressure change event for auxiliary judgment, it can accurately determine whether the operator has moved away from the smart shelf after performing a take and / or put operation on the smart shelf.

[0083] This embodiment uses a combination of hardware and software, with only radar 2 and a pressure sensor used as hardware, resulting in low hardware and software costs. Furthermore, the behavior event recognition system of this embodiment can automatically enable and disable tag recognition based on output behavior events, resulting in relatively low power consumption.

[0084] Example 2:

[0085] The above-mentioned embodiment 1 is a scenario with only one smart shelf 1, but in actual scenarios, multiple smart shelves 1 usually coexist and the channel gaps between smart shelves 1 are relatively limited. Therefore, directly using the boundary area Z1 divided by the maximum radar illumination distance L1 in the above-mentioned embodiment 1 as the trigger range of the target smart shelf 1 may be large, and will overlap with the trigger range of the radar 2 on other smart shelves 1, ultimately affecting the judgment accuracy, so the feasibility is not high.

[0086] Therefore, when multiple smart shelves 1 coexist, the problem of interference in the detection ranges of multiple radars 2 needs to be considered. If the solution of Example 1 is adopted, the problem of overlapping illumination ranges of the radars 2 will occur. Based on this, this embodiment optimizes and improves Example 1. The specific solution after the improvement is as follows:

[0087] When there are multiple smart shelves 1, Figure 3As shown, it is necessary to set a boundary distance threshold L2 at the front end of each smart shelf 1, and the boundary distance threshold L2 is less than the farthest illumination distance L1. The boundary distance threshold L2 divides the illumination space of the radar 2 at the front end of the smart shelf 1 into two fan-shaped areas, namely: the fan-shaped area at the front end of the smart shelf 1 formed by the radar 2 with the boundary distance threshold L2 as the radius is the inner boundary area Z2, and the fan-shaped area at the front end of the smart shelf 1 from the boundary distance threshold L2 to the farthest illumination distance L1 of the radar 2 is the outer boundary area Z1.

[0088] Taking into account the coexistence of multiple smart shelves 1, in order to avoid mutual interference in the detection range of the radar 2 between the smart shelves 1, the value of the boundary distance L2 is taken to cover as large a fan-shaped area as possible at the front end of the smart shelf 1, without causing the technical problem of overlapping of the boundary area Z2 of adjacent smart shelves 2.

[0089] like Figure 4-6 The radar 2 illumination area under different placement of multiple smart shelves 1. It can be seen from the figure that the boundary area Z2 illuminated by the radar 2 of each smart shelf 1 does not overlap with each other, and the boundary area Z2 basically covers the effective area at the front end of the smart shelf 1.

[0090] The boundary distance threshold L2 in this embodiment is set to 1.5 meters. The specific value is set according to actual conditions such as the height and gap width of the smart shelf 1.

[0091] like Figure 3 As shown, for a single smart shelf 1 scenario, the entire process of an operator expecting to pick up and place items in front of the smart shelf 1 generally involves the following steps:

[0092] (1) Start;

[0093] (2) From the outer boundary area Z1 to the inner boundary area Z2;

[0094] (3) Pick up and place items in front of smart shelf 1;

[0095] (4) Entering from the inner boundary area Z2 to the outer boundary area Z1;

[0096] (5) End;

[0097] From the perspective of the radar illumination area, the above steps are Figure 3 The process from point P3 to point P4 and then to point P5.

[0098] Considering the complexity of the actual scenario, if the operator is already in the boundary zone Z2 before the radar 2 is powered on, the above (2) process of entering the boundary zone Z2 from the boundary zone Z1 may not be satisfied, that is, Figure 3The process from point P3 to point P4 is shown.

[0099] Taking into account the actual situation of taking and placing items, the operator is allowed to move freely within the boundary area Z2 at the front of the smart shelf 1. However, as long as he finally leaves the smart shelf 1, the above process (4) of entering the boundary area Z2 from the boundary area Z1 will be satisfied, that is, Figure 3 The process from point P4 to point P5 is shown, which is a behavioral feature of the operator approaching the smart shelf 1 and then moving away from the smart shelf 1.

[0100] Therefore, the behavioral event recognition system is based on Figure 3 The process of the operator moving from point P4 to point P5 shown in the figure, that is, based on the distance data change characteristics collected from the entire process of entering the boundary area Z2 to the boundary area Z1 in the above step (4), the behavior detection model M1 is improved and optimized to generate the behavior detection model M1′.

[0101] The behavior detection model M1′ is used to determine whether the operator enters the outer boundary area Z1 from the inner boundary area Z2. If so, a behavior event E1′ is generated.

[0102] When both the behavior event E1′ and the pressure change event E3 occur, it means that the operator has taken and / or placed items away from the smart shelf.

[0103] This recognition system, through the behavior detection model M1′ and the events output by the pressure sensor, triggers the UHF card reader to perform corresponding operations, such as inventory. Of course, the behavior event recognition system in this embodiment triggers the UHF card reader to perform corresponding operations, which is not the technical problem to be solved by this embodiment, and therefore will not be described in detail here.

[0104] This embodiment uses the optimized behavior detection model M1′ to accurately detect and generate behavior events of an operator approaching and then moving away from a certain smart shelf 1, thereby accurately judging whether the operator has moved away from the smart shelf 1 after performing a take and / or put operation on the smart shelf 1. Combined with the pressure change event for auxiliary judgment, it can accurately judge whether the operator has moved away from the smart shelf 1 after performing a take and / or put operation on the smart shelf 1.

[0105] This embodiment can accurately determine whether the operator has completed the item picking and placing action on a certain smart shelf 1, and automatically control the smart shelf 1 to perform tag recognition after the operator completes the item picking and placing action, so that the problem of multiple smart shelves 1 performing tag recognition at the same time will not occur, and the problem of radio frequency signal interference and crosstalk between multiple smart shelves 1 will not occur.

[0106] This embodiment uses a combination of hardware and software, with only radar 2 and a pressure sensor used as hardware, resulting in low hardware and software costs. Furthermore, the behavior event recognition system of this embodiment can automatically enable and disable tag recognition based on output behavior events, resulting in relatively low power consumption.

[0107] Example 3:

[0108] Although the above embodiment 2 can effectively avoid the technical problem of overlapping of the radar 2 illumination areas by setting the boundary distance threshold L2, since the boundary distance threshold L2 is set and the boundary distance threshold L2 is less than the farthest illumination distance L1, it is very likely that the boundary area Z2 cannot completely cover the areas on both sides of the front end of the entire smart shelf 1. Figure 7 As shown in FIG, the edge side areas Z3 and Z4 on both sides of the front of the smart shelf 1 are the areas of the front of the smart shelf 1 that cannot be covered by the boundary area Z2. Then, when the operator goes to the edge side areas Z3 and Z4 to take and place items, Figure 7 In the process from point P6 to point P7 and then to point P8, we find that the entire process does not trigger behavior detection model M1′. In other words, behavior detection model M1′ alone cannot effectively detect all operator behaviors and ultimately cannot generate accurate trigger events. Therefore, this embodiment needs to further combine behavior detection model M2 for more accurate judgment.

[0109] Behavior detection model M2:

[0110] like Figure 8 As shown, after performing the pick-and-place operation in the edge area Z3 or Z4, the operator will eventually leave the smart shelf 1. Then, within a certain detection cycle, there will be a distance difference L4 between the two detection straight-line distances. For example, the straight-line distance L1_1 of the previous detection cycle is the straight-line distance from the far point of radar 2 to point P7, and the straight-line distance L1_2 of the next detection cycle is the straight-line distance from the far point of radar 2 to point P9. The distance difference L4 between the two is equal to the straight-line distance L1_2 minus the straight-line distance L1_1.

[0111] When the straight-line distance L1_2 of the next detection cycle is greater than the straight-line distance L1_1 of the previous detection cycle, that is, the distance difference L4>0 and L1_1>L2, L1_2>L2, and when the distance difference L4 is greater than the distance difference threshold L5, that is: the behavior detection model M2 is only effective within the edge area Z3 and Z4. This is another behavioral feature of the operator approaching smart shelf 1 and then moving away from smart shelf 1. Figure 8As shown in the figure, the process from point P6 to point P7, passing through point P9 and then to point P8, satisfies the above behavioral characteristics from point P7 to P9. Therefore, the behavioral event recognition system calculates and generates behavior detection model M2 based on the change characteristics of the distance data collected throughout this process. Behavior detection model M2 serves as a supplement to behavior detection model M1′ and is only effective outside the scope of behavior detection model M1′.

[0112] The size of the distance difference threshold L5 is set according to the detection period T1 and the average moving speed of the operator, and is used to screen out a set of distance data that meets the conditions from a large amount of real-time data, and confirm whether the operator's movement meets the distance index of the distance difference threshold L5 within the detection period. Only when this index is met can the distance data change characteristics be met, thereby avoiding the influence of interference data on the judgment result.

[0113] The purpose of setting the distance difference threshold L5 is to assist in determining whether the operator is away from the edge side area Z3 or the edge side area Z4 during the detection period.

[0114] The effective range of behavior detection module M1′ is much larger than that of behavior detection module M2, which covers the front end of smart shelf 1. Behavior detection module M2 operates within the effective range of edge areas Z3 and Z4, and is used to determine whether the operator approaches smart shelf 1 and then moves away from it, which is another behavioral feature of smart shelf 1. If so, it indicates that the operator has approached smart shelf 1 and then moved away from it, and generates behavior event E2.

[0115] When both the behavior event E2 and the pressure change event E3 occur, it means that the operator takes and / or places items in area Z3 or Z4 and then moves away from the smart shelf 1.

[0116] For example, at the start of a detection cycle T1, the operator is outside the bounded zone Z2. Radar 2 detects a straight-line distance L1_1 of 2.8 meters from the operator to the radar 2 origin. At the end of this detection cycle T1, radar 2 detects a straight-line distance L1_2 of 3.6 meters from the operator to the radar 2 origin. This distance difference L4 is L1_2 minus L1_1, or 0.8 meters. The preset distance difference threshold L5 is 0.7 meters. Since L4 > 0 and L4 > L5, the distance difference L4 corresponding to this detection cycle T1 meets the conditions for triggering behavior detection model M2, generating behavior event E2.

[0117] Similarly, if at the end time point of the detection cycle T1, radar 2 detects that the straight-line distance L1_2 between the operator and the origin of radar 2 is 3.1 meters, that is, the distance difference L4 is L1_2 minus L1_1, which is 0.3 meters. The preset distance difference threshold L5 is 0.7 meters. Since L4 > 0, but L4 < L5, the distance difference L4 corresponding to the detection cycle T1 does not meet the condition for triggering the behavior detection model M2, and the behavior event E2 will not be generated.

[0118] Similarly, if at the end time point of the detection cycle T1, radar 2 detects that the straight-line distance L1_2 between the operator and the origin of radar 2 is 1.9 meters, that is, the distance difference L4 is L1_2 minus L1_1. Since L4 is -0.9 meters and the preset distance difference threshold L5 is 0.7 meters, and L4 < 0, the distance difference L4 corresponding to the detection cycle T1 does not meet the condition for triggering the behavior detection model M2, and the behavior event E2 will not be generated.

[0119] The present recognition system will trigger the UHF card reader to perform corresponding operations, such as inventory operations, through the effective events output by the behavior detection model M1' or the behavior detection module M2 and the pressure sensor. Of course, triggering the UHF card reader by the behavior event recognition system in this embodiment to perform corresponding operations is not the technical problem to be solved in this embodiment, so it will not be described in detail this time.

[0120] Through the behavior detection model M1' and the behavior detection model M2 in this embodiment, it is possible to comprehensively, accurately and without dead angles determine whether the operator has the behavior of approaching and then leaving the intelligent shelf 1. Combined with the pressure sensor, it is possible to confirm whether the operator has the behavior of taking and / or placing items and then leaving the intelligent shelf 1.

[0121] Since this embodiment can accurately determine whether the operator has completed the item taking and placing action on a certain intelligent shelf 1, and will automatically control the intelligent shelf 1 to perform label recognition after the operator has completed the item taking and placing action, there will be no problem of multiple intelligent shelves 1 performing label recognition simultaneously, nor will there be a problem of radio frequency signal interference and crosstalk between multiple intelligent shelves 1.

[0122] In this embodiment, through a combination of software and hardware, only radar 2 and a pressure sensor are used hardware-wise, and the software and hardware costs are relatively low. Moreover, the behavior event recognition system in this embodiment can automatically turn on and off the label recognition function according to the output behavior event, and the power consumption is relatively low.

[0123] Embodiment 4:

[0124] Although the above-mentioned embodiment three can achieve comprehensive, accurate and no-dead-angle judgment on whether the operator has approached the smart shelf 1 and then moved away from the smart shelf 1, since the behavior detection model M1′ and the behavior detection model M2 are used at the same time, the scope covered by the behavior detection model M1′ is much larger than that of the behavior detection model M2. The coverage of the behavior detection model M2 is a supplement to the behavior detection model M1′. In fact, in most cases, only the behavior detection model M1′ will be triggered, and the scope of the behavior detection model M2 is outside the behavior detection model M1′.

[0125] Taking into account that in actual applications, operators usually move from the periphery of smart shelf 1 to the vicinity of smart shelf 1, then when approaching smart shelf 1 from the side, there is a possibility that they first enter the range of behavior detection model M2 and then enter the range of behavior detection model M1′. This will generate two behavior events continuously, which may trigger the behavior detection model M1′ and behavior detection model M2 at the same time. In fact, the operator only enters the behavior detection model M1′ from the behavior detection model M2, and then leaves the shelf within a certain period within the range of behavior detection model M1′. Therefore, priority must be set to ensure that only one behavior event is generated and output within a certain period.

[0126] The only difference between this embodiment and the third embodiment is that priorities are set for the behavior events generated by the behavior detection model M1′ and the behavior detection model M2, as follows:

[0127] (1) The behavior detection module M1′ is triggered first and generates the behavior event E1′. Within a certain period T2, the behavior detection module M2 is triggered and generates the behavior event E2. At this time, the behavior event E2 becomes invalid, and the behavior event E1′ is finally output as the triggering event.

[0128] (2) The behavior detection module M2 is triggered first and generates the behavior event E2. Within a certain period T2, the behavior detection module M1′ is triggered and generates the behavior event E1′. At this time, the behavior event E2 becomes invalid, and the behavior event E1′ is finally output as the triggering event.

[0129] (3) The behavior detection module M2 is triggered first and generates a behavior event E2. Within a certain period T2, the behavior detection module M1′ is not triggered. At this time, the behavior event E2 is finally output as the triggering event.

[0130] (4) After the behavior detection module M1′ is triggered and the behavior event E1′ is generated, within a certain period T2, the behavior detection module M1′ is triggered again and the behavior event E1′ is generated. Then the previous behavior event E1′ becomes invalid, and the last behavior event E1′ is output as the triggering event.

[0131] (5) After the behavior detection module M2 is triggered and the behavior event E2 is generated, within a certain period T2, the behavior detection module M2 is triggered again and the behavior event E2 is generated. Then, the previous behavior event E2 becomes invalid, and the last behavior event E2 is output as the triggering event.

[0132] At this point, based on the behavior detection module M1′ and the behavior detection module M2, the behavior events of the operator approaching and moving away from a smart shelf can be accurately detected and generated, and then combined with the pressure change event E3 of a smart shelf for auxiliary judgment, the operator's behavior of moving away from the smart shelf after taking and / or putting operations on the smart shelf can be accurately judged.

[0133] The event type finally output by a smart shelf in this embodiment, which triggers the system to perform the next operation, is as follows:

[0134] (1) When a behavior event E1′ or a behavior event E2 occurs and a pressure change event E3 also occurs, the behavior event recognition system triggers its tag recognition module to start the tag recognition operation;

[0135] (2) If the behavior event E1′ or behavior event E2 occurs but the pressure change event E3 does not occur, the behavior event recognition system will not trigger its tag recognition module to start the tag recognition operation;

[0136] (3) If neither behavior event E1′ nor behavior event E2 occurs, but pressure change event E3 occurs, the behavior event recognition system does not trigger its tag recognition module to start the tag recognition operation;

[0137] (4) If neither the behavior event E1′ nor the behavior event E2 occurs, nor the pressure change event E3 occurs, the behavior event recognition system does not trigger its tag recognition module to start the tag recognition operation.

[0138] Compared with the third embodiment, this embodiment sets priorities for the behavior events generated by the behavior detection model M1 ′ and the behavior detection model M2 , thereby ensuring that the behavior event recognition system only generates and outputs one behavior event within a certain period.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Therefore, although this specification has described the present invention in detail with reference to the above embodiments, it should be understood by those skilled in the art that the present invention can still be modified or replaced by equivalents. All technical solutions and improvements that do not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A behavior event recognition system, comprising a shelf, wherein the shelf has at least one layer, each layer is provided with a spacing area for placing items, and a pressure sensor is provided under each spacing area, wherein the pressure sensor is used to detect in real time whether the weight of the items in the corresponding area changes, and if so, a third pressure change event is generated, characterized in that: A radar, installed on the top of the shelf, for detecting the entry and exit behavior of operators in its irradiation area in real time and sending the real-time detected distance data to the behavior event recognition system; For the scenario of a single shelf, without considering the interference of multiple radar detection ranges, the shelf will form a fan-shaped area at the front of the shelf with the farthest irradiation distance of the radar, that is, the fan-shaped area at the front of the shelf formed by the radar with its farthest irradiation distance as the radius is the area within the boundary; The behavior event recognition system generates a first behavior detection module based on characteristic calculation of distance data changes during the entire process of the operator entering the area inside the boundary into the area outside the boundary; The first behavior detection module is used to determine whether the operator enters the area outside the boundary from the area inside the boundary, and if so, generate a first behavior event; When both the first behavior event and the third pressure change event occur, it means that the operator has taken and / or placed items away from the shelf.

2. The behavioral event recognition system according to claim 1, characterized in that: When there are multiple shelves coexisting, a boundary distance is set at the front end of each shelf, and the boundary distance divides the radar irradiation space at the front end of the shelf into two fan-shaped areas, that is, the fan-shaped area at the front end of the shelf formed by the radar irradiating with the boundary distance as the radius is the inner boundary area, and the fan-shaped area at the front end of the shelf from the boundary distance to the farthest irradiation distance of the radar is the outer boundary area; The value of the boundary distance covers as large a sector-shaped area as possible at the front end of the shelf, without causing the boundary areas of adjacent shelves to overlap; The behavior event recognition system optimizes the first behavior detection module based on the characteristics of the distance data change during the entire process of the operator entering the area inside the boundary into the area outside the boundary and generates a first behavior detection module'; The first behavior detection module' is used to determine whether the operator enters the area outside the boundary from the area inside the boundary, and if so, generate a first behavior event'; When both the first behavior event' and the pressure change event occur, it means that the operator has taken and / or placed items away from the shelf.

3. The behavioral event recognition system according to claim 2, characterized in that: When the area within the boundary does not completely cover the effective area at the front end of the shelf, that is, there are first edge side areas and / or second edge side areas on both sides of the front edge of the shelf that are not covered by the area within the boundary; if the operator moves in the first edge side area or the second edge side area, then within a certain detection period, there will be a distance difference between the straight-line distances of the operator detected by the radar before and after, and when the straight-line distance detected after is greater than the straight-line distance detected before, and the distance difference is greater than the distance difference threshold, the behavior event recognition system calculates and generates a second behavior detection module M2 based on the change characteristics of the above distance data; The size of the distance difference threshold is set according to the average moving speed of the operator in the detection period, and is used to filter out a set of distance data that meets the conditions from a large amount of real-time data, that is, to assist in determining whether the operator is far away from the first edge side area or the second edge side area in the detection period; The second behavior detection module acts only within the first edge side area and / or the second edge side area to determine whether the operator meets another behavior feature of approaching the shelf and then moving away from the shelf. If so, it means that the operator approaches the shelf and then moves away from the shelf, and generates a second behavior event; When both the second behavior event and the pressure change event occur, it means that the operator moves away from the shelf after taking and / or putting down an item.

4. The behavioral event recognition system according to claim 3, characterized in that: The effective range of the first behavior detection module' is much larger than the effective range of the second behavior detection module covering the front end of the shelf; The priority of the first behavior detection module' is higher than that of the second behavior detection module, and the priority distribution is: (1) The first behavior detection module 'is triggered first and generates a first behavior event '. Within a certain period, the second behavior detection module is triggered and generates a second behavior event. At this time, the second behavior event becomes invalid, and the first behavior event 'is finally output as the triggering event. (2) The second behavior detection module is triggered first and a second behavior event is generated. Within a certain period, the first behavior detection module is triggered and a first behavior event is generated. At this time, the second behavior event becomes invalid and the first behavior event is finally output as the triggering event. (3) The second behavior detection module is triggered first and a second behavior event is generated. Within a certain period, the first behavior detection module is not triggered. At this time, the second behavior event is finally output as a triggering event. (4) After the first behavior detection module 'is triggered and the first behavior event 'is generated, if the first behavior detection module 'is triggered again and the first behavior event 'is generated again within a certain period, the previous first behavior event 'will become invalid, and the last first behavior event 'will be output as the triggering event; (5) After the second behavior detection module is triggered and the second behavior event is generated, if the second behavior detection module is triggered again and the second behavior event is generated again within a certain period, the previous second behavior event becomes invalid, and the last second behavior event is output as the triggering event.

Images