Abandoned items detection and analysis system and method based on human behavior recognition

Abstract

The present invention relates to a system and method for detecting and analyzing left-behind articles based on human behavior recognition, and belongs to the field of electronic digital data processing technology. By designing an orderly coordinated action plan among a door-opening detection device, an article-carrying detection device, a public position detection device, a stay time detection device, a store-leaving analysis and judgment device, a target article recognition device, and an article-left warning device (i.e., the orderly execution of steps S1-S6), customers who carry articles and are prone to leaving articles behind in unattended stores can be screened out, and such customers can be warned when they leave articles behind. By designing a gaze direction detection device, it is determined whether the customer's gaze direction matches the direction of the public rules of the unattended store, thereby excluding the situation that "the customer may just accidentally stand at the location of the public rules, not reading the rules; even if the customer stays for a preset time, it is possible that he is playing with his mobile phone at that location."

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic digital data processing, and in particular relates to a system and method for detecting and analyzing leftover items based on human behavior recognition. Background Art

[0002] At present, since there are no staff on duty in unattended stores, users are required to consciously abide by the shopping rules in unattended stores. Among them, customers who have completed shopping in unattended stores are more familiar with the shopping rules, while customers who have not completed shopping in unattended stores may need to spend more time studying the shopping rules of unattended stores. When customers stay in unattended stores for a long time, they will pay more attention to the unattended stores, and their attention to themselves will be greatly reduced. If customers carry items with them, they are very likely to lose them in the unattended stores. At this stage, unattended stores do not have detection and analysis solutions for the above abnormal situations.

[0003] Therefore, at this stage, it is necessary to design a system and method for detecting and analyzing leftover items based on human behavior recognition to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide a system and method for detecting and analyzing left-behind items based on human behavior recognition, so as to solve the technical problems existing in the above-mentioned prior art. When customers stay in an unattended store for a long time, they will pay more attention to the unattended store, and their attention to themselves will be greatly reduced. If customers carry items with them, they are very likely to lose them in the unattended store. However, the current unattended stores do not have detection and analysis solutions for the above-mentioned abnormal situations.

[0005] To achieve the above object, the technical solution of the present invention is:

[0006] The method for detecting and analyzing leftover items based on human behavior recognition includes the following steps:

[0007] S1, detect whether the door control of the unattended store is open, if it is open, go to step S2;

[0008] S2, detecting whether the customer carries the target item, if so, proceeding to step S3;

[0009] S3, detecting whether the customer is at the location where the unattended store publicizes the rules. If the customer is at the location where the unattended store publicizes the rules, proceeding to step S4;

[0010] S4, detecting whether the customer's stay time reaches the preset time. If the customer's stay time reaches the preset time, proceeding to step S5;

[0011] S5, analyzing and determining whether the customer's actions indicate a trend of leaving the store. If the customer's actions indicate a trend of leaving the store, proceeding to step S6;

[0012] S6. Identify whether the target item is in the customer's hand. If the target item is in the customer's hand, issue a warning that the item is left behind.

[0013] Preferably, step S3 further includes a gaze direction detection step, which is specifically as follows:

[0014] When it is detected that the customer is at the location of the unattended store's public rules, it is synchronously detected whether the customer's gaze direction matches the direction of the unattended store's public rules. If the customer's gaze direction matches the direction of the unattended store's public rules, step S4 is entered.

[0015] Preferably, in step S1, when the access control is in automatic sensing opening mode;

[0016] Step S5 includes a moving direction detection step, a moving speed detection step, and a distance detection step;

[0017] The moving direction detection step specifically includes: detecting whether the moving direction of the customer matches the direction of the access control of the unattended store;

[0018] The moving speed detection step specifically includes: detecting whether the moving speed of the customer reaches a preset speed;

[0019] The distance detection step specifically includes: detecting whether the real-time distance between the customer and the door control of the unmanned store reaches a preset distance.

[0020] Preferably, in step S6, the target object is identified by means of cameras disposed on the left and right sides of the access control system.

[0021] Preferably, in step S1, when the access control is manually opened;

[0022] Step S5 includes a distance detection step and a handle contact detection step;

[0023] The distance detection step specifically includes: detecting whether the real-time distance between the customer and the door control of the unattended store reaches a preset distance;

[0024] The handle contact detection step is specifically used to detect whether the customer touches the door handle.

[0025] Preferably, in step S6, the target object is identified by cameras distributed on the left and right sides of the access control system and a camera facing the access control system.

[0026] The left-behind item detection and analysis system based on human behavior recognition comprises a door opening detection device, an item carrying detection device, a public position detection device, a stay time detection device, a store leaving analysis and judgment device, a target item recognition device, an item left-behind warning device, and a main control device; the door opening detection device is used to detect whether the door of an unattended store is opened; the item carrying detection device is used to detect whether a customer is carrying a target item in his hand; the public position detection device is used to detect whether a customer is located at the location of the public position rules of the unattended store; the stay time detection device is used to detect whether the customer's stay time reaches a preset time; the store leaving analysis and judgment device is used to analyze and judge whether the customer's action has a trend of leaving the store; the target item recognition device is used to recognize whether the target item is carried in the customer's hand; the item left-behind warning device is used to warn of item left-behind; wherein, the main control device controls the initial state of the door opening detection device to be normally open, controls the item carrying detection device, the public position detection device, the stop The initial states of the stay duration detection device, the store departure analysis and judgment device, the target item identification device, and the item left behind warning device are normally closed; when the door opening detection device detects that the door of the unattended store is opened, the main control device controls the item carrying detection device to be turned on; when the item carrying detection device detects that the customer is carrying the target item in his hand, the main control device controls the public position detection device to be turned on; when the public position detection device detects that the customer is located at the location of the public rule of the unattended store, the main control device controls the stay duration detection device to be turned on; when the stay duration detection device detects that the customer's stay duration reaches a preset duration, the main control device controls the store departure analysis and judgment device to be turned on; when the store departure analysis and judgment device analyzes and judges that the customer's action has a tendency to leave the store, the main control device controls the target item identification device to be turned on; when the target item identification device recognizes that the target item is not carried in the customer's hand, the main control device controls the item left behind warning device to be turned on.

[0027] Preferably, it also includes a wireless communication device and a cloud server, and the main control device is connected to the cloud server network via the wireless communication device.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] One of the beneficial effects of this solution is that by designing an orderly coordinated action plan among the door opening detection device, the item carrying detection device, the public location detection device, the stay time detection device, the store exit analysis and judgment device, the target item identification device, and the item left warning device, customers who carry items in unattended stores and are prone to leaving items behind can be screened out, and such customers can be warned when they leave items behind. By designing a gaze direction detection device to determine whether the customer's gaze direction matches the direction of the public rules of the unattended store, it is ruled out that "the customer may just accidentally stand at the location of the public rules, not reading the rules; even if the customer stays for a preset time, it is possible that he is playing with his mobile phone at that location." BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the method steps for implementing this scheme.

[0031] Figure 2 This is a schematic diagram of the specific method steps of S5 when the access control is opened by automatic sensing in the implementation manner of this solution.

[0032] Figure 3 This is a schematic diagram of the specific method steps of S5 when the access control is manually opened in the implementation manner of this solution. DETAILED DESCRIPTION

[0033] Embodiment 1:

[0034] like Figure 1 As shown, a left-behind item detection and analysis system based on human behavior recognition is proposed, including a door opening detection device, an item carrying detection device, a public location detection device, a stay time detection device, a store departure analysis and judgment device, a target item recognition device, an item left-behind warning device, and a main control device; the main control device is respectively connected to the door opening detection device, the item carrying detection device, the public location detection device, the stay time detection device, the store departure analysis and judgment device, the target item recognition device, and the item left-behind warning device;

[0035] The door opening detection device is used to detect whether the door of an unattended store is opened;

[0036] The object carrying detection device is used to detect whether the customer is carrying the target object;

[0037] The public display location detection device is used to detect whether the customer is located at the location where the unattended store publicizes the rules;

[0038] The stay time detection device is used to detect whether the customer's stay time reaches a preset time;

[0039] The store leaving analysis and judgment device is used to analyze and judge whether the customer's actions have a store leaving trend;

[0040] The target item recognition device is used to recognize whether the target item is carried by the customer;

[0041] The item left behind warning device is used to warn of item left behind;

[0042] Among them, Figure 2 As shown, the main control device controls the initial state of the door opening detection device to be normally open, and controls the initial state of the item carrying detection device, the public location detection device, the stay time detection device, the store departure analysis and judgment device, the target item recognition device, and the item left warning device to be normally closed;

[0043] When the door opening detection device detects that the door of the unattended store is opened, the main control device controls the article carrying detection device to be turned on; using the door opening detection device as a triggering link of the entire system can avoid malfunction of the system;

[0044] When the item carrying detection device detects that the customer is carrying the target item, the main control device controls the public position detection device to turn on; this can exclude the situation where the customer does not carry the target item, and the target item is the item carried by the customer;

[0045] When the posted location detection device detects that the customer is at the location of the posted rules of the unattended store, the main control device controls the stay time detection device to start; customers who are not familiar with the posted rules of the unattended store can be further screened out, and such customers are more likely to be distracted and leave items behind;

[0046] When the stay time detection device detects that the customer's stay time reaches the preset time, the main control device controls the store departure analysis and judgment device to start; since the customer may just stand at the location of the publicized rules by chance, and not read the rules, the stay time detection device is designed for further judgment; wherein the preset time can be set according to the time required for reading the publicized rules;

[0047] When the store leaving analysis and judgment device analyzes and judges that the customer's action has a tendency to leave the store, the main control device controls the target object identification device to turn on;

[0048] When the target item recognition device recognizes that the target item is not carried by the customer, the main control device controls the item left behind warning device to turn on.

[0049] Similarly, the method for detecting and analyzing leftover items based on human behavior recognition corresponding to the system for detecting and analyzing leftover items based on human behavior recognition comprises the following steps:

[0050] S1, detect whether the door control of the unattended store is open, if it is open, go to step S2;

[0051] S2, detecting whether the customer carries the target item, if so, proceeding to step S3;

[0052] S3, detecting whether the customer is at the location where the unattended store publicizes the rules. If the customer is at the location where the unattended store publicizes the rules, proceeding to step S4;

[0053] S4, detecting whether the customer's stay time reaches the preset time. If the customer's stay time reaches the preset time, proceeding to step S5;

[0054] S5, analyzing and determining whether the customer's actions indicate a trend of leaving the store. If the customer's actions indicate a trend of leaving the store, proceeding to step S6;

[0055] S6. Identify whether the target item is in the customer's hand. If the target item is in the customer's hand, issue a warning that the item is left behind.

[0056] In this embodiment, by designing an orderly coordinated action plan among the door opening detection device, the item carrying detection device, the public location detection device, the stay duration detection device, the store exit analysis and judgment device, the target item recognition device, and the item left behind warning device (i.e., the orderly execution of steps S1-S6), customers who carry items and are prone to leaving items behind in unattended stores can be screened out, and warnings can be issued when such customers leave items behind.

[0057] Embodiment 2:

[0058] In the above scheme, it is taken into consideration that the customer may just happen to stand at the location where the rules are posted, and not read the rules; even if the customer stays at the location for a preset time, it is possible that he or she is playing with a mobile phone at that location.

[0059] On the basis of embodiment 1, further comprising a gaze direction detection device, wherein the gaze direction detection device is connected to the main control device;

[0060] The gaze direction detection device is used to detect whether the gaze direction of the customer matches the direction of the publicized rules of the unattended store;

[0061] Wherein, the main control device controls the initial state of the gaze direction detection device to be normally closed;

[0062] When the posted position detection device detects that the customer is located at the posted rule position of the unattended store, the main control device controls the gaze direction detection device to turn on;

[0063] When the gaze direction detection device detects that the gaze direction of the customer matches the direction of the public notice rule of the unattended store, the main control device controls the stay duration detection device to turn on.

[0064] In the above scheme, a gaze direction detection device is designed to determine whether the customer's gaze direction matches the direction of the public rules of the unmanned store, thereby eliminating the above abnormal situation.

[0065] Embodiment 3:

[0066] like Figure 2 As shown, based on Example 2, when the door access control is in automatic sensing opening mode, the store departure analysis and judgment device includes a moving direction detection device, a moving speed detection device, and a distance detection device;

[0067] The moving direction detection device is used to detect whether the moving direction of the customer matches the direction of the door control of the unattended store;

[0068] The moving speed detection device is used to detect whether the moving speed of the customer reaches a preset speed;

[0069] The distance detection device is used to detect whether the real-time distance between the customer and the door control of the unattended store reaches a preset distance;

[0070] When the main control device controls the store departure analysis and judgment device to turn on, the moving direction detection device and the moving speed detection device are started first. If the customer's moving direction matches the direction of the unmanned store's access control and the customer's moving speed reaches a preset speed, the distance detection device is turned on. When the real-time distance between the customer and the unmanned store's access control reaches a preset distance, it is judged that the customer has a trend of leaving the store.

[0071] Furthermore, the target object recognition device is a camera device distributed on the left and right sides of the access control system, which recognizes the customer's hands from the left and right sides of the access control system.

[0072] Embodiment 4:

[0073] like Figure 3 As shown, based on Example 2, when the door is manually opened, the store departure analysis and judgment device includes a distance detection device and a handle contact detection device;

[0074] The distance detection device is used to detect whether the real-time distance between the customer and the door control of the unattended store reaches a preset distance;

[0075] The handle contact detection device is used to detect whether a customer touches the door handle.

[0076] Furthermore, the target object identification device includes camera devices distributed on the left and right sides of the door control and a camera device facing the door control. Since the customer may pass through the door control sideways when the door control is manually opened, the camera devices distributed on the left and right sides of the door control and a camera device facing the door control are designed, that is, three detection directions.

[0077] Furthermore, it also includes a wireless communication device and a cloud server. The main control device is connected to the cloud server network through the wireless communication device to achieve remote data interaction.

Claims

1. A method for detecting and analyzing leftover items based on human behavior recognition, characterized in that: The following steps are involved: S1, detect whether the door control of the unattended store is open, if it is open, go to step S2; S2, detecting whether the customer carries the target item, if so, proceeding to step S3; S3, detecting whether the customer is at the location where the unattended store publicizes the rules. If the customer is at the location where the unattended store publicizes the rules, proceeding to step S4; S4, detecting whether the customer's stay time reaches the preset time. If the customer's stay time reaches the preset time, proceeding to step S5; S5, analyzing and determining whether the customer's actions indicate a trend of leaving the store. If the customer's actions indicate a trend of leaving the store, proceeding to step S6; S6. Identify whether the target item is in the customer's hand. If the target item is not in the customer's hand, issue a warning that the item is left behind.

2. The method for detecting and analyzing leftover items based on human behavior recognition according to claim 1 is characterized in that: Step S3 also includes a gaze direction detection step, which is as follows: When it is detected that the customer is at the location of the unattended store's public rules, it is synchronously detected whether the customer's gaze direction matches the direction of the unattended store's public rules. If the customer's gaze direction matches the direction of the unattended store's public rules, step S4 is entered.

3. The method for detecting and analyzing leftover items based on human behavior recognition according to claim 1 is characterized in that: In step S1, when the access control is in automatic sensing opening mode; Step S5 includes a moving direction detection step, a moving speed detection step, and a distance detection step; The moving direction detection step specifically includes: detecting whether the moving direction of the customer matches the direction of the access control of the unattended store; The moving speed detection step specifically includes: detecting whether the moving speed of the customer reaches a preset speed; The distance detection step specifically includes: detecting whether the real-time distance between the customer and the door control of the unmanned store reaches a preset distance.

4. The method for detecting and analyzing leftover items based on human behavior recognition according to claim 3 is characterized in that: In step S6, the target object is identified by the camera devices distributed on the left and right sides of the access control system.

5. The method for detecting and analyzing leftover items based on human behavior recognition according to claim 1 is characterized in that: In step S1, when the access control is manually opened; Step S5 includes a distance detection step and a handle contact detection step; The distance detection step specifically includes: detecting whether the real-time distance between the customer and the door control of the unattended store reaches a preset distance; The handle contact detection step is specifically used to detect whether the customer touches the door handle.

6. The method for detecting and analyzing leftover items based on human behavior recognition according to claim 5 is characterized in that: In step S6, the target object is identified by the camera devices distributed on the left and right sides of the access control and the camera device facing the access control.

7. A system for detecting and analyzing leftover items based on human behavior recognition, characterized in that: It includes a door opening detection device, an item carrying detection device, a public position detection device, a stay time detection device, a store leaving analysis and judgment device, a target item recognition device, an item left warning device, and a main control device; the door opening detection device is used to detect whether the door of the unattended store is opened; the item carrying detection device is used to detect whether the customer is carrying the target item; the public position detection device is used to detect whether the customer is located at the location of the unattended store public rules; the stay time detection device is used to detect whether the customer's stay time reaches the preset time; The store leaving analysis and judgment device is used to analyze and judge whether the customer's actions have a store leaving trend; The target item identification device is used to identify whether the target item is carried by the customer; the item left behind warning device is used to warn of item left behind; wherein, the main control device controls the initial state of the door opening detection device to be normally open, and controls the initial states of the item carrying detection device, the public location detection device, the stay time detection device, the store departure analysis and judgment device, the target item identification device, and the item left behind warning device to be normally closed; when the door opening detection device detects that the door of the unattended store is opened, the main control device controls the item carrying detection device to be turned on; When the item carrying detection device detects that the customer is carrying the target item, the main control device controls the public position detection device to turn on; when the public position detection device detects that the customer is located at the location of the public notice rule of the unmanned store, the main control device controls the stay time detection device to turn on; when the stay time detection device detects that the customer's stay time reaches a preset time, the main control device controls the store departure analysis and judgment device to turn on; when the store departure analysis and judgment device analyzes and judges that the customer's action has a tendency to leave the store, the main control device controls the target item recognition device to turn on; when the target item recognition device recognizes that the target item is not carried by the customer, the main control device controls the item left behind warning device to turn on.

8. The system for detecting and analyzing leftover items based on human behavior recognition according to claim 7 is characterized in that: It also includes a wireless communication device and a cloud server, and the main control device is connected to the cloud server network through the wireless communication device.

Images