Mailing behavior identification and analysis system, method, device and storage medium based on monitoring screen
By integrating monitoring screen recognition and analysis systems in express cabinets, identifying user behavior and recommending appropriate interfaces and cabinets, the problem of difficulty in actively adjusting the operating interface and recommended cabinets is solved, and the delivery efficiency and user experience are improved.
Patent Information
- Application Number
- CN202411406801.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-10-10
AI Technical Summary
In the prior art, express cabinets lack a shipping behavior recognition and analysis system, which makes it difficult for the surveillance camera to actively adjust the operating interface and recommend appropriate cabinets, affecting the shipping efficiency and user experience.
A system of shipping behavior recognition and analysis based on surveillance screen was designed, including multiple express cabinets, surveillance cameras, touch panels, data synchronization modules, monitoring identification modules, cabinet recommendation modules and information analysis modules, through human contours and humans Face recognition distinguishes user types and recommends appropriate interfaces and cabinets, synchronizes user information, and optimizes user operation processes.
It improves user experience, reduces user operation steps, optimizes shipping efficiency, realizes automated pick-up and delivery, and reduces manual participation.
Smart Images

Figure CN118918533B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition systems, and in particular to a system, method, device and storage medium for identifying and analyzing mailing behavior based on monitoring images. Background Art
[0002] The mailing behavior recognition and analysis system based on surveillance images is a system that uses modern computer vision technology and artificial intelligence algorithms to intelligently analyze images taken by surveillance cameras to identify and analyze mailing behaviors. The system mainly relies on deep learning algorithms and realizes automatic recognition and analysis of mailing behaviors through training on a large amount of mailing behavior video data.
[0003] The existing mailing behavior recognition and analysis systems are generally used in express delivery stations, express post stations and other mailing sites. With the introduction of express lockers, the mailing points are no longer restricted by the geographical location of the storefront, the size of the space and the opening hours, which effectively improves the convenience of mailing. At the same time, the existing mailing sites currently need to be connected by personnel to determine whether the visitor is here to send or pick up a package, and the express lockers in the prior art lack corresponding mailing behavior recognition and analysis systems. Although they are equipped with surveillance cameras, the surveillance cameras are generally only used to record mailing behavior to prevent the loss or wrong collection of packages. It is difficult to adjust the operation interface for users according to the user's mailing behavior patterns and surveillance images, and it is difficult to actively recommend suitable cabinets to users, which affects the mailing efficiency and the user experience is poor, and there are certain defects. Summary of the invention
[0004] In view of the above-mentioned shortcomings of the prior art, the present invention provides a mailing behavior identification and analysis system, method, device and storage medium based on monitoring images, which can effectively solve the problem that the express cabinets in the prior art are difficult to actively adjust the operation interface for users and recommend suitable cabinet openings.
[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:
[0006] The present invention provides a mailing behavior identification and analysis system based on monitoring images, which comprises at least a plurality of express cabinets, wherein the express cabinets are provided with a plurality of slots for mailing and receiving, each of the slots is provided with an electric control cabinet door, the express cabinets are provided with a monitoring camera, and the express cabinets are provided with a touch panel for inputting information and controlling the opening and closing of the electric control cabinet door under the monitoring camera, and further comprises: a data synchronization module, a monitoring identification module, a cabinet slot recommendation module, and an information analysis module;
[0007] The express parcels in the express cabinet waiting for the user to sign for and take away are recorded as received parcels, and the express parcels in the express cabinet waiting for the courier to pick up and send out are recorded as sent parcels. The data synchronization module obtains the recipient and sender information of the received parcels and the sent parcels. After the user enters the recipient information, the electric control cabinet door of the corresponding compartment where the received parcel is located is controlled to open. After the user enters the sender information and the recipient, the user selects one of the compartments and controls the electric control cabinet door to open. The data synchronization module synchronously uploads all the recipient and sender information to the database;
[0008] The monitoring identification module includes a user identification unit and an object identification unit. The user identification unit identifies the human body contour and face data of the person entering the monitoring range and analyzes them, marking them as pending users, and marking pending users who stay for more than a set time as target users. The object identification unit analyzes the object held by the target user, marks them as a sender or a recipient, and recommends displaying the corresponding sending interface or receiving interface on the touch panel;
[0009] The cabinet opening recommendation module further analyzes the human body contour in the monitoring image and recommends suitable cabinet openings for the target user;
[0010] The information analysis module analyzes the recipient and sender information of each user, and recommends the corresponding sending interface or receiving interface to the user when the user's face is recognized based on the receiving and sending rules of each user.
[0011] Furthermore, the user identification unit includes a human body contour recognition model that can recognize the human body contour in the surveillance video, the human body contour in the surveillance picture is recognized by the human body contour recognition model and recorded as the target contour, the head area of the target contour is subjected to face recognition to obtain face data, and the same face data and its corresponding target contour are recorded as the pending user;
[0012] The face area ratio of the pending user in the monitoring screen and the corresponding target contour area ratio are obtained, and a face ratio threshold and a contour ratio threshold are set respectively. When the face area ratio and the target contour area ratio are both greater than or equal to the corresponding face ratio threshold and contour ratio threshold, the monitoring screen is regularly captured. A fixed height mark is set in the monitoring screen, and the corresponding height of the fixed height mark is 0.6m. The part below the fixed height mark of the instantaneous image of the monitoring screen captured regularly is recorded as the captured image. The captured images are compared for similarity. When the similarity between multiple consecutive captured images is greater than the preset similarity threshold, the pending user is marked as the target user.
[0013] Furthermore, the object recognition unit analyzes and recommends the target user as follows:
[0014] In the process of intercepting and judging the monitoring screen of the target user, the human body contour area value of the target user is obtained in real time, and a line graph of the human body contour area value changing with time is drawn, and the instantaneous image corresponding to the maximum value of the human body contour area value in the line graph is selected as the target image, and the human body contour and face data in the target image are obtained, and two eye contours in the face data are obtained, and an eye distance line segment is established with the two closest points on the adjacent sides of the two eye contours, and a straight line perpendicular to the horizontal plane passing through the midpoint of the eye distance line segment is constructed and recorded as a contour bisector, and the human body contour of the target user is divided into two parts by the contour bisector and recorded as ,calculate The absolute value of the difference in area between the two parts and divided by The smaller value in the area difference ratio , the calculation formula is ,in express The smaller value in the area difference threshold is preset. When the area difference ratio between the two parts is less than or equal to the area difference threshold, the target user is recorded as the recipient user, otherwise, he is recorded as the sender user;
[0015] When the target user is a sender, the first-level interface of the touch panel is adjusted to a sender interface; when the target user is a receiver, the first-level interface of the touch panel is adjusted to a receiver interface.
[0016] Furthermore, the analysis process of the cabinet recommendation module is as follows:
[0017] According to the interface where the target user finally chooses to operate, the current behavior is recorded as a sending behavior or a receiving behavior. When the target user's current behavior is recorded as a sending behavior, the target user's The absolute value of the area difference between the two parts is recorded as , substitute into the formula The first limit value is calculated by ;
[0018] The cabinet recommendation module is located on both sides of the express cabinet and is equipped with auxiliary cameras for capturing the side image of the target user. When the user's current behavior is marked as a mailing behavior, the side image of the target user is captured and recorded as a side view. The maximum height of the target user's human body outline in the side view is obtained and recorded as h. A curve of the change of the horizontal width w of the target user's human body outline in the side view versus the vertical height is drawn, and the maximum value of the horizontal width w within the height interval [0.4h, 0.7h] is obtained and recorded as the second limit value ;
[0019] Get the length, width, and height of the grid selected by the target user, recorded as ,choose The minimum value in With the first limit value and the second limit value The minimum value in Compare and choose The maximum value in With the first limit value and the second limit value The maximum value in For comparison, ≤ or ≤ When the grid size is not appropriate, an instruction is generated to prompt the target user to select a larger grid size and recommend ≥ and ≥ The grid.
[0020] Furthermore, the information analysis module recommendation process is specifically as follows:
[0021] Assign a user label to each unique face data , obtain the sender information of the package corresponding to the sending behavior and bind it with the corresponding facial data, obtain the recipient information of the package corresponding to the receiving behavior and bind it with the corresponding facial data, each user label corresponds to multiple sender information and multiple recipient information, and the multiple sender information and multiple recipient information together constitute the information data set corresponding to the user label. When the information data set of the same user label only contains the sender information, the user label is recorded as the sending label. Similarly, when the information data set of the same user label only contains the recipient information, the user label is recorded as the receiving label;
[0022] When the facial data of the shipping label is the same as the facial data of the target user, when the target user is the sender, the first-level interface of the touch panel is adjusted to the sending interface. When the facial data of the receiving label is the same as the facial data of the target user, when the target user is the recipient, the first-level interface of the touch panel is adjusted to the receiving interface.
[0023] Furthermore, the present invention also provides a method for identifying and analyzing mailing behaviors based on monitoring screens, which is applied to the mailing behaviors identification and analysis system based on monitoring screens, and the specific operating steps are as follows:
[0024] Step 1: The monitoring identification module includes a user identification unit and an object identification unit. The user identification unit performs image analysis on people entering the monitoring range and marks them as pending users. The pending users who stay for more than a set time are marked as target users.
[0025] Step 2: The object recognition unit analyzes the human body contour of the target user, determines whether the target user holds an object and marks the target user as a sender or a receiver, and recommends displaying a corresponding sender interface or a receiver interface on the touch panel;
[0026] Step 3: The data synchronization module extracts the recipient and sender information of the received or sent package and uploads it to the database. The monitoring and recognition module obtains the facial data of the corresponding recipient and sender. The recipient and sender are collectively referred to as users.
[0027] Step 4: The information analysis module binds the recipient and sender information of the received or sent package with the facial data of the corresponding user, and assigns a user label to each facial data. The multiple sender information and multiple recipient information of each user label together constitute the information data set of the corresponding user label. When the information data set of the same user label only contains the sender information, the user label is recorded as the sending label. When the information data set of the same user label only contains the recipient information, the user label is recorded as the receiving label.
[0028] Step 5: When the facial data of the shipping label is the same as the facial data of the target user, the target user is the sender and the first-level interface of the touch panel is adjusted to the sending interface. When the facial data of the receiving label is the same as the facial data of the target user, the target user is the recipient and the first-level interface of the touch panel is adjusted to the receiving interface.
[0029] The present invention also provides a device for identifying and analyzing mailing behavior based on monitoring screens, including a processor, a memory, a communication bus, a user interface, and a network interface. The communication bus is used to realize connection and communication between these components. The user interface is connected to the touch panel. The network interface includes a wired interface and a wireless interface. The memory includes a random access memory and a non-volatile memory. A computer program is stored in the memory. When the computer program is executed by the processor, the above-mentioned method for identifying and analyzing mailing behavior based on monitoring screens is realized.
[0030] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for identifying and analyzing mailing behavior based on monitoring images.
[0031] Compared with the known prior art, the technical solution provided by the present invention has the following beneficial effects:
[0032] 1. The data synchronization module of the present invention can timely and synchronously upload the recipient and sender information of the user, and the monitoring identification module can distinguish the target user from the pending users, and screen out the users who stay in front of the express cabinet from all the users who enter the monitoring range, and then conduct further behavioral analysis on them, eliminate the passers-by who pass by in the monitoring screen without sending or receiving the parcels, omit the subsequent analysis steps, and judge whether the user is holding an item by analyzing the degree of symmetry on both sides of the human body outline in the monitoring screen, so as to judge the user's intention to send or pick up the parcel, and recommend the corresponding interface to the user, effectively improve the user experience, reduce the user's unnecessary operations, and optimize the use process. At the same time, the express cabinet can also be applied to express stations, which can also reduce manpower to a certain extent and realize automatic collection and delivery of express parcels.
[0033] 2. The cabinet opening recommendation module of the present invention can further preliminarily calculate the volume parameters of the object according to the size of the image of the object held by the user, thereby recommending a compartment opening for the target user, which can help the target user select a suitable compartment opening that can accommodate the parcel, reduce the trial and error time of the target user in the process of selecting the compartment opening, improve the target user's mailing experience, and optimize the target user's mailing efficiency.
[0034] 3. The information analysis module of the present invention analyzes the recipient and sender information of each user, and distinguishes the target user through face recognition technology according to the receiving and sending rules of each user before each target user prepares to operate the express cabinet, and makes preparations for the target user in advance according to the sending and receiving habits of the target user, and intelligently distinguishes the displayed first-level interface to facilitate the operation of the target user, thereby further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0036] Figure 1 It is the overall module block diagram of the present invention;
[0037] Figure 2 It is a diagram of the method steps of the present invention;
[0038] Figure 3 It is a schematic diagram of the structure of the device of the present invention. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] The present invention will be further described below in conjunction with the embodiments.
[0041] See also Figure 1 The mailing behavior recognition and analysis system based on monitoring images at least includes: multiple express cabinets, each of which is provided with multiple slots for sending and receiving parcels, each slot is provided with an electric-controlled cabinet door, the express cabinet is provided with a monitoring camera, and the express cabinet is provided with a touch panel for entering information and controlling the opening and closing of the electric-controlled cabinet door under the monitoring camera, and also includes a data synchronization module, a monitoring recognition module, a cabinet port recommendation module, and an information analysis module.
[0042] The express parcels in the express cabinet waiting for the user to sign for and take away are recorded as received parcels, and the express parcels in the express cabinet waiting for the courier to pick up and send out are recorded as sent parcels.
[0043] The data synchronization module obtains the recipient and sender information of the received package or the sent package. The recipient and sender information include name, contact number and address. After the user enters the complete recipient information, the electric control cabinet door of the corresponding compartment where the received package is located is controlled to open, and the user completes the receipt. After the user enters the complete sender information and recipient, the user selects one of the compartments and controls the electric control cabinet door of the compartment to open. The user puts the package in and closes the cabinet door to complete the delivery. The data synchronization module synchronously uploads the recipient and sender information in any express cabinet to the database for subsequent analysis.
[0044] The monitoring identification module includes a user identification unit and an object identification unit. The user identification unit performs image analysis on people entering the monitoring range and marks them as pending users. Pending users who stay for more than a set time are marked as target users. The object identification unit analyzes the objects held by the target users and marks them as senders or recipients, and recommends (referring to priority display) the corresponding sending interface or receiving interface on the touch panel.
[0045] The user identification unit includes a human contour recognition model that can identify human contours in surveillance videos. The human contour in the surveillance image is identified through the human contour recognition model and recorded as the target contour. Face recognition is performed on the head area of the target contour to obtain face data (contour recognition has a higher priority than face recognition, thereby utilizing the fact that the human contour accounts for a large proportion of the image and preferentially distinguishing the human body through contour recognition before recognizing the human face, which is beneficial to improving recognition efficiency). The same face data and its corresponding target contour are recorded as pending users.
[0046] It should be noted that the human body contour recognition model is a mature image recognition technology in the existing technology. It mainly uses convolutional neural networks to recognize human body contours in multiple videos and perform deep algorithm learning, so as to realize the recognition of human body contours in dynamic videos. I will not go into details here. Face recognition technology is also an existing technology. It extracts key feature information from face images, such as facial contours, eyes, nose, mouth position, shape and size, etc., and encodes the extracted facial features into a specific format to obtain face data, such as face print coding (Faceprint), so as to realize face recognition and distinction between different people through face data.
[0047] Furthermore, the face area ratio value and the corresponding target contour area ratio value of the pending user in the monitoring screen are obtained (the ratio value is equal to the total number of pixels of the corresponding area image divided by the total number of pixels of the monitoring screen image), and a face ratio threshold value and a contour ratio threshold value are set respectively. When the face area ratio value and the target contour area ratio value are both greater than or equal to the corresponding face ratio threshold value and contour ratio threshold value (at this time, it means that the pending person is facing the monitoring camera and is very close to the express cabinet), the monitoring screen is regularly captured, and the capture interval is 1 second. A fixed height mark line is set in the monitoring screen, and the corresponding height is 0.6m (the monitoring camera is fixedly set, so a mark line with a corresponding height of 0.6m can be set in the monitoring screen, and the part of the monitoring screen above the mark line can be determined to be higher than 0.6 meters). The part below the fixed height mark line of the regularly captured instantaneous image of the monitoring screen is recorded as a captured image, and a similarity comparison is performed in the captured images. When the similarity between 5 consecutive captured images is greater than a preset similarity threshold (in a specific embodiment, the similarity threshold is 95%), the pending user is marked as a target user.
[0048] By distinguishing the target users from the pending users, it is possible to screen out the users who stay in front of the express cabinet from all the users who enter the monitoring range, and then conduct further behavioral analysis on them, eliminating passers-by who do not send or receive parcels, and omitting subsequent analysis steps.
[0049] The object recognition unit analyzes and recommends the target user as follows:
[0050] During the process of intercepting and judging the target user's monitoring screen (that is, from the start of regular interception of the monitoring screen to the determination of the target user), the human body contour area value of the target user is obtained in real time, and a line graph of the change of the human body contour area value over time is drawn, and the instantaneous image corresponding to the maximum value of the human body contour area value in the line graph is selected as the target image (the instantaneous image corresponding to the maximum value of the human body contour area value can be used as the front view of the target user), and the target image is further analyzed to obtain the human body contour and face data in the target image, obtain two eye contours in the face data, establish an eye distance line segment with the two closest points on the adjacent sides of the two eye contours, construct a straight line perpendicular to the horizontal plane passing through the midpoint of the eye distance line segment, and record it as the contour bisector, and divide the target user's body contour into two parts with the contour bisector and record them as ,calculate The absolute value of the difference in area between the two parts and divided by The smaller value in the area difference ratio , the calculation formula is ,in express The smaller value in the area difference threshold is preset. When the area difference ratio between the two parts is less than or equal to the area difference threshold, the target user is recorded as the recipient user, otherwise, he is recorded as the sender user.
[0051] It should be noted that the human body contour can be identified as a symmetrical figure when viewed completely straight on, so the contour bisector can be used to divide the human body contour into two parts. Under normal unarmed conditions, the contour areas of the two parts should be the same. In reality, the human body will have more or less hand movements during the mailing operation, which will cause area differences within a certain range. Therefore, when the area difference is within a certain range, it can be preliminarily determined that the target user is in an unarmed state, otherwise, it can be determined to be in a mailing state.
[0052] When the target user is a sender, the first-level interface of the touch panel is adjusted to a sender interface; when the target user is a receiver, the first-level interface of the touch panel is adjusted to a receiver interface.
[0053] It should be noted that the first-level interface refers to the interface directly displayed on the touch panel. Users can directly perform corresponding sending and receiving operations through the first-level interface. When the user does not need to operate the first-level interface, they can exit and switch by themselves. Compared with the fixed first-level interface, adjusting the display of the first-level interface based on the user status can set the corresponding operation interface for the user in advance, reduce the user's interface switching operations, and thus improve the user experience.
[0054] The cabinet opening recommendation module further analyzes the human body contour in the monitoring image and recommends suitable cabinet openings for the target user. The analysis process is as follows:
[0055] According to the interface (sending interface or receiving interface) that the target user finally chooses to operate, the current behavior is recorded as the sending behavior or the receiving behavior. When the current behavior of the target user is recorded as the sending behavior, the target user is obtained. The absolute value of the area difference between the two parts is recorded as , substitute into the formula The first limit value is calculated by ;
[0056] The cabinet recommendation module is located on both sides of the express cabinet and is equipped with auxiliary cameras for capturing the side image of the target user. When the user's current behavior is marked as a mailing behavior, the side image of the target user is captured and recorded as a side view. The maximum height of the target user's human body outline in the side view is obtained and recorded as h. A curve of the change of the horizontal width w of the target user's human body outline in the side view versus the vertical height is drawn, and the maximum value of the horizontal width w within the height interval [0.4h, 0.7h] is obtained and recorded as the second limit value ;
[0057] Get the length, width, and height of the grid selected by the target user, recorded as ,choose The minimum value in With the first limit value and the second limit value The minimum value in Compare and choose The maximum value in With the first limit value and the second limit value The maximum value in For comparison, ≤ or ≤ When the grid size is not appropriate, an instruction is generated to prompt the target user to select a larger grid size and recommend ≥ and ≥ The grid.
[0058] By recommending slots to target users, we can help them choose suitable slots that can accommodate their parcels, reduce the trial and error time in the process of selecting slots, improve their mailing experience, and optimize their mailing efficiency.
[0059] The information analysis module analyzes the recipient and sender information of each user, and recommends the corresponding sending interface or receiving interface to the user when the user's face is recognized based on the receiving and sending rules of each user.
[0060] Assign a user label to each unique face data , obtain the sender information of the package corresponding to the sending behavior and bind it with the corresponding facial data, obtain the recipient information of the package corresponding to the receiving behavior and bind it with the corresponding facial data, each user label corresponds to multiple sender information and multiple recipient information, and the multiple sender information and multiple recipient information together constitute the information data set corresponding to the user label. When the information data set of the same user label only contains the sender information, the user label is recorded as the sending label. Similarly, when the information data set of the same user label only contains the recipient information, the user label is recorded as the receiving label;
[0061] When the facial data of the shipping label is the same as the facial data of the target user, when the target user is the sender, the first-level interface of the touch panel is adjusted to the sending interface. When the facial data of the receiving label is the same as the facial data of the target user, when the target user is the recipient, the first-level interface of the touch panel is adjusted to the receiving interface.
[0062] By summarizing and summarizing the sending and receiving information of each user, we can discover the sending and receiving patterns of each user, and then distinguish the sending and receiving habits of each user. Before each target user is ready to operate the express cabinet, we use face recognition technology to distinguish the target user, and make preparations in advance for the target user based on his / her sending and receiving habits. We can intelligently distinguish the displayed first-level interface to facilitate the target user's operation, thereby further improving the user experience.
[0063] Reference Figure 2 The present invention also includes a mailing behavior identification and analysis method based on monitoring screen, which is applicable to the mailing behavior identification and analysis system based on monitoring screen in the embodiment:
[0064] Step 1: The monitoring identification module includes a user identification unit and an object identification unit. The user identification unit performs image analysis on people entering the monitoring range and marks them as pending users. The pending users who stay for more than a set time are marked as target users.
[0065] Step 2: The object recognition unit analyzes the human body contour of the target user, determines whether the target user holds an object and marks the target user as a sender or a receiver, and recommends displaying a corresponding sender interface or a receiver interface on the touch panel;
[0066] Step 3: The data synchronization module extracts the recipient and sender information of the received or sent package and uploads it to the database. The monitoring and recognition module obtains the facial data of the corresponding recipient and sender. The recipient and sender are collectively referred to as users.
[0067] Step 4: The information analysis module binds the recipient and sender information of the received or sent package with the facial data of the corresponding user, and assigns a user label to each facial data. The multiple sender information and multiple recipient information of each user label together constitute the information data set of the corresponding user label. When the information data set of the same user label only contains the sender information, the user label is recorded as the sending label. When the information data set of the same user label only contains the recipient information, the user label is recorded as the receiving label.
[0068] Step 5: When the facial data of the shipping label is the same as the facial data of the target user, the target user is the sender and the first-level interface of the touch panel is adjusted to the sending interface. When the facial data of the receiving label is the same as the facial data of the target user, the target user is the recipient and the first-level interface of the touch panel is adjusted to the receiving interface.
[0069] Reference Figure 3 , the present invention also provides a mailing behavior identification and analysis device based on monitoring screens, including: a processor, such as a central processing unit, a graphics processing unit, a memory, a communication bus, a user interface, and a network interface. Among them, the communication bus is used to realize the connection and communication between these components. The user interface is connected to the touch panel to realize the input data of the touch panel into the central processing unit. The network interface includes a wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface), and the memory includes a random access memory (Random Access Memory, RAM) for temporary data storage operations and a non-volatile memory (Non-Volatile Memory, NVM) for data storage.
[0070] It should be noted that the processor and the memory are used together to execute the program stored in the memory, and when the program is executed by the computer, the methods and systems described in the above embodiments can be implemented.
[0071] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for identifying and analyzing mailing behavior based on monitoring images in the present invention is implemented.
[0072] It should be noted that the storage medium includes permanent and non-permanent, removable and non-removable items that can be used to store information by any method or technology. For example, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0073] The express lockers using this system can not only be used in express delivery stations to form a self-service mode of picking up and sending express parcels, but also can be placed in various office buildings and other areas, which is highly practical.
[0074] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.
Claims
1. A mailing behavior recognition and analysis system based on monitoring images, comprising: a plurality of express cabinets, each of which is provided with a plurality of slots for mailing and receiving mail, each of which is provided with an electric control cabinet door, a monitoring camera, and a touch panel for inputting information and controlling the opening and closing of the electric control cabinet door located below the monitoring camera, characterized in that: Also includes: Data synchronization module, monitoring identification module, cabinet recommendation module, information analysis module; The express parcels in the express cabinet waiting for the user to sign for and take away are recorded as received parcels, and the express parcels in the express cabinet waiting for the courier to pick up and send out are recorded as sent parcels. The data synchronization module obtains the recipient and sender information of the received parcels and the sent parcels. After the user enters the recipient information, the electric control cabinet door of the corresponding compartment where the received parcel is located is controlled to open. After the user enters the sender information and the recipient, the user selects one of the compartments and controls the electric control cabinet door to open. The data synchronization module synchronously uploads all the recipient and sender information to the database; The monitoring identification module includes a user identification unit and an object identification unit. The user identification unit identifies the human body contour and face data of the person entering the monitoring range and analyzes them, marking them as pending users, and marking pending users who stay for more than a set time as target users. The object identification unit analyzes the object held by the target user, marks them as a sender or a recipient, and recommends displaying the corresponding sending interface or receiving interface on the touch panel; The user identification unit includes a human body contour recognition model for identifying the human body contour in the monitoring video, the human body contour in the monitoring picture is recognized by the human body contour recognition model and recorded as the target contour, the head area of the target contour is subjected to face recognition to obtain face data, and the same face data and its corresponding target contour are recorded as the pending user; The face area ratio value and the corresponding target contour area ratio value of the pending user in the monitoring screen are obtained, and a face ratio threshold value and a contour ratio threshold value are set respectively. When the face area ratio value and the target contour area ratio value are both greater than or equal to the corresponding face ratio threshold value and contour ratio threshold value, the monitoring screen is regularly intercepted. A fixed height mark is set in the monitoring screen, and the corresponding height of the fixed height mark is 0.6m. The part below the fixed height mark of the instantaneous image of the monitoring screen captured regularly is recorded as the intercepted image, and the intercepted images are compared for similarity. When the similarity between multiple consecutive intercepted images is greater than the preset similarity threshold, the pending user is marked as the target user; the object recognition unit analyzes and recommends the target user as follows: In the process of intercepting and judging the monitoring screen of the target user, the human body contour area value of the target user is obtained in real time, and a line graph of the human body contour area value changing with time is drawn, and the instantaneous image corresponding to the maximum value of the human body contour area value in the line graph is selected as the target image, and the human body contour and face data in the target image are obtained, and two eye contours in the face data are obtained, and an eye distance line segment is established with the two closest points on the adjacent sides of the two eye contours, and a straight line perpendicular to the horizontal plane passing through the midpoint of the eye distance line segment is constructed and recorded as a contour bisector, and the human body contour of the target user is divided into two parts by the contour bisector and recorded as ,calculate The absolute value of the difference in area between the two parts and divided by The smaller value in the area difference ratio , the calculation formula is ,in express The smaller value in the area difference threshold is preset. When the area difference ratio between the two parts is less than or equal to the area difference threshold, the target user is recorded as the recipient user, otherwise, he is recorded as the sender user; When the target user is a sender, the first level interface of the touch panel is adjusted to the sender interface; when the target user is a receiver, the first level interface of the touch panel is adjusted to the receiver interface; the analysis process of the counter recommendation module is as follows: According to the interface where the target user finally chooses to operate, the current behavior is recorded as a sending behavior or a receiving behavior. When the target user's current behavior is recorded as a sending behavior, the target user's The absolute value of the area difference between the two parts is recorded as , substitute into the formula The first limit value is calculated by ; The cabinet recommendation module is located on both sides of the express cabinet and is equipped with auxiliary cameras for capturing the side image of the target user. When the user's current behavior is marked as a mailing behavior, the side image of the target user is captured and recorded as a side view. The maximum height of the target user's human body outline in the side view is obtained and recorded as h. A curve of the change of the horizontal width w of the target user's human body outline in the side view versus the vertical height is drawn, and the maximum value of the horizontal width w within the height interval [0.4h, 0.7h] is obtained and recorded as the second limit value ; Get the length, width, and height of the grid selected by the target user, recorded as ,choose The minimum value in With the first limit value and the second limit value The minimum value in Compare and choose The maximum value in With the first limit value and the second limit value The maximum value in For comparison, ≤ or ≤ When the grid size is not appropriate, an instruction is generated to prompt the target user to select a larger grid size and recommend ≥ and ≥ The information analysis module analyzes the recipient and sender information of each user, and recommends the corresponding sending interface or receiving interface for each user when the user's face is recognized according to the receiving and sending rules of each user.
2. The mailing behavior identification and analysis system based on monitoring images according to claim 1 is characterized in that: The information analysis module recommendation process is as follows: Assign a user label to each unique face data , obtain the sender information of the package corresponding to the sending behavior and bind it with the corresponding facial data, obtain the recipient information of the package corresponding to the receiving behavior and bind it with the corresponding facial data, each user label corresponds to multiple sender information and multiple recipient information, and the multiple sender information and multiple recipient information together constitute the information data set corresponding to the user label. When the information data set of the same user label only contains the sender information, the user label is recorded as the sending label, and when the information data set of the same user label only contains the recipient information, the user label is recorded as the receiving label; When the facial data of the shipping label is the same as the facial data of the target user, when the target user is the sender, the first-level interface of the touch panel is adjusted to the sending interface. When the facial data of the receiving label is the same as the facial data of the target user, when the target user is the recipient, the first-level interface of the touch panel is adjusted to the receiving interface.
3. The mailing behavior identification and analysis method based on monitoring images is characterized by: The mailing behavior identification and analysis system based on monitoring screen as applied to claim 2 has the following specific operating steps: Step 1: The monitoring identification module includes a user identification unit and an object identification unit. The user identification unit performs image analysis on people entering the monitoring range and marks them as pending users. The pending users who stay for more than a set time are marked as target users. Step 2: The object recognition unit analyzes the human body contour of the target user, determines whether the target user holds an object and marks the target user as a sender or a receiver, and recommends displaying a corresponding sender interface or a receiver interface on the touch panel; Step 3: The data synchronization module extracts the recipient and sender information of the received or sent package and uploads it to the database. The monitoring and recognition module obtains the facial data of the corresponding recipient and sender. The recipient and sender are collectively referred to as users. Step 4: The information analysis module binds the recipient and sender information of the received or sent package with the facial data of the corresponding user, and assigns a user label to each facial data. The multiple sender information and multiple recipient information of each user label together constitute the information data set of the corresponding user label. When the information data set of the same user label only contains the sender information, the user label is recorded as the sending label. When the information data set of the same user label only contains the recipient information, the user label is recorded as the receiving label. Step 5: When the facial data of the shipping label is the same as the facial data of the target user, the target user is the sender and the first-level interface of the touch panel is adjusted to the sending interface. When the facial data of the receiving label is the same as the facial data of the target user, the target user is the recipient and the first-level interface of the touch panel is adjusted to the receiving interface.
4. The equipment for identifying and analyzing mailing behaviors based on monitoring images is characterized in that: The invention comprises a processor, a memory, a communication bus, a user interface and a network interface. The communication bus is used to realize the connection and communication between these components. The user interface is connected to the touch panel. The network interface comprises a wired interface and a wireless interface. The memory comprises a random access memory and a non-volatile memory. The memory stores a computer program. When the computer program is executed by the processor, the method for identifying and analyzing mailing behavior based on monitoring images as described in claim 3 is realized.
5. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for identifying and analyzing mailing behavior based on monitoring images as described in claim 3 is implemented.
Citation Information
Patent Citations
Express cabinet recommendation method and device, equipment and storage medium
CN110084553A
Intelligent express cabinet and system based on lightweight face biological information processing
CN115273193A
