Intelligent monitoring method and intelligent monitoring system
By monitoring video border analysis and identifying the items carried by the personnel and determining the scanning of the code, the problem of the risk of item loss in the existing technology is solved, and more efficient warehouse management and security are achieved.
Patent Information
- Application Number
- CN202310540991.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-12
AI Technical Summary
Existing intelligent monitoring technology is difficult to effectively identify whether people carry items and scan the code correctly, resulting in an increase in the risk of items losing, especially in the case of missing code scanning in and out of the warehouse.
By reading the surveillance video inside the warehouse, the tracking borders of personnel, items and code scanning stations are generated, and whether the personnel are carrying items is determined based on the border distance and overlap, and alarms are issued when leaving the warehouse to ensure the integrity of the code scanning of items.
It effectively avoids the loss of items. By identifying the unscanned code and missing code scans, the accuracy and security of warehouse management are improved and management costs are reduced.
Smart Images

Figure CN118942194A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of intelligent monitoring, and in particular to an intelligent monitoring method and an intelligent monitoring system. Background Art
[0002] With the continuous construction of warehouses, the cost of warehouse management is also increasing. How to do a good job of warehouse management has become the key to reducing costs. Compared with traditional manually managed warehouses, unmanned warehouses can significantly improve the efficiency, accuracy and safety of warehouse management, while also reducing management costs, labor costs and material loss rates. To achieve unmanned warehouses, various advanced technologies need to be used to achieve efficient management and monitoring. Artificial intelligence technology is a very important part of it, which can help realize the automation and intelligence of unmanned warehouses. For example, artificial intelligence technology is used to analyze and process surveillance video data to achieve management and monitoring of unmanned warehouses.
[0003] In recent years, with the rapid development of computer technology, related computer vision technologies represented by deep learning have been popularized and applied, making it possible to supervise the outbound delivery of goods through intelligent monitoring, making warehouse management more intelligent and cost-effective. For example, object detection technology is used to identify the type and quantity of inbound and outbound goods, but the current technology can only be applied to simple inbound and outbound scenarios, and there is a risk of item loss in some inbound and outbound scenarios. Summary of the invention
[0004] The embodiments of the present application provide an intelligent monitoring method and an intelligent monitoring system for identifying whether a person fails to scan a code or misses to scan a code while carrying items, thereby preventing the items from being lost.
[0005] The first aspect of the present application provides an intelligent monitoring method:
[0006] The first surveillance video inside the warehouse is read in, and the first surveillance video is processed to generate a tracking frame for selecting a person, an object, and a barcode scanning station in the first surveillance video. Based on the frame distance between the tracking frames of the person and the object, it is determined whether the person carries the object. If the person carries the object, it is determined whether the person places the object on the barcode scanning station based on the overlap between the tracking frames of the object and the barcode scanning station. If the object is not placed and the person leaves the warehouse, an alarm is issued.
[0007] In this application, based on the analysis of the first video, it can be determined whether the person is carrying any items. If so, it can be further determined whether the person places the items on the code scanning table. If not, and the person leaves the warehouse, it means that the code has not been scanned. Therefore, issuing an alarm can avoid the loss of items.
[0008] In a possible implementation, a second surveillance video of the scanning area of the barcode scanning station is also read in, and a first quantity of items placed by a person in the scanning area is determined based on the second surveillance video, and a second quantity of items scanned by the person is obtained. If the items are placed and the person leaves the warehouse, it is determined whether the first quantity is equal to the second quantity, and if not, an alarm is issued.
[0009] In the present application, if a person places the items he carries on the code scanning table, it is determined whether the first quantity is equal to the second quantity. If they are equal, it means that the person has scanned all the items placed in the code scanning area. If they are not equal, it means that some items placed in the code scanning area have been missed. Therefore, issuing an alarm can avoid the loss of items.
[0010] In a possible implementation, it is also determined whether the person and the object are moving in a preset direction. If so, an operation is triggered to determine whether the person is carrying the object based on the frame distance between the tracking frames of the person and the object. The preset direction is the direction toward the entrance and exit of the warehouse.
[0011] In this application, when people and items move toward the entrance and exit of the warehouse, it indicates that there is a possibility that the people and items will leave the warehouse. Only then is the operation of determining whether the person is carrying items triggered, thereby reducing unnecessary expenses.
[0012] In a possible implementation, if in N consecutive video frames of the first surveillance video, the border distance is less than or equal to a preset value, it is determined that the person is carrying an item.
[0013] In a possible implementation, if in at least one video frame of the first surveillance video, the overlap between the object and the tracking frame of the barcode scanning station is greater than or equal to a preset value, it is determined that the person places the item he carries on the barcode scanning station.
[0014] In a possible implementation, it is also determined whether the bottom of the tracking frame of the person reaches a preset image height of the first monitoring video and disappears in the first monitoring video. If so, it is determined that the person has left the warehouse.
[0015] The second aspect of the present application provides an intelligent monitoring system, comprising:
[0016] A video acquisition module, used to obtain a first surveillance video inside the warehouse;
[0017] A cloud server, used for reading the first surveillance video;
[0018] The cloud server is further used to process the first surveillance video to generate a tracking frame for selecting a person, an object, and a barcode scanning station in the first surveillance video;
[0019] The cloud server is also used to determine whether the person is carrying the item based on the border distance between the tracking borders of the person and the item;
[0020] If the person carries the item, the cloud server determines whether the person places the item on the barcode scanning station based on the overlap between the item and the tracking frame of the barcode scanning station;
[0021] If it is not placed and the person leaves the warehouse, the cloud server sends an abnormal alarm to the warehouse management system;
[0022] Warehouse management system, used to display abnormal alarms.
[0023] In a possible implementation, the intelligent monitoring system further includes a code scanning data acquisition module;
[0024] The video acquisition module is also used to obtain a second monitoring video of the scanning area of the scanning station;
[0025] The cloud server is also used to read the second surveillance video;
[0026] The cloud server is further used to determine a first quantity of objects placed by a person in a code scanning area based on a second surveillance video;
[0027] A code scanning data collection module, used to obtain a second quantity of items scanned by a person;
[0028] The cloud server is further used to obtain a second quantity from the code scanning data collection module;
[0029] If it is placed, and the person leaves the warehouse, the cloud server determines whether the first quantity is equal to the second quantity;
[0030] If not equal, the cloud server sends an abnormal alarm to the warehouse management system, so that the warehouse management system displays the abnormal alarm.
[0031] In one possible implementation,
[0032] The cloud server is also used to determine whether people and objects are moving in a preset direction. If so, it triggers an operation based on the frame distance between the tracking frames of the person and the object to determine whether the person is carrying the object. The preset direction is the direction toward the entrance and exit of the warehouse.
[0033] In a possible implementation, it is determined whether the border distance is less than or equal to a preset value in N consecutive video frames of the first surveillance video. If so, it is determined that the person is carrying an item; if not, it is determined that the person is not carrying an item.
[0034] In one possible implementation, it is determined whether the overlap degree is greater than or equal to a preset value in at least one video frame of the first surveillance video. If so, it is determined that the person places the item he carries on the code scanning table; if not, it is determined that the person does not place the item he carries on the code scanning table.
[0035] In one possible implementation,
[0036] The cloud server is also used to determine whether the bottom of the tracking frame of the person reaches a preset image height of the first monitoring video and disappears in the first monitoring video. If so, the cloud server determines that the person has left the warehouse.
[0037] A third aspect of the present application provides a cloud server, including:
[0038] The reading module is used to read the first surveillance video inside the warehouse.
[0039] The processing module is used to process the first surveillance video to generate a tracking frame for selecting a person, an object, and a code scanning station in the first surveillance video.
[0040] The processing module is used to determine whether a person carries an object based on a frame distance between tracking frames of the person and the object.
[0041] The processing module is also used to determine whether the person places the carried item on the code scanning table based on the overlap between the item and the tracking frame of the code scanning table.
[0042] The alarm module is used to issue an alarm if the items are not placed and the personnel leave the warehouse.
[0043] In one possible implementation,
[0044] The reading module is also used to read the second monitoring video of the scanning area of the scanning station.
[0045] The processing module is also used to determine the first quantity of items placed by the person in the code scanning area based on the second monitoring video.
[0046] The reading module is also used to obtain the second quantity of the items scanned by the personnel.
[0047] The processing module is also used to determine whether the first quantity is equal to the second quantity if the items are placed and the personnel leave the warehouse, and to issue an alarm if not.
[0048] In one possible implementation,
[0049] The processing module is also used to determine whether the person and the object are moving in a preset direction. If so, it triggers an operation based on the frame distance between the tracking frames of the person and the object to determine whether the person is carrying the object. The preset direction is the direction toward the entrance and exit of the warehouse.
[0050] In one possible implementation,
[0051] The processing module is specifically used to determine whether the border distance is less than or equal to a preset value in N consecutive video frames of the first monitoring video. If so, it is determined that the person is carrying the item; if not, it is determined that the person is not carrying the item.
[0052] In one possible implementation,
[0053] The processing module is specifically used to determine whether the overlap degree is greater than or equal to a preset value in at least one video frame of the first surveillance video. If so, it is determined that the person places the item he carries on the code scanning table; if not, it is determined that the person does not place the item he carries on the code scanning table.
[0054] In one possible implementation,
[0055] The processing unit is further configured to determine whether the bottom of the tracking frame of the person reaches a preset image height of the first monitoring video and disappears in the first monitoring video. If so, it is determined that the person has left the warehouse.
[0056] A fourth aspect of the present application provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when the computer program or instructions are executed, causes the computer to execute the method in the first aspect.
[0057] A fifth aspect of the present application provides a computer program product, which, when the computer program code is executed on a computer, enables the computer to implement the method in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] Figure 1 This is a schematic diagram of the structure of the intelligent monitoring system of this application;
[0059] Figure 2 A working diagram of the intelligent monitoring system of this application;
[0060] Figure 3 A schematic diagram of a flow chart of the intelligent monitoring method in this application;
[0061] Figure 4a A schematic diagram for determining whether a person is carrying items in this application;
[0062] Figure 4b Another schematic diagram for determining whether a person is carrying items in this application;
[0063] Figure 5 A schematic diagram for determining whether a person places the items they carry on the code scanning station in this application;
[0064] Figure 6 A schematic diagram of the structure of the cloud server in this application;
[0065] Figure 7 This is another structural diagram of the cloud server in this application. DETAILED DESCRIPTION
[0066] The following describes the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of the present application, rather than all embodiments. It is known to those skilled in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0067] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0068] In an existing technical solution, a video surveillance warehouse in-and-out counting system based on deep learning is proposed:
[0069] Monitoring video acquisition module: used to collect real-time video footage of vehicles entering and leaving the warehouse.
[0070] Video processing module: used to receive the video images sent by the monitoring video acquisition module and send the received video images to the object detection and recognition module.
[0071] Object detection and identification module: used to detect and identify the type and quantity of objects in the video image sent by the video processing module.
[0072] In and out warehouse logic judgment module: used to judge the status of outbound or inbound storage and record the specific time based on the object detection and recognition results of continuous frame video images.
[0073] Visual management record module: used to record photos of items entering and leaving the warehouse, the category and quantity of items, and perform front-end visual display.
[0074] The method comprises the following steps:
[0075] S1. Surveillance video acquisition: The surveillance video acquisition module collects real-time videos of vehicles carrying goods entering and leaving the warehouse at the warehouse entrance;
[0076] S2. Video segmentation: The video processing module segments the acquired video and divides the video into image data according to the number of frames;
[0077] S3. Manual marking: Manually mark the image data divided by the number of frames, and mark the vehicle position and object position in the video screen;
[0078] S4. Perform detection model training and classification model training respectively: Use a deep learning target detection algorithm to train the target detection model and a deep learning classification algorithm to train the classification model for the manually labeled image data, to form an object detection and recognition module;
[0079] S5. Determine the location and quantity of items on the vehicle: Use the trained object detection model to determine the location and quantity of vehicles and items entering and leaving the warehouse;
[0080] S6. Accurate identification: Use classification models to identify the specific category of objects;
[0081] S7. Record and display vehicle information: By identifying the continuous images of the vehicle carrying items passing through the warehouse door, the direction of the vehicle is determined and recorded as an outbound or inbound action.
[0082] The above-mentioned video surveillance warehouse in and out counting system determines the vehicle in and out and the carrying of objects through object detection in continuous frame video images. It is only applicable to the situation where vehicles are carrying objects in and out, and cannot detect the scene where people are carrying objects out of the warehouse. It cannot detect objects such as hand-held objects, objects in the arms, or objects carried in carts, and it cannot determine whether the outbound transportation of objects has been recorded, so there is a risk of object loss.
[0083] The embodiments of the present application provide an intelligent monitoring method and an intelligent monitoring system for identifying whether a person fails to scan a code or misses to scan a code while carrying items, thereby preventing the items from being lost.
[0084] This application can be applied to Figure 1The system architecture shown includes a code scanning data acquisition module, a video data acquisition module, a cloud server and a warehouse management system, wherein the cloud server includes a data reading module, a video analysis module and an abnormal alarm module. A code scanning station is set up inside the warehouse. After obtaining the items, the personnel need to place the items in the scanning area of the code scanning station and use the corresponding scanning device to scan the items before taking the items out of the warehouse. The code scanning data acquisition module can obtain the number of scanned items recorded by the scanning device, that is, the number of items scanned by the personnel; a camera is set up at the entrance and exit of the warehouse, which is used to shoot surveillance videos inside the warehouse. In addition, another camera is set up inside the warehouse, which is used to shoot surveillance videos of the scanning area of the code scanning station. The video data acquisition module can obtain the surveillance video data shot by the above two cameras; please refer to Figure 2 The data reading module of the cloud server can read the data obtained by the above-mentioned code scanning data acquisition module and the video data acquisition module, and send the monitoring video data obtained by the video data acquisition module to the video analysis module, and send the number of scanned items obtained by the code scanning data acquisition module to the abnormal alarm module; the video analysis module obtains the video analysis result according to the above-mentioned monitoring video data analysis, and sends the video analysis result to the abnormal alarm module; the abnormal alarm module determines whether it is necessary to send an abnormal alarm to the warehouse management system according to the video analysis result and the number of scanned items; the warehouse management system is a system for managing and monitoring items in the warehouse. Its function is to record the entry and exit of items, manage the items and equipment in the warehouse in real time, and can display a human-computer interaction interface. The human-computer interaction interface is used to display relevant information of the warehouse management system and display the abnormal alarm after receiving the abnormal alarm, so as to remind the warehouse management personnel to make corresponding processing.
[0085] See also Figure 3 , the following is a detailed introduction to a process of the intelligent monitoring method in this application:
[0086] 301. Read the first surveillance video inside the warehouse;
[0087] When there is human activity inside the warehouse, the two cameras start to shoot surveillance video at the same time. The video data acquisition module will save the captured surveillance video locally and notify the data reading module of the cloud server to read the surveillance video. For the convenience of subsequent description, the surveillance video inside the warehouse is recorded as the first surveillance video, and the surveillance video of the barcode scanning area of the barcode scanning station is recorded as the second surveillance video.
[0088] 302. Process the first surveillance video to generate a tracking frame for selecting a person, an object, and a barcode scanning station in the first surveillance video;
[0089] After the data reading module reads the first surveillance video and the second surveillance video, it sends the first surveillance video and the second surveillance video to the video analysis module. The video analysis module processes the first surveillance video to generate a tracking frame for selecting a person, an object, and a barcode scanning station in each video frame of the first surveillance video. The video analysis module can perform the above processing based on the target detection algorithm and the multi-target tracking algorithm. The specific process is not repeated here.
[0090] 303. Determine whether the person carries an item based on a frame distance between the tracking frames of the person and the item;
[0091] The video analysis module determines whether a person is carrying an object based on the frame distance between the tracking frames of the person and the object. Specifically, the video analysis module determines whether the frame distance between the tracking frames of the person and the object is less than or equal to a preset value in N consecutive video frames. If so, the video analysis module determines that the person is carrying the above-mentioned object. If not, the video analysis module determines that the person is not carrying the object. In a possible implementation, the above-mentioned N video frames can be 15 video frames, and the above-mentioned preset value can be 20 pixels. It should be understood that the above-mentioned frame distance can be calculated in various ways. In a possible implementation, please refer to Figure 4a , the border distance can be the distance between two adjacent edges of the two tracking borders; or in another possible implementation, see Figure 4b , the frame distance may also be the distance between the two closest corners of the two tracking frames. Optionally, the video analysis module makes the above judgment only when it is determined that both the person and the object are moving in a preset direction, and the preset direction is the direction toward the entrance and exit of the warehouse, so as to screen out the people and objects that are stationary or will not leave the warehouse.
[0092] 304. If the person carries the item, determine whether the person places the item on the scanning station based on the degree of overlap between the item and the tracking frame of the scanning station;
[0093] If the video analysis module determines that the person is carrying an item, it further determines whether the person has placed the item on the barcode scanning station based on the overlap between the item and the tracking frame of the barcode scanning station. Figure 5The video analysis module determines whether the overlap between the tracking frame of the barcode scanning platform and the object is greater than or equal to a preset value in at least one video frame. If so, it is determined that the person has placed the item on the barcode scanning platform. If not, it is determined that the person has not placed the item on the barcode scanning platform. It should be understood that the overlap between the tracking frame of the barcode scanning platform and the object is proportional to the area where the two tracking frames overlap. For example, when the tracking frame of the barcode scanning platform completely overlaps with the tracking frame of the object, the overlap is 100%, and when the tracking frame of the barcode scanning platform does not overlap with the tracking frame of the object, the overlap is 0%.
[0094] The video analysis module also determines the number of objects placed in the code scanning area by the personnel based on the second monitoring video. Specifically, the video analysis module identifies the number of objects C1 to C2 in the code scanning area frame by frame according to the target detection algorithm. n , where C represents the number of items in the scanned area, and 1 to n represent different video frames. The video analysis module calculates the number of items C in the scanned area of the current video frame. t The number of items in the scanned area of the previous video frame C t-1 For comparison (1≤t≤n), if C t >C t-1 , indicating that a new item has been placed in the scanning area, the total number increases by C t -C t-1 If C t ≤C t-1 , indicating that the items in the code scanning area are taken out or remain unchanged, and the total number remains unchanged. It should be understood that if the person does not place the items he carries on the code scanning table or the person does not carry any items, the number of items placed by the person in the code scanning area is 0.
[0095] In addition, the video analysis module will also determine whether the person has left the warehouse based on the first surveillance video. Specifically, the video analysis module determines whether the bottom of the tracking frame of the person has reached the preset image height of the screen of the first surveillance video. Exemplarily, the image height of the screen of the first surveillance video is calculated from top to bottom, with the top being 0, and the above-mentioned preset image height is, for example, 4 / 5. If it has been reached, the video analysis module will add the person to the set of people ready to leave the warehouse. If the bottom of the tracking frame of the person does not reach the preset image height of the screen of the first surveillance video subsequently, the person will be removed from the set of people ready to leave the warehouse. If the person disappears from the screen of the first surveillance video without being removed from the set of people ready to leave the warehouse, it is determined that the person has left the warehouse.
[0096] To sum up, the video analysis module can analyze the surveillance video data and obtain the video analysis results, which include whether the personnel have left the warehouse, whether they are carrying items, and the number of items placed by the personnel in the scanning area. If the personnel are carrying items, the video analysis results will further include whether the personnel place the items they carry on the scanning station. The video analysis module will send the above video analysis results to the abnormal alarm module.
[0097] 305. If it is not placed and the personnel leave the warehouse, an alarm will be issued.
[0098] After the abnormal alarm module obtains the video analysis results, if the video analysis results indicate that the person carries items, does not place the items on the barcode scanning station, and leaves the warehouse, an abnormal alarm is sent to the warehouse management system, and the warehouse management system displays the abnormal alarm after receiving the abnormal alarm. Or in another case, if the video analysis results indicate that the person carries items, places the items on the barcode scanning station, and leaves the warehouse, the abnormal alarm module will determine whether the number of items placed by the person in the barcode scanning area is equal to the number of scanned items recorded by the barcode scanning device. If not, an abnormal alarm is sent to the warehouse management system, and the warehouse management system displays the abnormal alarm after receiving the abnormal alarm.
[0099] Of course, in one possible implementation, Figure 1 The system architecture shown may also not include a code scanning data acquisition module, and the video data acquisition module may not obtain the second surveillance video. The abnormal alarm module will only send an abnormal alarm to the warehouse management system when it is determined that a person is carrying items, has not placed the items on the code scanning station, and has left the warehouse.
[0100] In this application, the cloud server can determine whether a person is carrying any items, whether the items they are carrying are placed on the barcode scanning table, whether they have left the warehouse, and the number of items placed by the person in the barcode scanning area based on the surveillance video data. In addition, the cloud server can also obtain the number of scanned items recorded by the barcode scanning device. The above information can be used to determine whether a person has left the warehouse without scanning the barcode or has missed scanning the barcode. If so, an alarm will be issued in time to avoid the loss of items in the warehouse.
[0101] The method in this application is introduced above, and the device in this application is introduced below:
[0102] See also Figure 6 The cloud server 600 in the present application includes a reading module 601 , a processing module 602 and an alarm module 603 .
[0103] The reading module 601 is used to read the first surveillance video inside the warehouse.
[0104] The processing module 602 is used to process the first surveillance video to generate a tracking frame for selecting a person, an object, and a barcode scanning station in the first surveillance video.
[0105] The processing module 602 is used to determine whether the person is carrying an item based on the frame distance between the tracking frames of the person and the item.
[0106] The processing module 602 is also used to determine whether the person places the carried item on the code scanning table based on the overlap between the item and the tracking frame of the code scanning table.
[0107] The alarm module 603 is used to generate an alarm if the items are not placed and the personnel leave the warehouse.
[0108] In one possible implementation,
[0109] The reading module 601 is also used to read the second monitoring video of the scanning area of the scanning station.
[0110] The processing module 602 is also used to determine a first quantity of items placed by a person in the code scanning area based on the second surveillance video.
[0111] The reading module 601 is also used to obtain the second quantity of the items scanned by the personnel.
[0112] The processing module 602 is also used to determine whether the first quantity is equal to the second quantity if the items are placed and the personnel leave the warehouse, and if not, to issue an alarm.
[0113] In one possible implementation,
[0114] Processing module 602 is also used to determine whether the person and the object are moving in a preset direction. If so, it triggers an operation based on the frame distance between the tracking frames of the person and the object to determine whether the person is carrying the object. The preset direction is the direction toward the entrance and exit of the warehouse.
[0115] In one possible implementation,
[0116] The processing module 602 is specifically used to determine whether the border distance is less than or equal to a preset value in N consecutive video frames of the first monitoring video. If so, it is determined that the person is carrying the item; if not, it is determined that the person is not carrying the item.
[0117] In one possible implementation,
[0118] Processing module 602 is specifically used to determine whether the overlap degree is greater than or equal to a preset value in at least one video frame of the first surveillance video. If so, it is determined that the person places the item he carries on the code scanning table; if not, it is determined that the person does not place the item he carries on the code scanning table.
[0119] In one possible implementation,
[0120] The processing unit 602 is further configured to determine whether the bottom of the tracking frame of the person reaches a preset image height of the first monitoring video and disappears in the first monitoring video. If so, it is determined that the person has left the warehouse.
[0121] Figure 7 It is a schematic diagram of the structure of a cloud server provided in the present application, which may include one or more processors 701 and a memory 705, wherein the memory 705 stores one or more application instructions or data.
[0122] The memory 705 may be a volatile memory or a non-volatile memory. The program stored in the memory 705 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the central processor 701 may be configured to communicate with the memory 705 and execute a series of instruction operations in the memory 705 on the cloud server 700.
[0123] Further optionally, the cloud server 700 may also include one or more power supplies 702, one or more wired or wireless network interfaces 703, one or more input and output interfaces 704, and / or one or more operating systems.
[0124] The CPU 701 can execute the aforementioned Figure 3 The operation of the cloud server in the illustrated embodiment will not be described in detail here.
[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0126] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0127] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0128] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0129] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.
Claims
1. An intelligent monitoring method, characterized in that: include: Read the first surveillance video inside the warehouse; Processing the first surveillance video to generate a tracking frame for selecting a person, an object, and a barcode scanning station in the first surveillance video; Determining whether the person carries the object based on a frame distance between the tracking frames of the person and the object; If the person carries the item, determining whether the person places the item on the scanning station based on the degree of overlap between the item and the tracking frame of the scanning station; If it is not placed and the person leaves the warehouse, an alarm is issued.
2. The method according to claim 1, characterized in that The method further comprises: Read in a second surveillance video of the code scanning area of the code scanning station; Determine a first quantity of the items placed by the person in the code scanning area based on the second surveillance video; Obtaining a second quantity of the item scanned by the person; If it is placed, and the person leaves the warehouse, determining whether the first quantity is equal to the second quantity; If not equal, an alarm is issued.
3. The method according to claim 1 or 2, characterized in that: The method further comprises: Determine whether the person and the object move in a preset direction, and if so, trigger an operation to determine whether the person is carrying the object based on the frame distance between the tracking frames of the person and the object, wherein the preset direction is the direction toward the entrance and exit of the warehouse.
4. The method according to any one of claims 1 to 3, characterized in that The determining whether the person carries the object based on the frame distance between the person and the tracking frame of the object comprises: Determine whether the border distance is less than or equal to a preset value in N consecutive video frames of the first surveillance video. If so, determine that the person is carrying the item; if not, determine that the person is not carrying the item.
5. The method according to claim 4, characterized in that The determining whether the person places the carried item on the code scanning platform based on the overlap between the item and the tracking frame of the code scanning platform includes: Determine whether the overlap degree is greater than or equal to a preset value in at least one video frame of the first surveillance video. If so, determine that the person places the item carried on the code scanning table; if not, determine that the person does not place the item carried on the code scanning table.
6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Determine whether the bottom of the tracking frame of the person reaches a preset image height of the first surveillance video and disappears in the first surveillance video; If so, it is determined that the person has left the warehouse.
7. An intelligent monitoring system, characterized in that: include: A video acquisition module, used to obtain a first surveillance video inside the warehouse; A cloud server, used for reading the first surveillance video; The cloud server is further used to process the first surveillance video to generate a tracking frame for selecting a person, an object, and a barcode scanning station in the first surveillance video; The cloud server is further configured to determine whether the person carries the object based on a frame distance between the person and the tracking frame of the object; If the person carries the item, the cloud server determines whether the person places the item on the barcode scanning station based on the degree of overlap between the item and the tracking frame of the barcode scanning station; If it is not placed, and the person leaves the warehouse, the cloud server sends an abnormal alarm to the warehouse management system; The warehouse management system is used to display the abnormal alarm.
8. The intelligent monitoring system according to claim 7, characterized in that: The intelligent monitoring system also includes a code scanning data acquisition module; The video acquisition module is also used to obtain a second monitoring video of the code scanning area of the code scanning station; The cloud server is further used to read the second surveillance video; The cloud server is further configured to determine a first quantity of the items placed by the person in the code scanning area based on the second monitoring video; The code scanning data collection module is used to obtain the second quantity of the items scanned by the person; The cloud server is further used to obtain the second quantity from the code scanning data collection module; If it is placed, and the person leaves the warehouse, the cloud server determines whether the first quantity is equal to the second quantity; If not equal, the cloud server sends an abnormal alarm to the warehouse management system, so that the warehouse management system displays the abnormal alarm.
9. The intelligent monitoring system according to claim 7 or 8, characterized in that: The cloud server is also used to determine whether the person and the object are moving in a preset direction. If so, it triggers an operation to determine whether the person is carrying the object based on the frame distance between the tracking frames of the person and the object, and the preset direction is the direction toward the entrance and exit of the warehouse.
10. The intelligent monitoring system according to any one of claims 7 to 9, characterized in that: The determining whether the person carries the object based on the frame distance between the person and the tracking frame of the object comprises: Determine whether the border distance is less than or equal to a preset value in N consecutive video frames of the first surveillance video. If so, determine that the person is carrying the item; if not, determine that the person is not carrying the item.
11. The intelligent monitoring system according to claim 10, characterized in that: The determining whether the person places the carried item on the code scanning platform based on the overlap between the item and the tracking frame of the code scanning platform includes: Determine whether the overlap degree is greater than or equal to a preset value in at least one video frame of the first surveillance video. If so, determine that the person places the item carried on the code scanning table; if not, determine that the person does not place the item carried on the code scanning table.
12. The intelligent monitoring system according to any one of claims 7 to 11, characterized in that: The cloud server is further used to determine whether the bottom of the tracking frame of the person reaches a preset image height of the screen of the first monitoring video and disappears in the screen of the first monitoring video; If so, the cloud server determines that the person has left the warehouse.
13. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed, the computer is caused to perform the method according to any one of claims 1 to 6.
14. A computer program product, comprising computer program code, characterized in that: When the computer program code is executed on a computer, the computer is caused to implement the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Self-service shopping method and system for self-service shop
CN108389110A
Commodity processing method and device and unmanned shelf system
CN110610358A
Intelligent monitoring goods theft and loss prevention method and system in warehouse management
CN111223260A
Self-service cashier AI loss prevention system and method
CN111311848A
Machine room carrying equipment detection method and system, equipment and medium
CN113343867A
