Warehouse monitoring processing method and device, server and storage medium

Abstract

The invention provides a warehouse monitoring processing method and device, a server and a storage medium. The method comprises the following steps: segmenting real-time video information to obtain a plurality of continuous frames; inputting each continuous frame into a character recognition model, and determining all continuous frames containing characters as character continuous frames; inputting the character continuous frames into a behavior recognition model, and determining all character continuous frames with abnormal character behaviors as target continuous frames; acquiring access control identity data of people in each target continuous frame, and calculating an identity abnormal value according to the access control identity data; determining a warehouse abnormal value according to the warehouse record data; determining a similarity value between the real-time video information and the historical video information, and calculating a video abnormal value according to the similarity value; obtaining a comprehensive abnormal value according to the identity abnormal value, the warehouse abnormal value and the video abnormal value; and if the comprehensive abnormal value exceeds a preset abnormal threshold value, determining that the warehouse is abnormal. According to the method, the abnormal misjudgment rate is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a warehouse monitoring and processing method, device, server and storage medium. Background Art

[0002] The emergence of automated storage centers has improved logistics efficiency, but there are also certain safety issues. Therefore, it is particularly important to identify abnormal situations in storage centers and respond to them.

[0003] At present, in the existing technology, the warehouse area in the storage center is monitored in real time through equipment such as surveillance cameras and infrared sensors. If unauthorized entry is detected, it is determined to be abnormal.

[0004] However, the existing method has a single standard for abnormality determination, which is prone to misjudgment when special circumstances require unauthorized entry, resulting in a high rate of abnormality misjudgment. Summary of the invention

[0005] The present application provides a warehouse monitoring and processing method, device, server and storage medium to solve the technical problem of high abnormal misjudgment rate.

[0006] In a first aspect, the present application provides a warehouse monitoring and processing method, comprising:

[0007] Receive any real-time video information of the warehouse sent by the image capture device; segment the real-time video information to obtain multiple continuous frames; input each continuous frame into the character recognition model to output whether each continuous frame contains a person, and determine all continuous frames containing a person as person continuous frames; input each person continuous frame into the behavior recognition model to output whether the person behavior in each person continuous frame is abnormal, and determine all person continuous frames with abnormal person behavior as target continuous frames; obtain the access control identity data of the person in each target continuous frame, and calculate the identity anomaly value based on the access control identity data; determine the warehouse anomaly value based on the warehouse record data; determine the similarity value between the real-time video information and the historical video information, and calculate the video anomaly value based on the similarity value; obtain the comprehensive anomaly value based on the identity anomaly value, the warehouse anomaly value and the video anomaly value; if the comprehensive anomaly value exceeds the preset anomaly threshold, it is determined that there is an abnormality in the warehouse.

[0008] In a possible design, the identity anomaly value is calculated based on the access control identity data, including: judging whether there is an abnormality in the person corresponding to the access control identity data from three aspects: warehouse sensitive area, warehouse entry time and warehouse entry time period according to the access control identity data; if it is determined that there is an abnormality, determining the type of abnormality; wherein the abnormality type includes at least one of the following: warehouse sensitive area abnormality, warehouse entry time abnormality and warehouse entry time period abnormality; assigning weights and abnormality values ​​to warehouse sensitive area abnormalities, and obtaining warehouse sensitive area abnormality values; assigning weights and abnormality values ​​to warehouse entry time abnormalities, and obtaining warehouse entry time abnormality values; assigning weights and abnormality values ​​to warehouse entry time period abnormalities, and obtaining warehouse entry time period abnormality values; and accumulating and summing the warehouse sensitive area abnormality values, warehouse entry time abnormality values ​​and warehouse entry time period abnormality values ​​to obtain the identity anomaly value.

[0009] In a possible design, the warehouse record data includes inventory data, warehouse in-and-out data, and cargo movement data; accordingly, the warehouse outliers are determined based on the warehouse record data, including: obtaining inventory outliers based on the inventory data and a preset inventory benchmark value; obtaining warehouse in-and-out outliers based on the warehouse in-and-out data and a preset warehouse in-and-out benchmark value; obtaining cargo movement outliers based on the cargo movement data and a preset cargo movement benchmark value; and determining the warehouse outliers through a weighted summation method based on the inventory outliers, the inventory outliers, and the cargo movement outliers.

[0010] In one possible design, similarity values ​​between real-time video information and historical video information are determined, and video anomaly values ​​are calculated based on the similarity values, including: extracting features from real-time video information to obtain real-time feature data; extracting features from historical video information to obtain historical feature data; determining similarity values ​​between real-time feature data and historical feature data through a similarity measurement method to obtain similarity values ​​between real-time video information and historical video information; and calculating video anomaly values ​​based on the similarity values.

[0011] In one possible design, the video outlier value is calculated based on the similarity value, and the calculation formula is:

[0012]

[0013] Where V represents the video outlier value, Represents similarity value, Represents the maximum similarity value, which is pre-set; the similarity value ranges from 0 to the maximum similarity value.

[0014] In a possible design, if the comprehensive abnormal value exceeds the preset abnormal threshold, after determining that an abnormality exists in the warehouse, it also includes: marking the target continuous frames in the real-time video information, and sending the marked real-time video information to the user management terminal, so that the user management terminal can issue an alarm based on the marked real-time video information.

[0015] In a second aspect, the present application provides a warehouse monitoring and processing device, comprising:

[0016] The receiving module is used to receive any real-time video information of the warehouse sent by the image capturing device.

[0017] The first acquisition module is used to segment the real-time video information to obtain a plurality of continuous frames.

[0018] The first determination module is used to input each continuous frame into a person recognition model to output whether each continuous frame contains a person, and determine all continuous frames containing a person as continuous frames containing a person.

[0019] The second determination module is used to input each character continuous frame into the behavior recognition model to output whether the character behavior in each character continuous frame is abnormal, and determine all character continuous frames with abnormal character behavior as target continuous frames.

[0020] The first calculation module is used to obtain the access control identity data of the person in each target continuous frame, and calculate the identity anomaly value according to the access control identity data.

[0021] The third determination module is used to determine warehouse abnormal values ​​according to warehouse record data.

[0022] The second calculation module is used to determine the similarity value between the real-time video information and the historical video information, and calculate the video abnormality value according to the similarity value.

[0023] The second acquisition module is used to obtain a comprehensive outlier value according to the identity outlier value, the warehouse outlier value and the video outlier value.

[0024] The determination module is used to determine that an abnormality exists in the warehouse if the comprehensive abnormality value exceeds a preset abnormality threshold.

[0025] In a third aspect, the present application provides a server, including:

[0026] at least one processor and memory;

[0027] The memory stores computer-executable instructions;

[0028] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the warehouse monitoring and processing method as described in the first aspect and various possible designs of the first aspect.

[0029] In a fourth aspect, the present application provides a computer storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the warehouse monitoring and processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0030] In a fifth aspect, the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the warehouse monitoring and processing method as described in the first aspect and various possible designs of the first aspect.

[0031] The warehouse monitoring processing method, device, server and storage medium provided by the present application segment any real-time video information of the warehouse to obtain multiple continuous frames; determine all continuous frames containing characters as character continuous frames, and determine all character continuous frames with abnormal character behaviors as target continuous frames. According to the warehouse abnormality value, video abnormality value and the identity abnormality value of the character in the target continuous frame, it is comprehensively judged whether there is an abnormality in the warehouse. Avoid a single abnormality judgment standard, thereby reducing the abnormality misjudgment rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0033] Figure 1 A schematic diagram of an application scenario of the warehouse monitoring and processing method provided in an embodiment of the present application;

[0034] Figure 2 A flowchart of a warehouse monitoring and processing method provided by an embodiment of the present application;

[0035] Figure 3 A schematic diagram of the structure of a monitoring and processing device for a warehouse provided in an embodiment of the present application;

[0036] Figure 4 A schematic diagram of the hardware structure of the server provided in an embodiment of the present application.

[0037] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0038] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0039] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0040] The emergence of automated storage centers has improved logistics efficiency, but there are also certain security issues. Therefore, it is particularly important to identify abnormal situations in storage centers and respond to them. At present, in the prior art, the warehouse area in the storage center is monitored in real time through equipment such as surveillance cameras and infrared sensors. If unauthorized personnel enter, it is judged to be abnormal. However, this method of prior art has a single standard for abnormal judgment, and it is easy to misjudge when there are special circumstances that require unauthorized personnel to enter, resulting in a high rate of abnormal misjudgment.

[0041] In order to solve the above technical problems, the embodiments of the present application propose the following technical ideas: Considering that the single standard for abnormality determination will lead to a high rate of abnormal misjudgment. The inventor thought of a comprehensive judgment from three aspects: warehouse abnormality, video abnormality and character identity abnormality. Segment any real-time video information of the warehouse to obtain multiple continuous frames; determine all continuous frames containing characters as character continuous frames, and determine all character continuous frames with abnormal character behaviors as target continuous frames. According to the warehouse abnormality value, video abnormality value and character identity abnormality value in the target continuous frame, comprehensively judge whether there is an abnormality in the warehouse. Avoid a single standard for abnormality determination, thereby reducing the rate of abnormal misjudgment.

[0042] The warehouse monitoring and processing method provided in this application is intended to solve the above technical problems of the prior art.

[0043] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0044] Figure 1Schematic diagram of application scenario of the warehouse monitoring and processing method provided in the embodiment of the present application. Figure 1 As shown, the application scenario includes: an image capturing device 101 , a server 102 and a user management terminal 103 .

[0045] The image capturing device 101 may be a video camera, a video recorder, a camera, a monitoring device, etc.

[0046] The server 102 may be a single server or a cluster of multiple servers.

[0047] The user management terminal 103 may be a mobile phone, a tablet, a computer, etc.

[0048] refer to Figure 1 The server 102 receives any real-time video information of the warehouse sent by the image capture device 101, and segments the real-time video information to obtain multiple continuous frames, performs a series of processing on the real-time video information, warehouse record data and each continuous frame to obtain a comprehensive abnormality value; if the comprehensive abnormality value exceeds the preset abnormality threshold, it is determined that there is an abnormality in the warehouse; the target continuous frames with abnormalities are marked in the real-time video information, and the marked real-time video information is sent to the user management terminal 103, so that the user management terminal 103 can alarm according to the marked real-time video information.

[0049] Figure 2 A flowchart of a warehouse monitoring and processing method provided by an embodiment of the present application. The execution subject of this embodiment can be Figure 1 The server 102 in the embodiment may also be other servers with similar functions, and this embodiment does not impose any special restrictions here. Figure 2 As shown, the method includes:

[0050] S201: Receive any real-time video information of the warehouse sent by an image capture device.

[0051] Among them, real-time video information has the temporal immediacy and continuity.

[0052] In this embodiment, full coverage monitoring of the warehouse is achieved by optimizing the layout and number of image capture devices and combining them with artificial intelligence analysis.

[0053] Specifically, any real-time video information of the warehouse sent by the image capture device can be received through wireless communication, and any real-time video information of the warehouse sent by the image capture device can also be received through wired communication.

[0054] S202: Segment the real-time video information to obtain a plurality of continuous frames.

[0055] Specifically, the real-time video information is decoded to obtain each frame of the image in the real-time video information; and a plurality of continuous frames are extracted from the decoded real-time video information at fixed time intervals.

[0056] S203: Input each continuous frame into a person recognition model to output whether each continuous frame contains a person, and determine all continuous frames containing a person as continuous frames containing a person.

[0057] Specifically, each consecutive frame is input into the person recognition model one by one, and the person recognition model analyzes the content of each frame and determines whether the frame contains a person. For each frame, the person recognition model outputs a result indicating whether each frame contains a person; if multiple consecutive frames are detected to contain a person, then these frames are considered to be continuous person frames.

[0058] Optionally, it may be set that if 5 consecutive frames contain a person, these frames are determined as consecutive frames containing a person.

[0059] In this embodiment, the process of obtaining a person recognition model is as follows: collect a large number of image or video data sets containing people. This data set contains various people, facial expressions, postures, lighting conditions and backgrounds; for the collected data sets, mark the positions of the people, use the labeled data sets, and use the target detection algorithm for training; during and after the training process, optimize the person recognition model by running the model on the validation set or test set and calculating indicators such as accuracy, recall rate, F1 score, etc., adjusting the model architecture and hyperparameters, etc.

[0060] Optionally, the target detection model includes but is not limited to: YOLO, SSD, and Faster R-CNN.

[0061] S204: Inputting each character's continuous frames into a behavior recognition model to output whether the character's behavior in each character's continuous frames is abnormal, and determining all character's continuous frames with abnormal character behaviors as target continuous frames.

[0062] Among them, the behavior recognition model can identify various behavior patterns and compare them with predefined normal or abnormal behaviors.

[0063] In this embodiment, each character's continuous frames are input into the behavior recognition model, which analyzes the input character's continuous frames and outputs the detection results of the character's behavior in each character's continuous frames; if the detection results do not meet the predefined normal behavior standards, the character's continuous frames are determined as target continuous frames.

[0064] Optionally, the detection result may be a classification of the behavior or a confidence level of the behavior.

[0065] S205: Obtain access control identity data of the person in each target continuous frame, and calculate an identity anomaly value based on the access control identity data.

[0066] Specifically, step S205 includes: Sa~Sf:

[0067] Sa: Based on the access control identity data, determine whether there is any abnormality in the person corresponding to the access control identity data from three aspects: sensitive areas of the warehouse, length of time entering the warehouse, and time period of entering the warehouse.

[0068] Sb: If it is determined that an abnormality exists, the type of abnormality is determined; wherein the abnormality type includes at least one of the following: abnormality in a sensitive area of ​​the warehouse, abnormality in the length of time to enter the warehouse, and abnormality in the time period to enter the warehouse.

[0069] Among them, according to the access control identity data, the warehouse area that the person corresponding to the access control identity normally enters, the normal entry time and the normal entry time period of the warehouse are obtained.

[0070] Sc: Assign weights and outlier values ​​to warehouse sensitive area anomalies, and obtain warehouse sensitive area anomalies.

[0071] In this embodiment, different warehouse sensitive areas, as well as weights and abnormal values ​​corresponding to different warehouse sensitive areas are pre-set. According to the warehouse sensitive area entered by the person corresponding to the access control identity, the weight and abnormal value corresponding to the warehouse sensitive area are determined, and the abnormal value of the warehouse sensitive area is obtained by multiplying them.

[0072] For example, sensitive areas of a warehouse include a valuables area, an important document area, and a security area. The weight of the valuables area is preset to be 0.4, and the abnormal value is 50; the weight of the important document area is preset to be 0.4, and the abnormal value is 100; the weight of the security area is preset to be 0.2, and the abnormal value is 30.

[0073] Sd: Assign weights and outliers to the abnormal time of entering the warehouse, and obtain the abnormal value of the time of entering the warehouse.

[0074] In this embodiment, multiple time thresholds and weights and abnormal values ​​corresponding to each time threshold are preset. If the time for a person corresponding to the access control identity to enter the warehouse normally exceeds a certain threshold, the weight and abnormal value corresponding to the time threshold are obtained.

[0075] For example, the duration thresholds include 5 minutes, 15 minutes, 30 minutes, and 2 hours. If the time to enter the warehouse exceeds 5 minutes, the weight is 0.1, and the abnormal value is 10; if the time to enter the warehouse exceeds 15 minutes, the weight is 0.2, and the abnormal value is 20; if the time to enter the warehouse exceeds 30 minutes, the weight is 0.3, and the abnormal value is 30; if the time to enter the warehouse exceeds 2 hours, the weight is 0.4, and the abnormal value is 40.

[0076] Se: Assign weights and outlier values ​​to the anomalies during the time period of entering the warehouse, and obtain the anomaly values ​​during the time period of entering the warehouse.

[0077] In this embodiment, multiple abnormal time periods are preset, as well as weights and abnormal values ​​corresponding to each abnormal time period. The abnormal time period includes a time period close to the end of get off work, a non-working time period on a working day, and a time period on a non-working day.

[0078] For example, the weight of the time period near the end of get off work is 0.1, and the outlier value is 30; the weight of the non-working time period on working days is 0.3, and the outlier value is 50; the weight of the non-working time period is 0.6, and the outlier value is 100.

[0079] Sf: Accumulate and sum the warehouse sensitive area outliers, warehouse entry time outliers, and warehouse entry time period outliers to obtain the identity outliers.

[0080] S206: Determine warehouse abnormal values ​​according to warehouse record data.

[0081] Among them, warehouse record data includes inventory data, inbound and outbound data, and goods movement data.

[0082] In this embodiment, cloud computing technology is used to manage inventory data, warehouse in and out data, and cargo movement data.

[0083] Specifically, step S206 includes: Sg~Sj:

[0084] Sg: Get the inventory abnormal value based on the inventory data and the preset inventory benchmark value.

[0085] Among them, the preset inventory benchmark value is to set the minimum inventory threshold and the maximum inventory threshold based on multiple factors such as historical data, supply conditions and market demand.

[0086] If the inventory data is higher than the maximum inventory threshold, the calculation formula for the inventory outlier value is as follows:

[0087]

[0088] If the inventory data is lower than the minimum inventory threshold, the calculation formula for the inventory outlier is as follows:

[0089]

[0090] Sh: Get the abnormal value of inbound and outbound according to the inbound and outbound data and the preset inbound and outbound benchmark values.

[0091] Among them, the preset inventory in and out benchmark values ​​are based on historical data, supply conditions, market demand and other factors to set the minimum and maximum inventory in and out thresholds.

[0092] In this embodiment, if the inbound and outbound data is higher than the highest inbound and outbound threshold, the calculation formula of the inbound and outbound abnormal value is as follows:

[0093]

[0094] If the inbound and outbound data is lower than the minimum inbound and outbound threshold, the calculation formula for the inbound and outbound abnormal value is as follows:

[0095]

[0096] Si: According to the cargo movement data and the preset cargo movement benchmark value, the cargo movement abnormal value is obtained.

[0097] Among them, the preset cargo movement reference value is pre-set according to the cargo movement frequency.

[0098] In this embodiment, the calculation formula of the cargo movement abnormal value is as follows:

[0099]

[0100] Sj: Determine warehouse outliers based on inventory outliers, inventory outliers, and goods movement outliers through a weighted sum method.

[0101] Specifically, weights are assigned to inventory anomalies, inbound and outbound anomalies, and cargo movement anomalies, respectively, and the warehouse anomaly value is determined by combining the inventory anomaly value, inventory anomaly value, and cargo movement anomaly value through a weighted summation method.

[0102] S207: Determine a similarity value between the real-time video information and the historical video information, and calculate a video anomaly value based on the similarity value.

[0103] Specifically, step S207 includes: Sk~Sm:

[0104] Sk: Extract features from real-time video information to obtain real-time feature data; extract features from historical video information to obtain historical feature data.

[0105] The historical video information is obtained by selecting video clips similar to the real-time video information from a historical video database.

[0106] In this embodiment, feature extraction is performed on real-time video information and historical video information respectively, and these features may include color, texture, shape, motion trajectory, etc.

[0107] Sl: Determine the similarity value between the real-time feature data and the historical feature data through a similarity measurement method to obtain the similarity value between the real-time video information and the historical video information.

[0108] Specifically, the extracted real-time feature data and historical feature data are encoded into vector form, and a similarity measurement method is used to calculate the similarity value between the feature vector of the real-time video data and the feature vector of the historical video data to obtain the similarity value between the real-time video information and the historical video information.

[0109] Optionally, similarity measurement methods include but are not limited to: Euclidean distance method, cosine similarity method, Manhattan distance method, etc.

[0110] Sm: Calculate video outliers based on similarity values.

[0111] Specifically, the video outlier value is calculated based on the similarity value, and the calculation formula is:

[0112]

[0113] Where V represents the video outlier value, Represents similarity value, Represents the maximum similarity value, which is pre-set; the similarity value ranges from 0 to the maximum similarity value.

[0114] S208: Obtain a comprehensive outlier value according to the identity outlier value, the warehouse outlier value, and the video outlier value.

[0115] Specifically, weights are assigned to identity anomalies, warehouse anomalies, and video anomalies respectively, and the comprehensive anomaly value is obtained by combining the identity anomaly value, warehouse anomaly value, and video anomaly value through weighted summation method.

[0116] S209: If the comprehensive abnormal value exceeds the preset abnormal threshold, it is determined that there is an abnormality in the warehouse.

[0117] In this embodiment, the preset abnormality threshold is a preset maximum limit of the warehouse comprehensive abnormality value.

[0118] In summary, any real-time video information of the warehouse is segmented to obtain multiple continuous frames; all continuous frames containing people are determined as continuous frames of people, and all continuous frames of people with abnormal behavior are determined as target continuous frames. According to the warehouse abnormality value, video abnormality value and the identity abnormality value of the person in the target continuous frame, it is comprehensively judged whether there is an abnormality in the warehouse. Avoid a single abnormality judgment standard, thereby reducing the abnormality misjudgment rate.

[0119] In another embodiment of the present application, based on the above embodiment, a detailed description is given on how to handle after determining that the warehouse is abnormal. The execution subject of this embodiment can be Figure 1 The server 102 in the illustrated embodiment may also be other servers having similar functions, as described in detail below:

[0120] Mark target continuous frames in real-time video information, and send the marked real-time video information to the user management terminal, so that the user management terminal issues an alarm according to the marked real-time video information.

[0121] Optionally, they can be marked with a bounding box or with a color overlay.

[0122] In this embodiment, the marked real-time video information is sent to the user management terminal through various channels, such as text messages, emails, and APP push.

[0123] In summary, marking the target continuous frames in the real-time video information makes it easier for the user management terminal to quickly locate and handle exceptions; sending the marked real-time video information to the user management terminal through multiple channels reduces the reaction time and improves the efficiency of warehouse safety management.

[0124] Figure 3 This is a schematic diagram of the structure of the warehouse monitoring and processing device provided in the embodiment of the present application. Figure 3 As shown, the monitoring and processing device of the warehouse includes: a receiving module 301, a first acquisition module 302, a first determination module 303, a second determination module 304, a first calculation module 305, a third determination module 306, a second calculation module 307, a second acquisition module 308 and a judgment module 309.

[0125] The receiving module 301 is used to receive any real-time video information of the warehouse sent by the image capturing device.

[0126] The first acquisition module 302 is used to segment the real-time video information to obtain a plurality of continuous frames.

[0127] The first determination module 303 is used to input each continuous frame into a person recognition model to output whether each continuous frame contains a person, and determine all continuous frames containing a person as continuous frames containing a person.

[0128] The second determination module 304 is used to input each character continuous frame into the behavior recognition model to output whether the character behavior in each character continuous frame is abnormal, and determine all character continuous frames with abnormal character behavior as target continuous frames.

[0129] The first calculation module 305 is used to obtain the access control identity data of the person in each target continuous frame, and calculate the identity anomaly value according to the access control identity data.

[0130] The third determination module 306 is used to determine warehouse abnormal values ​​according to the warehouse record data.

[0131] The second calculation module 307 is used to determine the similarity value between the real-time video information and the historical video information, and calculate the video abnormality value according to the similarity value.

[0132] The second acquisition module 308 is used to obtain a comprehensive outlier value according to the identity outlier value, the warehouse outlier value and the video outlier value.

[0133] The determination module 309 is used to determine that an abnormality exists in the warehouse if the comprehensive abnormality value exceeds a preset abnormality threshold.

[0134] In one possible design, the first calculation module 305 is specifically used to: determine whether there is an abnormality in the person corresponding to the access control identity data from three aspects: warehouse sensitive area, warehouse entry time and warehouse entry time period according to the access control identity data; if it is determined that there is an abnormality, determine the type of abnormality; wherein the abnormality type includes at least one of the following: warehouse sensitive area abnormality, warehouse entry time abnormality and warehouse entry time period abnormality; assign weights and abnormal values ​​to warehouse sensitive area abnormalities, and obtain warehouse sensitive area abnormality values; assign weights and abnormal values ​​to warehouse entry time abnormalities, and obtain warehouse entry time abnormality values; assign weights and abnormal values ​​to warehouse entry time period abnormalities, and obtain warehouse entry time period abnormality values; assign weights and abnormal values ​​to warehouse entry time period abnormalities, and obtain warehouse entry time period abnormality values; accumulate and sum the warehouse sensitive area abnormality values, warehouse entry time abnormality values, and warehouse entry time period abnormality values ​​to obtain the identity abnormality value.

[0135] In one possible design, the warehouse record data includes inventory data, warehouse in-and-out data, and cargo movement data; accordingly, the third determination module 306 is specifically used to: obtain inventory abnormal values ​​based on inventory data and preset inventory benchmark values; obtain warehouse in-and-out abnormal values ​​based on warehouse in-and-out data and preset warehouse in-and-out benchmark values; obtain cargo movement abnormal values ​​based on cargo movement data and preset cargo movement benchmark values; determine warehouse abnormal values ​​through a weighted summation method based on inventory abnormal values, inventory abnormal values, and cargo movement abnormal values.

[0136] In one possible design, the second computing module 307 is specifically used to: perform feature extraction on real-time video information to obtain real-time feature data; perform feature extraction on historical video information to obtain historical feature data; determine the similarity value between the real-time feature data and the historical feature data by a similarity measurement method to obtain the similarity value between the real-time video information and the historical video information; and calculate the video anomaly value based on the similarity value.

[0137] In a possible design, the calculation formula of the second calculation module 307′ is:

[0138]

[0139] Where V represents the video outlier value, Represents similarity value, Represents the maximum similarity value, which is pre-set; the similarity value ranges from 0 to the maximum similarity value.

[0140] In one possible design, the warehouse monitoring and processing device also includes: a sending module, which is used to mark the target continuous frames in the real-time video information, and send the marked real-time video information to the user management terminal, so that the user management terminal can alarm according to the marked real-time video information.

[0141] Figure 4 The hardware structure diagram of the server provided in the embodiment of the present application is shown in FIG. Figure 4 As shown, the server of this embodiment includes: at least one processor 401 and a memory 402; the memory stores computer execution instructions; at least one processor executes the computer execution instructions stored in the memory, so that at least one processor executes the above warehouse monitoring and processing method.

[0142] Optionally, the memory 402 may be independent or integrated with the processor 401 .

[0143] When the memory 402 is independently provided, the server further includes a bus 403 for connecting the memory 402 and the processor 401 .

[0144] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the warehouse monitoring and processing method as described above is implemented.

[0145] An embodiment of the present application also provides a computer program product, including a computer program, which is stored in a computer storage medium. At least one processor can read the computer program from the computer storage medium. When at least one processor executes the computer program, the above-mentioned warehouse monitoring and processing method can be implemented.

[0146] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0147] The modules described above as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0148] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0149] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the above-mentioned method in each embodiment of the present application.

[0150] It should be understood that the above processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0151] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0152] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0153] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0154] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0155] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A warehouse monitoring and processing method, characterized in that: Applicable to servers, including: Receive any real-time video information of the warehouse sent by the image capture device; Segmenting the real-time video information to obtain a plurality of continuous frames; Inputting each continuous frame into a person recognition model to output whether each continuous frame contains a person, and determining all continuous frames containing a person as continuous frames containing a person; Inputting each character continuous frame into a behavior recognition model to output whether the character behavior in each character continuous frame is abnormal, and determining all character continuous frames with abnormal character behavior as target continuous frames; Obtaining access control identity data of the person in each target continuous frame, and calculating an identity anomaly value based on the access control identity data; Determine warehouse outliers based on warehouse record data; Determine a similarity value between the real-time video information and the historical video information, and calculate a video anomaly value based on the similarity value; Obtaining a comprehensive outlier value according to the identity outlier value, the warehouse outlier value, and the video outlier value; If the comprehensive abnormal value exceeds the preset abnormal threshold, it is determined that an abnormality exists in the warehouse.

2. The method according to claim 1, characterized in that The calculating of the identity anomaly value according to the access control identity data comprises: Based on the access control identity data, determine whether there is any abnormality in the person corresponding to the access control identity data from three aspects: warehouse sensitive area, length of time entering the warehouse, and time period of entering the warehouse; If it is determined that an abnormality exists, the type of abnormality is determined; wherein the type of abnormality includes at least one of the following: abnormality in a sensitive area of ​​the warehouse, abnormality in the length of time to enter the warehouse, and abnormality in the time period to enter the warehouse; Assigning weights and abnormal values ​​to the abnormalities in the warehouse sensitive area, and obtaining the abnormal values ​​in the warehouse sensitive area; Assigning a weight and an abnormal value to the abnormal time length for entering the warehouse, and obtaining the abnormal value of the time length for entering the warehouse; Assigning a weight and an abnormal value to the abnormal time period of entering the warehouse, and obtaining the abnormal value of the time period of entering the warehouse; The warehouse sensitive area abnormal value, the warehouse entry time abnormal value, and the warehouse entry time period abnormal value are accumulated and summed to obtain the identity abnormal value.

3. The method according to claim 1, characterized in that The warehouse record data includes inventory data, in-and-out warehouse data, and goods movement data; Accordingly, determining the warehouse abnormal value according to the warehouse record data includes: Obtaining an abnormal inventory value according to the inventory data and a preset inventory reference value; Obtaining abnormal values ​​of inbound and outbound storage according to the inbound and outbound storage data and preset inbound and outbound storage benchmark values; Obtaining a cargo movement abnormality value according to the cargo movement data and a preset cargo movement reference value; The warehouse outlier value is determined by a weighted sum method according to the inventory outlier value, the inventory outlier value and the cargo movement outlier value.

4. The method according to claim 1, characterized in that The determining of the similarity value between the real-time video information and the historical video information, and calculating the video abnormality value according to the similarity value, comprises: Extracting features from the real-time video information to obtain real-time feature data; Extract features from historical video information to obtain historical feature data; Determining similarity values ​​between the real-time feature data and the historical feature data by a similarity measurement method to obtain similarity values ​​between the real-time video information and the historical video information; A video outlier value is calculated according to the similarity value.

5. The method according to claim 4, characterized in that The calculation formula for calculating the video abnormality value based on the similarity value is: Wherein, V represents the video abnormal value, represents the similarity value, represents a maximum similarity value, and the maximum similarity value is preset; wherein the similarity value ranges from 0 to the maximum similarity value.

6. The method according to any one of claims 1 to 5, characterized in that: If the comprehensive abnormal value exceeds the preset abnormal threshold, after determining that the warehouse is abnormal, the method further includes: The target continuous frames are marked in the real-time video information, and the marked real-time video information is sent to the user management terminal, so that the user management terminal issues an alarm according to the marked real-time video information.

7. A warehouse monitoring and processing device, characterized in that: Applicable to servers, including: A receiving module, used for receiving any real-time video information of the warehouse sent by the image capturing device; A first acquisition module is used to segment the real-time video information to obtain a plurality of continuous frames; A first determination module is used to input each continuous frame into a person recognition model to output whether each continuous frame contains a person, and determine all continuous frames containing a person as continuous frames containing a person; The second determination module is used to input each character continuous frame into the behavior recognition model to output whether the character behavior in each character continuous frame is abnormal, and determine all character continuous frames with abnormal character behavior as target continuous frames; A first calculation module is used to obtain access control identity data of a person in each target continuous frame, and calculate an identity anomaly value based on the access control identity data; A third determination module is used to determine warehouse abnormal values ​​based on warehouse record data; A second calculation module is used to determine the similarity value between the real-time video information and the historical video information, and calculate the video abnormality value according to the similarity value; A second acquisition module is used to obtain a comprehensive outlier value according to the identity outlier value, the warehouse outlier value and the video outlier value; The determination module is used to determine that there is an abnormality in the warehouse if the comprehensive abnormal value exceeds a preset abnormality threshold.

8. A server, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the warehouse monitoring and processing method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that: The computer storage medium stores computer execution instructions. When the processor executes the computer execution instructions, the warehouse monitoring and processing method as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the warehouse monitoring and processing method according to any one of claims 1 to 6.

Images