Warehouse monitoring system based on video monitoring electronic fence intelligent early warning function

Abstract

The invention provides a warehouse monitoring system based on a video monitoring electronic fence intelligent early warning function, and relates to the technical field of video monitoring, and video monitoring data in an electronic fence area in a warehouse are collected in real time; selecting an image frame in the preprocessed video monitoring data, determining a color distribution difference degree, performing background filtering according to the color distribution difference degree to obtain a target recognition image, and determining a moving target detection factor through the target recognition image; obtaining audio data and target behavior track data based on the moving target detection factor, and performing feature extraction on the audio data and the target behavior track data to obtain a fundamental tone feature vector and a behavior track feature vector; and inputting the fundamental tone feature vector and the behavior track feature vector into a prediction model for behavior prediction, and sending early warning information to a warehouse monitoring center according to a behavior prediction result, so that the abnormal behavior in the electronic fence area can be accurately identified, and the early warning reliability of a warehouse monitoring system is improved.

Description

Technical Field

[0001] This application relates to the field of video surveillance technology. More specifically, this application relates to a warehouse monitoring system based on the intelligent early warning function of a video surveillance electronic fence. Background Art

[0002] With the rapid development of the logistics industry, as the core node for material storage and transfer, the security and management efficiency of warehouses have become the focus of attention for enterprises. Traditional warehouse monitoring systems mainly rely on manual inspections and simple video surveillance, but there are problems such as many monitoring blind spots, high labor supervision costs, and lagging alarm systems, making it difficult to meet the security requirements of modern warehouses. For this reason, a new type of warehouse monitoring system based on video surveillance + electronic fence + intelligent early warning has emerged. It combines multiple technologies such as AI visual analysis, target detection, behavior recognition, and environmental perception to achieve real-time monitoring of the internal and external environments of warehouses, analysis of abnormal behaviors, and early warning linkages, improving the intelligent level of warehouse security.

[0003] However, in the actual application process, video surveillance electronic fences rely on high-precision image recognition and intelligent algorithms, but are prone to false alarms or missed alarms in complex environments (such as light changes, interference from obstacles, and high-shelf structures inside the warehouse), affecting the accuracy of early warnings, that is, misidentifying normal behaviors as abnormal behaviors, which will reduce the reliability and usability of intelligent monitoring systems. Therefore, how to accurately identify abnormal behaviors within the electronic fence area to improve the early warning reliability of warehouse monitoring systems is a difficult problem faced by the industry. Summary of the Invention

[0004] This application provides a warehouse monitoring system based on the intelligent early warning function of a video surveillance electronic fence, which can accurately identify abnormal behaviors within the electronic fence area to improve the early warning reliability of the warehouse monitoring system.

[0005] This application provides a warehouse monitoring system based on the intelligent early warning function of a video surveillance electronic fence, and the monitoring system includes:

[0006] A data acquisition module, which is used to collect video surveillance data within the electronic fence area of the warehouse in real time and preprocess the collected video surveillance data;

[0007] A data analysis module, which is used to select image frames from the preprocessed video surveillance data, determine the color distribution difference degree between the selected image frames and adjacent image frames, filter the background of the selected image frames and adjacent image frames based on the color distribution difference degree, and then obtain the target recognition image within the electronic fence area, and determine the moving target detection factor through the target recognition image;

[0008] A feature extraction module, configured to obtain audio data and target behavior trajectory data within the electronic fence area based on the moving target detection factor, and respectively extract features from the audio data and the target behavior trajectory data to obtain a pitch feature vector and a behavior trajectory feature vector within the electronic fence area;

[0009] An anomaly warning module, configured to input the pitch feature vector and the behavior trajectory feature vector into a prediction model based on deep learning for behavior prediction, and then send a warning message to the warehouse monitoring center according to the behavior prediction result.

[0010] In this embodiment, video monitoring data within the electronic fence area in the warehouse is collected in real time through a high-definition camera.

[0011] In this embodiment, preprocessing the collected video monitoring data is to remove noise and perform color correction on the collected video monitoring data.

[0012] In this embodiment, determining the color distribution difference degree between the selected image frame and the adjacent image frames specifically includes:

[0013] Extracting the previous frame image and the next frame image of the selected image frame in the preprocessed video monitoring data as adjacent image frames;

[0014] Respectively extracting the color distribution features of the selected image frame, the previous frame image, and the next frame image;

[0015] Determining the previous frame difference value through the color distribution feature of the selected image frame and the color distribution feature of the previous frame image;

[0016] Determining the next frame difference value through the color distribution feature of the selected image frame and the color distribution feature of the next frame image;

[0017] Determining the color distribution difference degree between the selected image frame and the adjacent image frames according to the previous frame difference value and the next frame difference value.

[0018] In this embodiment, filtering the background of the selected image frame and the adjacent image frames according to the color distribution difference degree, and then obtaining the target recognition image within the electronic fence area specifically includes:

[0019] When the color distribution difference degree is greater than a preset difference threshold, performing differencing on the selected image frame and the adjacent image frames to obtain a first differenced image and a second differenced image;

[0020] Respectively performing background division on the first differenced image and the second differenced image, and then fusing the background-divided first differenced image and the background-divided second differenced image to obtain the target recognition image within the electronic fence area.

[0021] In this embodiment, when the moving target detection factor is greater than a preset detection threshold, audio data within the electronic fence area is acquired through a microphone array.

[0022] In this embodiment, when the moving target detection factor is greater than a preset detection threshold, target behavior trajectory data within the electronic fence area is acquired through a target detection algorithm.

[0023] In this embodiment, feature extraction is respectively performed on the audio data and the target behavior trajectory data to obtain a pitch feature vector and a behavior trajectory feature vector within the electronic fence area, which specifically includes:

[0024] Perform threshold filtering on the audio data to obtain audio-filtered data;

[0025] Perform feature construction on the audio-filtered data to obtain a pitch feature vector within the electronic fence area;

[0026] Perform feature extraction on the target behavior trajectory data to obtain a behavior trajectory feature vector within the electronic fence area.

[0027] In this embodiment, performing feature construction on the audio-filtered data to obtain a pitch feature vector within the electronic fence area specifically includes:

[0028] Determine the scale feature of the audio-filtered data;

[0029] Extract the fundamental frequency value of each frame of audio in the audio-filtered data, and then obtain all the fundamental frequency values. Determine the pitch mean of the audio-filtered data based on all the fundamental frequency values;

[0030] Determine the unvoiced fluctuation feature of the audio-filtered data;

[0031] Construct a pitch feature vector within the electronic fence area through the scale feature, the pitch mean, and the unvoiced fluctuation feature.

[0032] In this embodiment, sending a warning message to the warehouse monitoring center according to the behavior prediction result specifically includes:

[0033] When the behavior prediction result is normal, no warehouse warning is triggered, and the behavior trajectory data is stored;

[0034] When the behavior prediction result is abnormal, a warehouse warning is triggered, and then a warning message is generated according to the abnormal level in the behavior prediction result, and the warning message is sent to the warehouse monitoring center.

[0035] The technical solutions provided by the disclosed embodiments of this application have the following beneficial effects:

[0036] The video surveillance data within the electronic fence area in the warehouse is collected in real time through the data collection module, and the collected video surveillance data is preprocessed; the data analysis module selects an image frame from the preprocessed video surveillance data, determines the color distribution difference degree between the selected image frame and the adjacent image frames, filters the background of the selected image frame and the adjacent image frames based on the color distribution difference degree, and then obtains the target recognition image within the electronic fence area, and determines the moving target detection factor through the target recognition image; the feature extraction module obtains the audio data and the target behavior trajectory data within the electronic fence area based on the moving target detection factor, and respectively extracts features from the audio data and the target behavior trajectory data to obtain the pitch feature vector and the behavior trajectory feature vector within the electronic fence area; the abnormal warning module inputs the pitch feature vector and the behavior trajectory feature vector into the prediction model based on deep learning for behavior prediction, and then sends a warning message to the warehouse monitoring center according to the behavior prediction result.

[0037] Thus, it can be seen that in this application, first, by selecting an image frame from the preprocessed video surveillance data and calculating the color distribution difference degree, the dynamically changing parts in the image can be effectively identified. On the basis of background filtering, the obtained target recognition image can more clearly identify the moving targets in the warehouse; then, based on the moving target detection factor, the audio data and the target behavior trajectory data within the electronic fence area are obtained, and features are extracted from them to obtain the pitch feature vector and the behavior trajectory feature vector, which helps to improve the recognition accuracy of abnormal behaviors; finally, inputting the pitch feature vector and the behavior trajectory feature vector into the prediction model based on deep learning for behavior prediction can comprehensively analyze the sound signal and movement pattern of the target, accurately identify potential abnormal behaviors, reduce false alarms and missed alarms, and improve the warning reliability of the warehouse monitoring system.

[0038] In summary, the technical solution adopted in this application can accurately identify abnormal behaviors within the electronic fence area to improve the warning reliability of the warehouse monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0040] Figure 1 It is a module structure diagram of a warehouse monitoring system with an intelligent warning function based on video surveillance electronic fence provided by the present application;

[0041] Figure 2It is an exemplary flowchart for determining a target recognition image within an electronic fence area provided according to the present application;

[0042] Figure 3 It is an exemplary flowchart for determining a pitch feature vector and a behavior trajectory feature vector within an electronic fence area provided according to the present application. Specific embodiments

[0043] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present application.

[0044] The embodiment of the present application provides a warehouse monitoring system with an intelligent early warning function based on video surveillance electronic fence. Its core is to collect video surveillance data within the electronic fence area in the warehouse in real time through a data collection module, and preprocess the collected video surveillance data; the data analysis module selects image frames from the preprocessed video surveillance data, determines the color distribution difference degree between the selected image frames and adjacent image frames, filters the background of the selected image frames and adjacent image frames according to the color distribution difference degree, and then obtains a target recognition image within the electronic fence area, and determines a moving target detection factor through the target recognition image; the feature extraction module obtains audio data and target behavior trajectory data within the electronic fence area based on the moving target detection factor, and respectively extracts features from the audio data and the target behavior trajectory data to obtain a pitch feature vector and a behavior trajectory feature vector within the electronic fence area; the abnormal early warning module inputs the pitch feature vector and the behavior trajectory feature vector into a prediction model based on deep learning for behavior prediction, and then sends an early warning message to the warehouse monitoring center according to the behavior prediction result. By adopting the above solution, abnormal behaviors within the electronic fence area can be accurately identified to improve the early warning reliability of the warehouse monitoring system.

[0045] To better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments. Refer to Figure 1 As shown, this figure is a module structure diagram of a warehouse monitoring system with an intelligent early warning function based on video surveillance electronic fence according to the present embodiment of the present application. The monitoring system includes: a data collection module 100, a data analysis module 200, a feature extraction module 300, and an abnormal early warning module 300, which are described as follows:

[0046] The data collection module 100 is used to collect video surveillance data within the electronic fence area in the warehouse in real time and preprocess the collected video surveillance data.

[0047] In specific implementation, high-definition cameras can be used to collect video surveillance data in the electronic fence area of ​​the warehouse in real time; through reasonable high-definition camera deployment, network transmission, and data optimization, the efficiency, real-time and stability of warehouse safety monitoring can be ensured. Among them, the video surveillance data is composed of a series of image frames arranged in chronological order.

[0048] In this embodiment, the preprocessing of the collected video surveillance data is to remove noise and perform color correction on the collected video surveillance data; in specific implementation, firstly, the video surveillance data may have noise in low-light environment, camera jitter, network transmission, etc., which affects the monitoring effect. The goal of noise removal is to reduce random noise, improve video quality, and enhance target clarity. A neural network-based image denoising algorithm can be used to adaptively learn noise characteristics and perform denoising; then, the purpose of color correction is to correct lighting changes, improve white balance, and enhance contrast to ensure that the video picture truly restores the warehouse environment. A multi-scale contrast enhancement method can be used to optimize the color balance under different lighting conditions, thereby achieving color correction of the collected video surveillance data.

[0049] The data analysis module 200 is used to select an image frame from the preprocessed video surveillance data, determine the color distribution difference between the selected image frame and adjacent image frames, perform background filtering on the selected image frame and adjacent image frames based on the color distribution difference, and then obtain a target recognition image within the electronic fence area, and determine a moving target detection factor through the target recognition image.

[0050] In this embodiment, the color distribution difference between the selected image frame and the adjacent image frames may be determined in the following manner, namely:

[0051] Extracting a previous image frame and a subsequent image frame of the selected image frame from the preprocessed video surveillance data as adjacent image frames;

[0052] respectively extracting the color distribution features of the selected image frame, the previous image frame and the next image frame;

[0053] Determine the difference value of the previous frame by using the color distribution characteristics of the selected image frame and the color distribution characteristics of the previous frame image;

[0054] Determine a difference value of a subsequent frame by using the color distribution characteristics of the selected image frame and the color distribution characteristics of the subsequent frame image;

[0055] The color distribution difference between the selected image frame and the adjacent image frame is determined according to the previous frame difference value and the next frame difference value.

[0056] In specific implementation, first, the previous frame image and the next frame image of the selected image frame can be extracted from the preprocessed video surveillance data as adjacent image frames. Then, the color distribution features of the selected image frame, the previous frame image, and the next frame image can be extracted respectively. The color distribution feature is a feature that describes the color distribution characteristics of an image. The first moment, second moment, and third moment of the color of the selected image frame can be calculated, and thus the first moment, second moment, and third moment of the color of the selected image frame are used as distribution feature values, and the feature vector composed of all distribution feature values is used as the color distribution feature of the selected image frame. Through the above method, the color distribution features of the selected image frame, the previous frame image, and the next frame image can be obtained.

[0057] In addition, in specific implementation, first, the previous frame difference value can be determined through the color distribution feature of the selected image frame and the color distribution feature of the previous frame image. The previous frame difference value represents the degree of color distribution difference between the selected image frame and the previous frame image. In actual implementation, the previous frame difference value can be determined by the following formula:

[0058]

[0059] where γ represents the previous frame difference value, F a (i) represents the i-th distribution feature value in the color distribution feature of the selected image frame, and F a-1 (i) represents the i-th distribution feature value in the color distribution feature of the previous frame image. Then, the next frame difference value can be determined in the above way. The next frame difference value represents the degree of color distribution difference between the selected image frame and the next frame image, which will not be elaborated here. Finally, the color distribution difference degree between the selected image frame and the adjacent image frames can be determined according to the previous frame difference value and the next frame difference value. The color distribution difference degree represents the degree of color distribution difference between the selected image frame and the adjacent image frames. In actual implementation, the previous frame difference value and the next frame difference value can be used as two values of the color distribution difference degree, so as to obtain the color distribution difference degree between the selected image frame and the adjacent image frames.

[0060] Preferably, in this embodiment, as shown in Figure 2 This figure is an exemplary flowchart for determining the target recognition image within the electronic fence area in an embodiment of the present application. In this embodiment, background filtering is performed on the selected image frame and the adjacent image frames according to the color distribution difference degree, and then the target recognition image within the electronic fence area can be obtained specifically by the following steps:

[0061] In step S21, when the color distribution difference degree is greater than a preset difference threshold, the selected image frame and the adjacent image frames are differentiated to obtain a first differential image and a second differential image.

[0062] In step S22, background division is respectively performed on the first difference image and the second difference image, and then the first difference image after background division and the second difference image after background division are fused to obtain a target recognition image within the electronic fence area.

[0063] In specific implementation, first, a difference threshold can be preset according to historical experience and data analysis, which will not be elaborated here; then, when the color distribution difference degree is greater than the preset difference threshold, that is, when both values of the color distribution difference degree are greater than the preset difference threshold, the selected image frame can be differentiated from the adjacent image frames, that is, the selected image frame is respectively differentiated from the previous frame image and the next frame image, so that the difference image between the selected image frame and the previous frame image is used as the first difference image, and the difference image between the selected image frame and the next frame image is used as the second difference image; further, background division can be respectively performed on the first difference image and the second difference image, that is, each pixel point in the first difference image is compared with the set adaptive threshold, and the adaptive threshold can be set by the method of maximum between-class variance. When the pixel point value is greater than the adaptive threshold, it is determined that the pixel point belongs to the moving target, and the value of the pixel point is set to 1, and the pixel points with values less than the adaptive threshold are divided into the background area, and the pixel values are set to 0, so that the first difference image after background division can be obtained. By the above method, the background division of the first difference image and the second difference image can be completed, which will not be elaborated here; finally, the first difference image after background division and the second difference image after background division can be fused, that is, the AND operation is performed on the first difference image after background division and the second difference image after background division, so that the obtained image can be used as the target recognition image within the electronic fence area.

[0064] It should be noted that when the color distribution difference degree is not greater than the preset difference threshold, an image frame needs to be reselected from the preprocessed video surveillance data, and the color distribution difference degree needs to be recalculated until the color distribution difference degree is greater than the preset difference threshold.

[0065] In this embodiment, a moving target detection factor is determined through the target recognition image; it should be noted that in this application, the moving target detection factor is an index used to judge the possibility of the existence of a moving target in the target recognition image; in actual implementation, the total number of pixel points with a value of 1 in the target recognition image can be counted, and the ratio of the statistical result to the total number of pixel points in the target recognition image is used as the moving target detection factor.

[0066] It should be noted that by selecting image frames from the preprocessed video surveillance data and calculating the color distribution difference degree, the dynamically changing parts in the image can be effectively identified. By comparing the color changes between adjacent frames, the background and foreground can be accurately separated, thereby improving the accuracy of target recognition. Based on background filtering, the target recognition image obtained can more clearly identify the moving targets in the warehouse.

[0067] The feature extraction module 300 is configured to obtain audio data and target behavior trajectory data within the electronic fence area based on the moving target detection factor, and perform feature extraction on the audio data and the target behavior trajectory data respectively to obtain a pitch feature vector and a behavior trajectory feature vector within the electronic fence area.

[0068] In this embodiment, when the moving target detection factor is greater than a preset detection threshold, audio data within the electronic fence area is obtained through a microphone array. When the moving target detection factor is greater than a preset detection threshold, target behavior trajectory data within the electronic fence area is obtained through a target detection algorithm.

[0069] Specifically, first, a detection threshold can be set according to historical experience. This detection threshold is used to determine whether there are moving targets within the electronic fence area. Then, the moving target detection factor is compared with the detection threshold. When the moving target detection factor is not greater than the preset detection threshold, it is determined that there are no moving targets within the electronic fence area at this time, and image frames need to be reselected from the preprocessed video surveillance data, and the moving target detection factor is recalculated until the moving target detection factor is greater than the preset detection threshold. When the moving target detection factor is greater than the preset detection threshold, it is determined that there are moving targets within the electronic fence area at this time. Then, audio data within the electronic fence area can be obtained through a microphone array, and target behavior trajectory data within the electronic fence area can be obtained through a target detection algorithm. It should be noted that the audio data and the target behavior trajectory data are obtained starting from the moment of the selected image frame corresponding to the moving target detection factor greater than the preset detection threshold.

[0070] Preferably, in this embodiment, refer to Figure 3 As shown in the figure, this figure is an exemplary flowchart for determining the pitch feature vector and the behavior trajectory feature vector within the electronic fence area in an embodiment of the present application. In this embodiment, feature extraction is respectively performed on the audio data and the target behavior trajectory data to obtain the pitch feature vector and the behavior trajectory feature vector within the electronic fence area, which can be specifically implemented by the following steps:

[0071] In step S31, threshold filtering is performed on the audio data to obtain audio filtered data;

[0072] In step S32, feature construction is performed on the audio filtering data to obtain a pitch feature vector within the electronic fence area;

[0073] In step S33, feature extraction is performed on the target behavior trajectory data to obtain a behavior trajectory feature vector within the electronic fence area.

[0074] In specific implementation, first, threshold filtering can be performed on the audio data, that is, the short-time energy mean of all audio frames in the audio data is used as the short-time energy of the audio data, and this short-time energy is used to reflect the characteristics of the voiced part in the audio data. The zero-crossing rate mean of all audio frames in the audio data is used as the zero-crossing rate of the audio data, and this zero-crossing rate is used to reflect the characteristics of the unvoiced part in the audio data. The short-time energy and the zero-crossing rate are input into a double-threshold endpoint detection algorithm to perform threshold filtering on the audio data stream, and audio filtering data is obtained. Through threshold filtering, background noise and interference content in the audio data can be removed. Then, feature construction can be performed on the audio filtering data to obtain a pitch feature vector within the electronic fence area. Finally, feature extraction can be performed on the target behavior trajectory data. The starting position coordinates, acceleration, target movement direction, and behavior pattern features of the moving target at the corresponding moment are extracted through a target detection algorithm. Thus, the target position, speed, acceleration, movement direction, and movement pattern features are combined into a feature vector, and this feature vector is used as the behavior trajectory feature vector. This behavior trajectory feature vector can reflect the dynamic behavior of the moving target in the warehouse and can effectively distinguish normal behavior and abnormal behavior.

[0075] In this embodiment, the following method can be specifically adopted to perform feature construction on the audio filtering data to obtain a pitch feature vector within the electronic fence area, that is:

[0076] Determine the scale feature of the audio filtering data;

[0077] Extract the fundamental frequency value of each frame of audio in the audio filtering data, and then all fundamental frequency values are obtained. The pitch mean of the audio filtering data is determined according to all the fundamental frequency values;

[0078] Determine the unvoiced fluctuation feature of the audio filtering data;

[0079] Construct a pitch feature vector within the electronic fence area through the scale feature, the pitch mean, and the unvoiced fluctuation feature.

[0080] In specific implementation, first, the scale features of the audio filtering data can be calculated through a Mel filter bank, and the scale features are used to represent the overall scale characteristics of the audio filtering data; second, the fundamental frequency value of each frame of audio in the audio filtering data can be extracted through the cepstrum method. The fundamental frequency value represents the basic frequency value of the audio frame. The mean value of all the fundamental frequency values can be used as the pitch mean of the audio filtering data, and the pitch mean can be used to distinguish the high-pitched part in the audio filtering data; then, the variance of the zero-crossing rate of each frame of audio in the audio filtering data can be calculated, and the mean value of all the zero-crossing rate variances can be used as the voiceless fluctuation feature of the monitored audio segment. The voiceless fluctuation feature is used to represent the fluctuation degree of the voiceless part in the audio filtering data; finally, an empty vector with a dimension of 14 is initialized, and the scale features, the pitch mean, and the voiceless fluctuation feature are filled into the empty vector respectively, that is, construction is performed, so as to obtain the pitch feature vector in the electronic fence area. In this application, the dimension of the scale features is 12, the dimension of the pitch mean is 1, and the dimension of the voiceless fluctuation feature is 1; it should be noted that in this application, the pitch feature vector is a feature vector used to describe the audio feature information in the electronic fence area.

[0081] It should be noted that obtaining the audio data and the target behavior trajectory data in the electronic fence area based on the moving target detection factor and performing feature extraction on them to obtain the pitch feature vector and the behavior trajectory feature vector helps to improve the recognition accuracy of abnormal behaviors. The audio data can capture sudden sounds in the environment, while the behavior trajectory data provides information about the movement pattern of the target.

[0082] The abnormal warning module 400 is used to input the pitch feature vector and the behavior trajectory feature vector into a prediction model based on deep learning for behavior prediction, and then send a warning message to the warehouse monitoring center according to the behavior prediction result.

[0083] It should be noted that through the prediction model based on deep learning, the system can learn the potential complex relationships in these feature vectors, identify the behavior trends of the target. The model can automatically learn and extract the deep-level connections between the features, so as to effectively distinguish normal behaviors and abnormal behaviors; in this application, the selected prediction model based on deep learning is the long short-term memory network model. In actual implementation, other prediction models based on deep learning can also be selected, which is not limited here; for the prediction model based on deep learning, the input pitch feature vector and behavior trajectory feature vector will be processed and the behavior type of the moving target will be predicted, that is, the behavior prediction result. The behavior prediction result includes the behavior type of the moving target, such as normal behavior and abnormal behavior. If the behavior type is abnormal behavior, the behavior prediction result also includes the abnormal level of the abnormal behavior, and the abnormal level is used to measure the abnormal degree of the moving target behavior.

[0084] In this embodiment, the warning information is sent to the warehouse monitoring center according to the behavior prediction result, which can be specifically implemented in the following manner, that is:

[0085] When the behavior prediction result is normal, the warehouse warning is not triggered, and the behavior trajectory data is stored.

[0086] When the behavior prediction result is abnormal, the warehouse warning is triggered, and then a warning information is generated according to the abnormal level in the behavior prediction result, and the warning information is sent to the warehouse monitoring center.

[0087] Specifically, when the behavior prediction result is normal, the system will not trigger any alarm or alert notification, and for subsequent analysis and recording, the system will store the behavior trajectory data of the target in the database as historical data backup; when the behavior prediction result is abnormal, the system will immediately trigger a warning. The system generates corresponding warning information according to the abnormal type and abnormal level (such as: minor, serious, critical, etc.) in the behavior prediction result. The warning information will include the abnormal behavior type, abnormal level, occurrence location and time, and target information. The system will send the generated warning information to the warehouse monitoring center in real time, including relevant parties such as monitoring personnel and management personnel, to ensure that the warning information can be quickly known and responded to by relevant personnel.

[0088] It should be noted that inputting the pitch feature vector and the behavior trajectory feature vector into the prediction model based on deep learning for behavior prediction can comprehensively analyze the sound signal and motion pattern of the target, accurately identify potential abnormal behaviors. The warehouse monitoring system can send warning information in a timely manner when abnormal behaviors occur, thereby providing an accurate and rapid response basis for management personnel, effectively improving the warning reliability of the warehouse monitoring system, reducing false alarms and missed alarms, and ensuring that the safety management of the warehouse is more efficient and intelligent.

[0089] Thus, in this application, first, by selecting image frames in the preprocessed video monitoring data and calculating the color distribution difference degree, the dynamically changing parts in the image can be effectively identified. On the basis of background filtering, the obtained target recognition image can more clearly identify the moving targets in the warehouse; then, based on the moving target detection factor, the audio data and target behavior trajectory data in the electronic fence area are obtained, and their features are extracted to obtain the pitch feature vector and the behavior trajectory feature vector, which helps to improve the recognition accuracy of abnormal behaviors; finally, inputting the pitch feature vector and the behavior trajectory feature vector into the prediction model based on deep learning for behavior prediction can comprehensively analyze the sound signal and motion pattern of the target, accurately identify potential abnormal behaviors, reduce false alarms and missed alarms, and improve the warning reliability of the warehouse monitoring system.

[0090] In summary, the technical solution adopted by this application can accurately identify abnormal behaviors within the electronic fence area to improve the early warning reliability of the warehouse monitoring system.

[0091] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 block or multiple blocks.

[0092] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable storage medium, and the storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM), or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0093] It should also be noted that the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the element.

Claims

1. A warehouse monitoring system with an intelligent early warning function based on a video surveillance electronic fence, characterized in that The monitoring system includes: A data acquisition module, which is used to collect video monitoring data in the electronic fence area of the warehouse in real time and preprocess the collected video monitoring data; A data analysis module, which is used to select image frames from the preprocessed video monitoring data, determine the color distribution difference degree between the selected image frames and adjacent image frames, filter the background of the selected image frames and adjacent image frames according to the color distribution difference degree, and then obtain the target recognition image in the electronic fence area, and determine the moving target detection factor through the target recognition image; A feature extraction module, which is used to obtain the audio data and target behavior trajectory data in the electronic fence area based on the moving target detection factor, extract features from the audio data and the target behavior trajectory data respectively, and obtain the pitch feature vector and behavior trajectory feature vector in the electronic fence area; An abnormal warning module, which is used to input the pitch feature vector and the behavior trajectory feature vector into a prediction model based on deep learning for behavior prediction, and then send a warning message to the warehouse monitoring center according to the behavior prediction result.

2. The warehouse monitoring system with an intelligent early warning function based on a video surveillance electronic fence according to claim 1, wherein The video monitoring data in the electronic fence area of the warehouse is collected in real time through a high-definition camera.

3. A warehouse monitoring system with an intelligent early warning function based on a video surveillance electronic fence as claimed in claim 1, characterized in that Preprocessing the collected video monitoring data is to remove noise and correct the color of the collected video monitoring data.

4. A warehouse monitoring system with an intelligent early warning function based on a video surveillance electronic fence as described in claim 1, characterized in that, Determining the color distribution difference degree between the selected image frame and the adjacent image frames specifically includes: Extracting the previous frame image and the next frame image of the selected image frame as adjacent image frames from the preprocessed video monitoring data; Extracting the color distribution features of the selected image frame, the previous frame image and the next frame image respectively; Determining the previous frame difference value through the color distribution features of the selected image frame and the previous frame image; Determining the next frame difference value through the color distribution features of the selected image frame and the next frame image; Determining the color distribution difference degree between the selected image frame and the adjacent image frames according to the previous frame difference value and the next frame difference value.

5. A warehouse monitoring system with an intelligent early warning function based on a video surveillance electronic fence, as described in claim 1, wherein Filtering the background of the selected image frame and the adjacent image frames according to the color distribution difference degree, and then obtaining the target recognition image in the electronic fence area specifically includes: When the color distribution difference degree is greater than the preset difference threshold, differentiating the selected image frame and the adjacent image frames to obtain a first difference image and a second difference image; Dividing the background of the first difference image and the second difference image respectively, and then fusing the first difference image after background division and the second difference image after background division to obtain the target recognition image in the electronic fence area.

6. The warehouse monitoring system with an intelligent early warning function based on a video surveillance electronic fence as claimed in claim 1, wherein When the moving target detection factor is greater than the preset detection threshold, the audio data in the electronic fence area is obtained through a microphone array.

7. The warehouse monitoring system with an intelligent early warning function based on a video surveillance electronic fence according to claim 1, wherein, When the moving target detection factor is greater than the preset detection threshold, the target behavior trajectory data in the electronic fence area is obtained through a target detection algorithm.

8. The warehouse monitoring system with an intelligent early warning function based on a video surveillance electronic fence as described in claim 1, wherein, Extracting features from the audio data and the target behavior trajectory data respectively, and obtaining the pitch feature vector and behavior trajectory feature vector in the electronic fence area specifically includes: Performing threshold filtering on the audio data to obtain audio filtered data; Construct features for the audio filtering data to obtain a pitch feature vector within the electronic fence area; Extract features from the target behavior trajectory data to obtain a behavior trajectory feature vector within the electronic fence area.

9. The warehouse monitoring system with an intelligent early warning function based on a video surveillance electronic fence according to claim 8, characterized in that, Constructing features for the audio filtering data to obtain a pitch feature vector within the electronic fence area specifically includes: Determine the scale feature of the audio filtering data; Extract the fundamental frequency value of each frame of audio in the audio filtering data, and then obtain all the fundamental frequency values. Determine the pitch mean of the audio filtering data based on all the fundamental frequency values; Determine the unvoiced fluctuation feature of the audio filtering data; Construct a pitch feature vector within the electronic fence area through the scale feature, the pitch mean, and the unvoiced fluctuation feature.

10. A warehouse monitoring system with an intelligent early warning function based on a video surveillance electronic fence as described in claim 1, characterized in that, Sending a warning message to the warehouse monitoring center according to the behavior prediction result specifically includes: When the behavior prediction result is normal, no warehouse warning is triggered, and the behavior trajectory data is stored; When the behavior prediction result is abnormal, trigger a warehouse warning, and then generate a warning message according to the abnormal level in the behavior prediction result, and send the warning message to the warehouse monitoring center.

Images