Abnormal behavior monitoring and warning method based on deep learning

Abstract

The invention provides an abnormal behavior monitoring and warning method based on deep learning, and relates to the technical field of deep learning, and the method comprises the steps: monitoring the operation process of a current fuel business in real time, and obtaining a first monitoring video; performing key frame extraction and image preprocessing on the first monitoring video to obtain a first image set; and inputting the first image set into a corresponding feature analysis model to generate a feature analysis result, identifying an abnormal behavior according to the feature analysis result, and giving an abnormal alarm when the abnormal behavior is identified. The method comprises the following steps: performing key frame extraction and image preprocessing on a first monitoring video obtained by monitoring an operation process of a current fuel service in real time to obtain a first image set; the first image set is input into the corresponding feature analysis model to generate the feature analysis result, then the abnormal behavior is recognized according to the feature analysis result, and when the abnormal behavior is recognized, abnormal warning is performed, so that the monitoring precision of the abnormal behavior can be effectively improved, timely warning is ensured, and the business efficiency of an enterprise is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning, and particularly relates to a method for monitoring and alarming abnormal behaviors based on deep learning. Background Art

[0002] In recent years, with the development of technology, monitoring systems have been widely applied in multiple fields such as public security, industrial production, and financial transactions. However, traditional monitoring systems mainly rely on manual monitoring or simple rule judgments, and it is difficult to cope with complex and changeable abnormal behaviors, resulting in inaccurate monitoring of abnormal behaviors. Therefore, how to improve the monitoring accuracy of abnormal behaviors, ensure timely alarming, and improve the business efficiency of enterprises has become one of the current research focuses.

[0003] Therefore, the present invention provides a method for monitoring and alarming abnormal behaviors based on deep learning. Summary of the Invention

[0004] The present invention provides a method for monitoring and alarming abnormal behaviors based on deep learning, which is used to extract key frames and perform image preprocessing on a first monitoring video obtained by real-time monitoring of the operation process of the current fuel business to obtain a first image set; input the first image set into a corresponding feature analysis model to generate a feature analysis result, then identify abnormal behaviors according to the feature analysis result, and perform abnormal alarming when abnormal behaviors are identified, which can effectively improve the monitoring accuracy of abnormal behaviors, ensure timely alarming, and improve the business efficiency of enterprises.

[0005] The present invention provides a method for monitoring and alarming abnormal behaviors based on deep learning, including: Step 1: Use a set monitoring tool to real-time monitor the operation process of the current fuel business to obtain a first monitoring video; Step 2: Extract key frames and perform image preprocessing on the first monitoring video to obtain a first image set; Step 3: Input the first image set into a corresponding feature analysis model to generate a feature analysis result, then identify abnormal behaviors according to the feature analysis result, and perform abnormal alarming when abnormal behaviors are identified.

[0006] Preferably, the fuel business includes metering business, sampling business, sample preparation business, and laboratory testing business.

[0007] Preferably, extracting key frames and performing image preprocessing on the first monitoring video to obtain a first image set includes: Obtain a set dynamic ratio at the current moment, divide the video frames in the first monitoring video to obtain a number of first frame sets; Evaluate the image quality of the video frames in each first frame set using a set image quality evaluation index to obtain a first quality coefficient; Output the video frame corresponding to the largest first quality coefficient in each set of first frames as the candidate frame; After performing image enhancement and denoising processing on all the obtained candidate frames, summarize and generate an image set, and output it as the first image set.

[0008] Preferably, obtaining the set dynamic ratio at the current moment includes: Calculate the average gray difference value between adjacent video frames in the current first monitoring video; Accumulate the average gray difference values of each pair of adjacent frames obtained to obtain the cumulative gray difference value; Determine the first observation pixel block according to the number of the current set monitoring tool; Use the set running estimation algorithm to sequentially estimate the first motion vectors between adjacent video frames of the first observation pixel block in the current first monitoring video; Calculate the first vector modulus length of the first motion vector, and accumulate the first vector modulus lengths of each pair of adjacent frames obtained to obtain the cumulative vector modulus length; By combining and analyzing the cumulative gray difference value and the cumulative vector modulus length, adjust the set dynamic ratio at the previous moment to obtain the set dynamic ratio at the current moment.

[0009] Preferably, the calculation formula of the set dynamic ratio is as follows: ; where, represents the set dynamic ratio at the current moment; represents the set dynamic ratio at the previous moment; represents the usage duration of the current set monitoring tool; represents the expected service life of the current set monitoring tool; represents the influence weight of the usage situation of the monitoring tool on calculating the set dynamic ratio; represents the cumulative gray difference value obtained at the current moment; represents the cumulative gray difference value obtained at the previous moment; represents the contribution weight of the gray difference between adjacent frames to analyzing the dynamic degree of video content; represents the cumulative vector modulus length of the i-th first observation pixel block at the current moment; represents the ratio of the area of the i-th first observation pixel block to the video frame in the first monitoring video at the current moment, where i = 1, 2, 3, , n; n represents the total number of first observation pixel blocks; represents the total area of the video frames in the current first monitoring video; represents the cumulative vector modulus length of the i-th first observation pixel block at the previous moment; It represents the contribution weight of the adjacent frame motion vectors to the dynamic degree analysis of the video content; It represents the influence weight of the video content dynamic degree on the calculation of the set dynamic ratio.

[0010] Preferably, input the first image set into the corresponding feature analysis model to generate a feature analysis result, then identify abnormal behaviors according to the feature analysis result, and when an abnormal behavior is identified, perform abnormal alarm, including: Input the first image set into the feature analysis model established based on deep learning pre-loaded in the current set monitoring tool, and output the feature analysis result; If there is personnel abnormal behavior data in the current feature analysis result, send a personnel alarm signal to the fuel supervision platform, and combine the number of the current set monitoring tool and the personnel abnormal behavior data as the personnel abnormal alarm information to be transmitted to the fuel supervision platform; After receiving the personnel alarm signal, the fuel supervision platform quickly analyzes and correspondingly processes the received personnel alarm information.

[0011] Preferably, after receiving the alarm signal, the fuel supervision platform quickly analyzes and correspondingly processes the received alarm information, including: When the fuel supervision platform receives the personnel abnormal alarm information, extract the number of the set monitoring tool in the personnel abnormal alarm information, and determine the personnel operation work area and the business to which the personnel belong of the current abnormal-operator; Extract the personnel abnormal behavior data in the personnel abnormal alarm information, and input the abnormal behavior map in the personnel abnormal behavior data into the pre-established personnel recognition model to determine the basic information of the abnormal-operator with abnormal behavior currently; Use the abnormal behavior content in the personnel abnormal behavior data as a matching condition to screen out the corresponding correct behavior guidance and real-time deduction items from the preset behavior database; Based on the basic information of the abnormal-operator, the personnel operation work area and the business to which the personnel belong, locate the current abnormal-operator, and transmit the behavior warning signal and the correct behavior guidance to the abnormal-operator; Based on the real-time deduction items and the abnormal behavior content, perform score deduction and abnormal behavior record respectively on the corresponding personal operation supervision page of the abnormal-operator in the fuel supervision platform.

[0012] Preferably, it further includes: Regularly extract the historical feature recognition data of the feature analysis model pre-loaded in the current set monitoring tool in the preset time period from the model analysis database; Analyze the historical feature recognition data. When there is a misrecognition situation, based on the corresponding misbehavior recognition data, determine all the misrecognition behavior descriptions and the corresponding historical misrecognition frequencies that exist. Obtain the corresponding correct recognition behavior descriptions for each misrecognition behavior description. Adjust the benchmark supplementary sample size using the historical misrecognition frequencies to determine the actual supplementary sample size for each misrecognition behavior description. Among them, the calculation formula for the actual supplementary sample size is as follows: ; In the formula, represents the actual supplementary sample size of the a-th misrecognition behavior description at the current r-th moment. represents the benchmark supplementary sample size. represents the historical misrecognition frequency of the a-th misrecognition behavior description at the current r-th moment. represents the sum of the historical misrecognition frequencies of all misrecognition behavior descriptions at the current r-th moment. represents the historical misrecognition frequency of the a-th misrecognition behavior description at the j-th moment, where j = 1, 2, 3. Construct a misrecognition supplementary sample graph of the actual supplementary sample size that includes misrecognition behavior descriptions to obtain the first misrecognition sample set. Construct a correct supplementary sample graph of the actual supplementary sample size that includes the corresponding correct recognition behavior descriptions to obtain the first correct sample set. Based on the first misrecognition sample set and the first correct sample set, retrain and optimize the current feature analysis model according to the set optimization mechanism.

[0013] Compared with the prior art, the beneficial effects of the present application are as follows: By extracting key frames and performing image preprocessing on the first monitoring video obtained by real-time monitoring of the operation process of the current fuel business, a first image set is obtained; the first image set is input into the corresponding feature analysis model to generate a feature analysis result, and then abnormal behaviors are identified according to the feature analysis result, and an abnormal alarm is given when an abnormal behavior is identified, which can effectively improve the monitoring accuracy of abnormal behaviors, ensure timely alarm, and improve the business efficiency of the enterprise.

[0014] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained by the structure specifically pointed out in the written specification and the drawings.

[0015] The technical solutions of the present invention will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0016] The accompanying drawings are used to provide a further understanding of the present invention and form a part of the specification. They are used together with the embodiments of the present invention to explain the present invention, but do not constitute a limitation to the present invention. In the accompanying drawings: Figure 1 It is a flowchart of a method for abnormal behavior monitoring and warning based on deep learning in an embodiment of the present invention. Detailed implementation manners

[0017] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to explain and illustrate the present invention and are not used to limit the present invention.

[0018] An embodiment of the present invention provides a method for abnormal behavior monitoring and warning based on deep learning, as Figure 1 shown, including: Step 1: Use a set monitoring tool to monitor the operation process of the current fuel business in real time to obtain a first monitoring video; Step 2: Extract key frames from the first monitoring video and perform image preprocessing to obtain a first image set; Step 3: Input the first image set into a corresponding feature analysis model to generate a feature analysis result, then identify abnormal behaviors according to the feature analysis result, and issue an abnormal warning when an abnormal behavior is identified.

[0019] In this embodiment, the set monitoring tool refers to a device used to monitor the operation process of the fuel business in real time, such as a high-definition camera; the fuel business includes metering business, sampling business, sample preparation business, and testing business; the first monitoring video refers to a video record of the operation process of the fuel business captured in real time by the set monitoring tool; image preprocessing refers to image enhancement and denoising processing to improve the image quality and reduce noise interference; the first image set refers to an image set obtained through key frame extraction and image preprocessing; the feature analysis model is obtained by collecting a large number of undetermined images containing abnormal behaviors of personnel and standard abnormal behaviors of the undetermined images, then performing preprocessing and feature extraction on the collected undetermined images to obtain behavior features, and using all the obtained behavior features and undetermined images as training data to train a neural network, which is used to extract features from the input images and match abnormal features, and output abnormal behavior data; the feature analysis result refers to the result output by the model after inputting the first image set into the feature analysis model, for example, abnormal behavior data of personnel.

[0020] Beneficial effects of the above technical solution: By performing key frame extraction and image preprocessing on the first monitoring video obtained by real-time monitoring of the current fuel operation process, a first image set is obtained; the first image set is input into the corresponding feature analysis model to generate a feature analysis result, and then abnormal behaviors are identified based on the feature analysis result, and an abnormal alarm is issued when an abnormal behavior is identified, which can effectively improve the monitoring accuracy of abnormal behaviors, ensure timely alarm, and improve the business efficiency of the enterprise.

[0021] An embodiment of the present invention provides a method for monitoring and alarming abnormal behaviors based on deep learning. Key frame extraction and image preprocessing are performed on the first monitoring video to obtain a first image set, including: Obtain a set dynamic ratio at the current moment, divide the video frames in the first monitoring video to obtain a number of first frame sets; Use a set quality evaluation index to evaluate the image quality of the video frames in each first frame set to obtain a first quality coefficient; Output the video frame corresponding to the largest first quality coefficient in each first frame set as a candidate frame; After performing image enhancement and denoising processing on all the obtained candidate frames, summarize and generate an image set, and output it as the first image set.

[0022] In this embodiment, the set dynamic ratio refers to the selection ratio of frames determined according to the dynamic change degree of the video content during the key frame extraction process; the first frame set refers to the set obtained by dividing the video frames in the first monitoring video according to the set dynamic ratio; the set quality evaluation index refers to the index used to evaluate the quality of video frames, including image clarity and image contrast. Among them, the image clarity is obtained by weighted averaging the index values obtained by evaluating the image using the set clarity index, where the set clarity index includes gradient mean and gradient variance; the image contrast is obtained by calculating the standard deviation of the image gray level; the first quality coefficient is obtained by weighted averaging the values obtained by evaluating each video frame according to the set quality evaluation index, and the weights assigned to the set quality evaluation index are obtained by solving the matrix constructed by pairwise comparison and relative importance scoring using the analytic hierarchy process; the candidate frame refers to the video frame corresponding to the largest first quality coefficient in each first frame set; the first image set refers to the image set obtained through key frame extraction and image preprocessing.

[0023] Beneficial effects of the above technical solution: After performing key frame extraction on the obtained first monitoring video based on the reliability analysis of the image and then performing corresponding image preprocessing to obtain the first image set, it can provide a favorable data basis for subsequent abnormal behavior monitoring.

[0024] An embodiment of the present invention provides a method for abnormal behavior monitoring and warning based on deep learning, and obtaining a set dynamic ratio at the current moment, including: Calculate the average grayscale difference value between adjacent video frames in the current first monitoring video; Accumulate the average grayscale difference values of each pair of adjacent frames obtained to obtain an accumulated grayscale difference value; Determine a first observation pixel block according to the number of the currently set monitoring tool; Use a set running estimation algorithm to sequentially estimate the first motion vectors between adjacent video frames of the first observation pixel block in the current first monitoring video; Calculate the first vector modulus length of the first motion vector, and accumulate the first vector modulus lengths of each pair of adjacent frames obtained to obtain an accumulated vector modulus length; By combining and analyzing the accumulated grayscale difference value and the accumulated vector modulus length, adjust the set dynamic ratio at the previous moment to obtain the set dynamic ratio at the current moment.

[0025] In this embodiment, the average grayscale difference value refers to the absolute difference between the average grayscale value of the current frame and the average grayscale value of the previous frame in adjacent frames; the accumulated grayscale difference value is obtained by accumulating the average grayscale difference values of each pair of adjacent frames; the first observation pixel block refers to a part of the pixel area pre-selected in the video frame for analysis, which is pre-specified based on the number of the set monitoring tool, and different monitoring tools may focus on different areas in the video; the set running estimation algorithm is used to estimate the motion vector between adjacent frames, such as the block matching method and the optical flow method; the first motion vector is used to represent the moving direction and distance of the first observation pixel block from the previous frame to the current frame in adjacent video frames; the first vector modulus length refers to the length of the first motion vector; the accumulated vector modulus length is obtained by accumulating the modulus lengths of the corresponding first motion vectors of the current first observation pixel block.

[0026] The beneficial effects of the above technical solution are: By obtaining the set dynamic ratio at the current moment, accurate extraction of key frames can be achieved, which is helpful for providing favorable data basis for subsequent abnormal behavior monitoring and improving the monitoring accuracy.

[0027] An embodiment of the present invention provides a method for abnormal behavior monitoring and warning based on deep learning, and the calculation formula of the set dynamic ratio is as follows: ; In the formula, represents the set dynamic ratio at the current moment; represents the set dynamic ratio at the previous moment; represents the usage duration of the currently set monitoring tool; represents the expected service life of the currently set monitoring tool; Indicates the influence weight of the usage of the monitoring tool on calculating the set dynamic ratio; Indicates the cumulative grayscale difference value obtained at the current moment; Indicates the cumulative grayscale difference value obtained at the previous moment; Indicates the contribution weight of the grayscale difference between adjacent frames to analyzing the dynamic degree of video content; Indicates the cumulative vector norm length of the i-th first observation pixel block at the current moment; Indicates the ratio of the area of the i-th first observation pixel block at the current moment to the area of the video frame in the first monitoring video, where i = 1, 2, 3, , n; n indicates the total number of first observation pixel blocks; Indicates the total area of the video frame in the current first monitoring video; Indicates the cumulative vector norm length of the i-th first observation pixel block at the previous moment; Indicates the contribution weight of the motion vector between adjacent frames to analyzing the dynamic degree of video content; Indicates the influence weight of the dynamic degree of video content on calculating the set dynamic ratio.

[0028] In this embodiment, the weights assigned to the usage of the monitoring tool and the dynamic degree of video content are obtained by solving the matrix constructed after pairwise comparison and relative importance scoring using the analytic hierarchy process; the weights assigned to the grayscale difference between adjacent frames and the motion vector between adjacent frames are obtained by solving the matrix constructed after pairwise comparison and relative importance scoring using the analytic hierarchy process.

[0029] The beneficial effects of the above technical solution are: By calculating the set dynamic ratio at the current moment, accurate extraction of key frames can be achieved, which further helps to provide accurate data basis for subsequent abnormal behavior monitoring and improve the monitoring accuracy.

[0030] The embodiment of the present invention provides an abnormal behavior monitoring and warning method based on deep learning. The first image set is input into the corresponding feature analysis model to generate a feature analysis result, and then abnormal behavior is identified according to the feature analysis result, and an abnormal warning is given when abnormal behavior is identified, including: Input the first image set into the feature analysis model established based on deep learning pre-loaded in the current set monitoring tool, and output the feature analysis result; If there is personnel abnormal behavior data in the current feature analysis result, send a personnel warning signal to the fuel supervision platform, and combine the number of the current set monitoring tool and the personnel abnormal behavior data as the personnel abnormal warning information to be transmitted to the fuel supervision platform; After receiving the personnel alarm signal, the fuel supervision platform quickly analyzes and correspondingly processes the received personnel alarm information.

[0031] In this embodiment, the feature analysis model is obtained by collecting a large number of pending images containing abnormal personnel behaviors, annotating the abnormal behaviors for the pending images, then preprocessing and extracting features from the collected pending images to obtain behavior features, using all the obtained behavior features and the pending images as training data to train a neural network, and is used to extract features and match abnormal features for the input images, and output abnormal behavior data; the feature analysis result refers to the result output by the model after inputting the first image set into the feature analysis model. For example, the abnormal behavior data of personnel.

[0032] In this embodiment, the abnormal behavior data of personnel includes the content of the abnormal behavior and the abnormal behavior image. Among them, the abnormal behavior image refers to the video frame with abnormal personnel behavior; the fuel supervision platform refers to a system for receiving, analyzing, and processing alarm information from set monitoring tools; the personnel alarm signal refers to the alarm signal sent to the fuel supervision platform when the feature analysis model identifies an abnormal behavior; the personnel abnormal alarm information refers to the alarm information including abnormal behavior data, monitoring tool number, etc., and is used to provide detailed alarm content to the fuel supervision platform.

[0033] The beneficial effects of the above technical solution are: by inputting the first image set into the corresponding feature analysis model to generate a feature analysis result, then identifying abnormal behaviors according to the feature analysis result, and performing abnormal alarms when abnormal behaviors are identified, the monitoring accuracy of abnormal behaviors can be effectively improved, ensuring timely alarms to improve the business efficiency of the enterprise.

[0034] An embodiment of the present invention provides an abnormal behavior monitoring and alarming method based on deep learning. After receiving the alarm signal, the fuel supervision platform quickly analyzes and correspondingly processes the received alarm information, including: When the fuel supervision platform receives the abnormal alarm information of personnel, extract the number of the set monitoring tool in the abnormal alarm information of personnel, and determine the personnel operation work area and the business to which the personnel belong of the current abnormal-operator; Extract the abnormal behavior data of personnel in the abnormal alarm information of personnel, and input the abnormal behavior image in the abnormal behavior data of personnel into a pre-established personnel recognition model to determine the basic information of the abnormal-operator with abnormal behavior currently; Use the content of the abnormal behavior in the abnormal behavior data of personnel as a matching condition to screen out the corresponding correct behavior guidance and real-time deduction items from the preset behavior database; Based on the basic information of the abnormal-operator, the operator's working area, and the business to which the operator belongs, locate the current abnormal-operator, and transmit the behavior warning signal and correct behavior guidance to the abnormal-operator; Based on the real-time deduction amount and the content of the abnormal behavior, perform score deduction and abnormal behavior recording respectively on the corresponding personal operation supervision page of the abnormal-operator in the fuel supervision platform.

[0035] In this embodiment, the number of the monitoring tool is a combination of numbers and letters pre-set for uniquely identifying the monitoring tool; the abnormal-operator refers to the operator with abnormal behavior currently; the operator's working area refers to the working area where the current abnormal-operator makes abnormal behavior; the business to which the operator belongs refers to the business type to which the current abnormal-operator's operation belongs; the personnel recognition model is used to perform face recognition and information matching on the current abnormal-operator according to the input abnormal behavior map, and output the basic information of the current abnormal-operator. It is obtained by collecting the image data of the facial features of all operators, annotating the basic information of the personnel (such as name, work number, department, etc.) and the key points of the facial features on the obtained image data, and then using it as training data to train the neural network; the basic information of the personnel includes the name of the personnel, the work position of the personnel, the personnel number, and the contact information of the personnel, etc.

[0036] In this embodiment, for example, there is a personnel abnormal alarm information 1 that includes the set monitoring tool JLD01. The set monitoring tool JLD01 represents the first set monitoring tool in Area D of the metering business. And the working area of the current abnormal-operator is the corresponding Area D of the metering business, and the business to which the operator belongs is the metering business.

[0037] In this embodiment, the content of the abnormal behavior includes the type of abnormal behavior (including two types: illegal operation and unsafe behavior) and the behavior description (for example, the operator puts hands or tools into the observation port / sampling bucket, or the supervisor is not on duty or leaves the post during the sample preparation process); the preset behavior database is a database storing correct behaviors and deduction clauses, and is used to provide correct behavior guidance and deduction basis for abnormal behaviors; the correct behavior guidance refers to the correct behavior suggestions filtered from the preset behavior database corresponding to the abnormal behavior; the real-time deduction amount refers to the specific deduction value determined according to the content of the abnormal behavior and the deduction clauses in the preset behavior database; the personal operation supervision page is a page in the fuel supervision platform, which is used to display and record the personal information, abnormal behavior records, deduction situations, etc. of the abnormal-operator.

[0038] The beneficial effect of the above technical solution is that after receiving the alarm signal through the fuel supervision platform, quickly analyze and correspondingly process the received alarm information, which can ensure timely and accurate alarm to improve the business efficiency of the enterprise.

[0039] An embodiment of the present invention provides a method for abnormal behavior monitoring and warning based on deep learning, further including: Regularly extract the historical feature recognition data of the feature analysis model pre-loaded in the currently set monitoring tool within a preset time period from the model analysis database; Analyze the historical feature recognition data. When there is a misrecognition situation, based on the corresponding misbehavior recognition data, determine all misrecognition behavior descriptions and the corresponding historical misrecognition frequencies; Obtain the corresponding correct recognition behavior description for each misrecognition behavior description; Adjust the benchmark supplementary sample size using the historical misrecognition frequency to determine the actual supplementary sample size for each misrecognition behavior description; Among them, the calculation formula for the actual supplementary sample size is as follows: ; In the formula, represents the actual supplementary sample size of the a-th misrecognition behavior description at the current r moment; represents the benchmark supplementary sample size; represents the historical misrecognition frequency of the a-th misrecognition behavior description at the current r moment; represents the sum of the historical misrecognition frequencies of all misrecognition behavior descriptions at the current r moment; represents the historical misrecognition frequency of the a-th misrecognition behavior description at the j-th moment, where j = 1, 2, 3; Construct an error supplementary sample graph of the actual supplementary sample size containing misrecognition behavior descriptions to obtain the first error sample set; Construct a correct supplementary sample graph of the actual supplementary sample size containing the corresponding correct recognition behavior descriptions to obtain the first correct sample set; Based on the first error sample set and the first correct sample set, retrain and optimize the current feature analysis model according to the set optimization mechanism.

[0040] In this embodiment, the model analysis database refers to a database that stores a large amount of historical feature recognition data, which usually includes the results of the feature analysis model's recognition and analysis of the monitored video over a certain period of time (i.e., the preset time period); the preset time period is determined in advance; historical feature recognition data refers to the data generated after the feature analysis model recognizes and analyzes the monitored video within the preset time period, including video frames, recognition results (including correct recognition and incorrect recognition), timestamps, and other information; the correct recognition behavior description refers to the correct behavior description corresponding to the incorrect recognition behavior description; the benchmark supplementary sample size refers to the minimum sample size benchmark value preset for constructing new training data; the actual supplementary sample size refers to the actual sample size of the new training data that needs to be constructed after adjusting the benchmark supplementary sample size.

[0041] In this embodiment, the setting optimization mechanism refers to the specific steps preset for retraining and optimizing the feature analysis model, including: 1. Divide the first incorrect sample set and the first correct sample set respectively according to the set division ratio to obtain training-incorrect samples, training-correct samples, test-incorrect samples, and test-correct samples; 2. Merge the training-incorrect samples and training-correct samples and then retrain the current feature analysis model, and then use the test-incorrect samples and test-correct samples to test the feature analysis model to obtain new test results; 3. Use the set evaluation metrics (including accuracy, recall rate, and F1 score) to perform model performance analysis based on the new test results to obtain model performance metric values, and perform weighted averaging on all the obtained model performance metric values to obtain a model performance evaluation coefficient. Among them, the weights assigned to the model performance metric values are obtained by solving the matrix constructed after pairwise comparison and relative importance scoring of the set evaluation metrics using the analytic hierarchy process; When the model performance evaluation coefficient does not meet the set performance evaluation threshold, use the preset adjustment strategy to iteratively adjust the model parameters until the model performance evaluation coefficient meets the set performance evaluation threshold. Among them, the preset adjustment strategy consists of adjustment parameters (such as learning rate, batch size, number of network layers, number of neurons, etc.), adjustment direction (increase or decrease), and adjustment amplitude.

[0042] The beneficial effects of the above technical solution are: By regularly extracting and analyzing historical feature recognition data, determining incorrect recognition behaviors and their frequencies to adjust the benchmark supplementary sample size, constructing a new training data set, and retraining and optimizing the feature analysis model. It can effectively improve the accuracy and performance of the model.

[0043] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims

1. A method for abnormal behavior monitoring and alarm based on deep learning, characterized in that: include: Step 1: Use the set monitoring tool to monitor the operation process of the current fuel business in real time to obtain the first monitoring video; Step 2: extract key frames and perform image preprocessing on the first monitoring video to obtain a first image set; Step 3: Input the first image set into a corresponding feature analysis model to generate a feature analysis result, then identify abnormal behavior based on the feature analysis result, and issue an abnormal alarm when abnormal behavior is identified.

2. According to the method of abnormal behavior monitoring and alarm based on deep learning in claim 1, it is characterized in that: Fuel business includes measurement business, sampling business, sample preparation business and testing business.

3. According to the method of abnormal behavior monitoring and alarm based on deep learning in claim 1, it is characterized in that: Performing key frame extraction and image preprocessing on the first monitoring video to obtain a first image set includes: Obtaining a set dynamic ratio at the current moment, dividing the video frames in the first monitoring video to obtain a plurality of first frame sets; Performing image quality assessment on each video frame in the first frame set using a set quality assessment index to obtain a first quality coefficient; Outputting the corresponding video frame with the largest first quality coefficient in each first frame set as a candidate frame; After image enhancement and denoising are performed on all the acquired candidate frames, an image set is generated and output as the first image set.

4. According to claim 3, a method for abnormal behavior monitoring and alarming based on deep learning is characterized in that: Get the current dynamic ratio, including: Calculate the average grayscale difference value between adjacent video frames in the current first monitoring video; The average grayscale difference value of each pair of adjacent frames is accumulated to obtain a cumulative grayscale difference value; Determine a first observed pixel block according to the number of the currently set monitoring tool; Using a set running estimation algorithm, sequentially estimating first motion vectors of the first observed pixel block between adjacent video frames in the current first monitoring video; Calculating a first vector modulus of the first motion vector, and accumulating the first vector modulus of each pair of adjacent frames to obtain a cumulative vector modulus; By combining and analyzing the cumulative grayscale difference value and the cumulative vector modulus length, the set dynamic ratio at the previous moment is adjusted to obtain the set dynamic ratio at the current moment.

5. According to claim 4, a method for abnormal behavior monitoring and alarming based on deep learning is characterized in that: The calculation formula for setting the dynamic ratio is as follows: ; In the formula, It represents the set dynamic ratio at the current moment; It is expressed as the set dynamic ratio of the previous moment; Indicates the usage time of the currently set monitoring tool; Represents the estimated useful life of the monitoring tool for the current setting; It represents the influence weight of the usage of monitoring tools on the calculation of dynamic ratio; Represents the accumulated grayscale difference value obtained at the current moment; It represents the cumulative grayscale difference value obtained at the previous moment; It is expressed as the contribution weight of the grayscale difference of adjacent frames to the dynamic degree of the analyzed video content; It is represented as the cumulative vector modulus length of the first observed pixel block i at the current moment; It is represented as the area ratio of the i-th first observed pixel block at the current moment to the video frame in the first monitoring video, where i=1, 2, 3, , n; n represents the total number of the first observed pixel blocks; It is represented as the total area of ​​the video frame in the current first monitoring video; It is represented as the cumulative vector modulus length of the first observed pixel block of the i-th pixel at the previous moment; It is expressed as the contribution weight of the motion vectors of adjacent frames to the dynamic degree of the analyzed video content; Represents the weight of the influence of the dynamic degree of the video content on the calculation of the set dynamic ratio.

6. The abnormal behavior monitoring and alarm method based on deep learning according to claim 1 is characterized in that: Inputting the first image set into a corresponding feature analysis model to generate a feature analysis result, identifying abnormal behavior according to the feature analysis result, and issuing an abnormal alarm when abnormal behavior is identified, including: Inputting the first image set into a feature analysis model based on deep learning pre-loaded in the currently set monitoring tool, and outputting feature analysis results; If there is abnormal personnel behavior data in the current feature analysis results, a personnel alarm signal is sent to the fuel supervision platform, and the number of the currently set monitoring tool and the abnormal personnel behavior data are combined and transmitted to the fuel supervision platform as abnormal personnel alarm information; After receiving the personnel alarm signal, the fuel monitoring platform quickly analyzes and processes the received personnel alarm information accordingly.

7. The abnormal behavior monitoring and alarm method based on deep learning according to claim 6 is characterized in that: After receiving the alarm signal, the fuel monitoring platform quickly analyzes and processes the received alarm information, including: When the fuel supervision platform receives the personnel abnormality alarm information, it extracts the number of the set monitoring tool in the personnel abnormality alarm information, and determines the personnel work area and the business to which the personnel belongs of the current abnormal operator; Extracting personnel abnormal behavior data from the personnel abnormal alarm information, and inputting the abnormal behavior graph in the personnel abnormal behavior data into a pre-established personnel recognition model to determine the basic personnel information of the abnormal-operating personnel who currently has abnormal behavior; Using the abnormal behavior content in the personnel abnormal behavior data as the matching condition, the corresponding correct behavior guidance and real-time deduction items are screened out from the preset behavior database; Based on the basic information of the abnormal operator, the operator's work area and the business to which the abnormal operator belongs, the current abnormal operator is located, and a behavior warning signal and correct behavior guidance are transmitted to the abnormal operator; Based on the real-time deduction amount and abnormal behavior content, the points are deducted and the abnormal behavior is recorded respectively on the corresponding personal operation supervision page of the abnormal operator in the fuel supervision platform.

8. The abnormal behavior monitoring and alarm method based on deep learning according to claim 6 is characterized in that: Also includes: Periodically extract historical feature recognition data within a preset time period of the feature analysis model pre-loaded in the currently set monitoring tool from the model analysis database; Analyze the historical feature recognition data, and when there is a misrecognition situation, determine all the misrecognition behavior descriptions and the corresponding historical misrecognition frequency based on the corresponding misrecognition behavior recognition data; Obtain the corresponding correct recognition behavior description for each incorrect recognition behavior description; The baseline supplemental sample size was adjusted using historical misidentification frequencies to determine the actual supplemental sample size for each misidentification behavior description; The calculation formula for the actual supplementary sample size is as follows: ; In the formula, represents the actual number of supplementary samples describing the a-th type of misidentification behavior at the current time r; It is indicated as the baseline supplementary sample size; It is represented by the historical misidentification frequency described by the a-th misidentification behavior at the current time r; It is represented as the sum of the historical misidentification frequencies described by all misidentification behaviors at the current time r; Expressed as The historical misidentification frequency described by the a-th misidentification behavior at time j, where j=1,2,3; Construct an error supplement sample graph containing descriptions of error recognition behaviors of an actual supplement sample quantity to obtain a first error sample set; Constructing a correct supplementary sample graph containing corresponding correct recognition behavior descriptions of the actual supplementary sample quantity to obtain a first correct sample set; Based on the first error sample set and the first correct sample set, the current feature analysis model is retrained and optimized according to a set optimization mechanism.

Images