Intelligent inspection method and device combined with panoramic video monitoring

Abstract

The invention discloses an intelligent inspection method and device combined with panoramic video monitoring, and belongs to the field of equipment monitoring, and the method comprises the steps: carrying out the equipment parameter monitoring and panoramic video fusion monitoring, and obtaining a parameter monitoring result and a panoramic video monitoring result; performing anomaly analysis to obtain an abnormal device and an abnormal amplitude; monitoring requirements are obtained, and the parameter monitoring unit and the video monitoring unit are controlled; analyzing and acquiring a plurality of pieces of deflection information, and controlling a plurality of normal video monitors; based on the control results of the multiple pieces of deflection information, multiple distortion degrees are obtained, multiple optimized distortion correction schemes are obtained, and distortion correction and monitoring are carried out on the multiple normally monitored pictures. The technical problem that parameter monitoring and video monitoring in the prior art are lack of collaboration, abnormal recognition and response are insufficient, and the poorer inspection effect is caused is solved, and the technical effects of actively recognizing abnormities through monitoring collaboration, dynamically deploying monitoring resources and quickly responding, and improving the inspection effect are achieved.

Description

Technical Field

[0001] The present invention relates to the field of equipment monitoring, and in particular to an intelligent inspection method and device combined with panoramic video monitoring. Background Art

[0002] With the rapid development of the industrial field, the operation and maintenance of various types of equipment are becoming increasingly difficult. For the numerous devices scattered in complex industrial environments, parameter monitoring and video inspection are mainly used to build the parameter monitoring and image inspection functions of the equipment, which to a certain extent realizes the supervision of the operating status of online equipment. However, existing parameter monitoring and video monitoring are often separate and lack effective integration, making it difficult to achieve real-time cross-correlation analysis. Once a device fails or an abnormal situation occurs, the response to the abnormal situation of the equipment is slow because the two types of monitoring results cannot be collaboratively identified and located. Summary of the invention

[0003] This application provides an intelligent inspection method and device combined with panoramic video surveillance, aiming to solve the technical problems in the prior art of lack of coordination between parameter monitoring and video monitoring, insufficient abnormality recognition and response, and resulting in poor inspection results.

[0004] In view of the above problems, the present application provides a smart inspection method and device combined with panoramic video surveillance.

[0005] The first aspect disclosed in the present application provides a smart inspection method combined with panoramic video monitoring, the method comprising: monitoring the equipment parameters of multiple devices in a target area through multiple parameter monitors in a parameter monitoring unit, and performing panoramic video fusion monitoring on multiple devices in the target area through multiple video monitors in a video monitoring unit to obtain multiple parameter monitoring results and panoramic video monitoring results; performing equipment parameter abnormality analysis based on the multiple parameter monitoring results, and performing video monitoring abnormality analysis on multiple devices based on the panoramic video monitoring results, obtaining abnormal devices when equipment parameter abnormalities or video monitoring abnormalities occur, and obtaining abnormal amplitudes based on the parameter monitoring results or video monitoring results of the abnormal devices; performing multiple parameter monitors and multiple video monitors based on the abnormal amplitudes The monitoring requirements are adjusted to obtain multiple adjustment parameter monitoring requirements and multiple adjustment video monitoring requirements, and the parameter monitoring unit and the video monitoring unit are controlled; according to the abnormal amplitude, multiple deflection information of multiple normal video monitors other than the abnormal video monitor corresponding to the abnormal device for deflection monitoring of the abnormal device is analyzed and obtained, and multiple normal video monitors are controlled, wherein the deflection information includes the deflection amplitude information of the video monitor deflection to make the abnormal device fall into the monitoring frame; based on the control results of the multiple deflection information, multiple distortion degrees of the monitoring image of the abnormal device in the multiple normal monitoring frames are obtained; according to the abnormal amplitude and the multiple distortion degrees, multiple optimized distortion correction schemes for distortion correction of the multiple normal monitoring frames are optimized to obtain, and the multiple normal monitoring frames are subjected to distortion correction for monitoring.

[0006] Another aspect disclosed in the present application provides a smart inspection device combined with panoramic video monitoring, the device comprising: a monitoring result acquisition module, used to monitor the equipment parameters of multiple devices in the target area through multiple parameter monitors in the parameter monitoring unit, and to perform panoramic video fusion monitoring of multiple devices in the target area through multiple video monitors in the video monitoring unit, to obtain multiple parameter monitoring results and panoramic video monitoring results; an abnormal amplitude analysis module, used to perform equipment parameter abnormality analysis based on multiple parameter monitoring results, and to perform video monitoring abnormality analysis of multiple devices based on panoramic video monitoring results, to obtain abnormal devices when equipment parameter abnormalities or video monitoring abnormalities occur, and to obtain abnormal amplitudes based on the parameter monitoring results or video monitoring results of the abnormal devices; a monitoring requirement adjustment module, used to adjust multiple parameter monitors and multiple video monitors based on the abnormal amplitudes; The monitoring requirements are adjusted to obtain multiple adjustment parameter monitoring requirements and multiple adjustment video monitoring requirements, and the parameter monitoring unit and the video monitoring unit are controlled; a normal monitoring control module is used to analyze and obtain multiple deflection information of multiple normal video monitors other than the abnormal video monitor corresponding to the abnormal device for deflection monitoring of the abnormal device according to the abnormal amplitude, and control the multiple normal video monitors, wherein the deflection information includes the deflection amplitude information of the video monitor deflection to make the abnormal device fall into the monitoring frame; a distortion degree acquisition module is used to obtain multiple distortion degrees of the monitoring image of the abnormal device in multiple normal monitoring frames based on the control results of the multiple deflection information; a monitoring distortion correction module is used to optimize and obtain multiple optimized distortion correction schemes for distortion correction of multiple normal monitoring frames according to the abnormal amplitude and multiple distortion degrees, perform distortion correction on multiple normal monitoring frames, and perform monitoring.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: By using parameter monitoring units and video monitoring units to perform parameter monitoring and panoramic video monitoring on multiple devices in the target area, the parameter monitoring results and panoramic video monitoring results of the devices are obtained for subsequent judgment of the device status; abnormal analysis is performed based on the parameter monitoring results and panoramic video monitoring results to obtain abnormal device and abnormal amplitude information so as to adopt targeted monitoring optimization strategies; the monitoring requirements of the parameter monitoring unit and the video monitoring unit are adjusted according to the abnormal amplitude, that is, the monitoring frequency and quality of abnormal devices are improved, the monitoring occupancy of normal devices is reduced, and the optimal scheduling of monitoring resources is realized; multiple normal devices are calculated based on the abnormal amplitude. The video monitor needs to be deflected at an angle to monitor abnormal equipment, ensuring clear capture of abnormal equipment while taking into account the monitoring needs of normal equipment; obtaining the degree of distortion of abnormal equipment in the monitoring frame to provide a basis for subsequent distortion correction; combining the abnormal amplitude and distortion degree to select the optimal distortion correction scheme, correct the video monitoring frame, and improve the monitoring effect. This technical solution solves the technical problems in the existing technology of lack of coordination between parameter monitoring and video monitoring, insufficient recognition and response to anomalies, resulting in poor inspection results, and achieves the technical effect of actively identifying anomalies through monitoring coordination and dynamically allocating monitoring resources for rapid response, thereby improving the inspection effect.

[0008] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A flowchart of a smart inspection method combined with panoramic video monitoring is provided for an embodiment of the present application; Figure 2 A flowchart of obtaining multiple distortion degrees in a smart inspection method combined with panoramic video monitoring is provided for an embodiment of the present application; Figure 3 A structural schematic diagram of a smart inspection device combined with panoramic video monitoring is provided for an embodiment of the present application.

[0010] Explanation of the reference numerals: monitoring result acquisition module 11 , abnormal amplitude analysis module 12 , monitoring requirement adjustment module 13 , normal monitoring control module 14 , distortion degree acquisition module 15 , monitoring distortion correction module 16 . DETAILED DESCRIPTION

[0011] The overall idea of ​​the technical solution provided by this application is as follows: The embodiment of the present application provides a smart inspection method and device combined with panoramic video monitoring. First, through the collaboration of the parameter monitoring unit and the video monitoring unit, a comprehensive status monitoring of multiple devices in the target area is performed, and it is determined whether the equipment has abnormal conditions based on the monitoring results. Secondly, after determining the abnormal equipment and abnormal amplitude, the monitoring proportion of the abnormal equipment is dynamically increased, and the monitoring requirements of the normal equipment are appropriately reduced, so as to optimize the monitoring quality and response speed of the current abnormal equipment. At the same time, with the help of the controllability of the pan-tilt head, multiple video monitors are used to centrally monitor the abnormal equipment, and the degree of distortion in each monitoring frame is calculated to provide a basis for selecting the optimal distortion correction scheme. Subsequently, the frame distortion generated during the video monitoring process is eliminated to ensure the continued monitoring effect of the normal equipment after transferring part of the monitoring resources to the abnormal equipment, thereby ensuring the overall monitoring quality. The present application realizes the rapid positioning and clear monitoring of abnormal equipment while ensuring the continuous monitoring capability of normal equipment and improving the inspection effect.

[0012] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically described below in conjunction with the drawings in the specification.

[0013] Embodiment 1

[0014] like Figure 1 As shown, an embodiment of the present application provides a smart inspection method combined with panoramic video monitoring, the method is applied to a smart inspection device for panoramic video monitoring, the device includes a parameter monitoring unit and a video monitoring unit, the smart inspection method includes: The device parameters of multiple devices in the target area are monitored by multiple parameter monitors in the parameter monitoring unit, and the multiple devices in the target area are monitored by panoramic video fusion through multiple video monitors in the video monitoring unit to obtain multiple parameter monitoring results and panoramic video monitoring results; In an embodiment of the present application, multiple devices in a target area are monitored to obtain the operating status of the devices, wherein the monitoring includes parameter monitoring and video monitoring.

[0015] Parameter monitoring refers to using multiple parameter monitors in the parameter monitoring unit to monitor the equipment parameters of multiple devices in the target area and obtain multiple parameter monitoring results. The multiple parameter monitors in the parameter monitoring unit include but are not limited to temperature monitors, current monitors, voltage monitors and other parameter monitors, which are used to detect the temperature, current, voltage and other parameters of the equipment and obtain the operating status of the equipment.

[0016] Video monitoring refers to the use of multiple video monitors in the video monitoring unit to acquire and analyze images of multiple devices in the target area to obtain panoramic video monitoring results. The video monitoring unit uses panoramic video technology to acquire video images from multiple perspectives in the target area through multiple video monitors, and performs image fusion to generate panoramic video images as panoramic video monitoring results, which intuitively reflect the overall operating status of the equipment.

[0017] Through the combination of parameter monitoring and panoramic video monitoring, comprehensive monitoring of multiple devices in the target area is carried out from both quantitative and qualitative aspects, and multiple parameter monitoring results and panoramic video monitoring results are obtained, laying the foundation for subsequent equipment abnormality analysis.

[0018] According to the multiple parameter monitoring results, perform device parameter abnormality analysis, and according to the panoramic video monitoring results, perform video monitoring abnormality analysis of multiple devices. When device parameter abnormality or video monitoring abnormality occurs, obtain the abnormal device, and obtain the abnormal amplitude according to the parameter monitoring result or video monitoring result analysis of the abnormal device; Furthermore, the embodiment of the present application also includes: Based on the multiple devices, construct a monitoring anomaly analyzer; Based on the monitoring anomaly analyzer, the device parameter anomaly analysis is performed on the multiple parameter monitoring results, and the panoramic video monitoring results are segmented and the video monitoring anomaly analysis is performed to obtain the abnormal device and abnormal amplitude.

[0019] Furthermore, the embodiment of the present application also includes: Based on the monitoring data records of multiple devices, multiple sample parameter monitoring result sets are obtained, and multiple sample parameter abnormal amplitude value sets are obtained by analysis; Acquire a plurality of sample video monitoring result sets, and analyze and obtain a plurality of sample video abnormal amplitude value sets; Using the plurality of sample parameter monitoring result sets and the plurality of sample parameter abnormal amplitude value sets, constructing a plurality of parameter abnormality analysis branches; Using the multiple sample video monitoring result sets and the multiple sample video abnormal amplitude value sets, based on a convolutional neural network, construct multiple video abnormality analysis branches; Based on the multiple devices, the multiple parameter anomaly analysis branches and the multiple video anomaly analysis branches are divided and combined to obtain multiple monitoring anomaly analysis channels, and to obtain the monitoring anomaly analyzer.

[0020] In a feasible implementation, first, based on the historical monitoring data records of multiple devices in different working states stored in advance, the monitoring results of the device parameters are extracted to form multiple sample parameter monitoring result sets, which reflect the parameter range of the device. Then, the multiple sample parameter monitoring result sets are analyzed to determine whether each parameter monitoring result is within the normal range. If some parameter monitoring results exceed the normal range, it is determined that the parameter is abnormal, and the abnormal amplitude of the parameter is calculated to obtain multiple sample parameter abnormal amplitude value sets. At the same time, based on the holographic video monitoring results of multiple devices in different working states stored in advance, multiple sample video monitoring result sets are formed to reflect the image features of the device in various working states. The sample video monitoring result sets are analyzed in turn to determine whether each sample video monitoring result is clear and stable. If there are abnormal conditions such as blur, vibration, and occlusion, it is determined that the video is abnormal. At the same time, the degree of the abnormal condition is calculated to obtain the amplitude of the corresponding video abnormality, forming multiple sample video abnormal amplitude value sets.

[0021] Then, after obtaining multiple sample parameter monitoring result sets to obtain multiple sample parameter abnormal amplitude value sets, sample sets are extracted for different types of parameters, such as temperature sample sets for temperature parameters and voltage sample sets for voltage parameters, and each sample set contains multiple groups of "parameter value-abnormal amplitude". Subsequently, the sample sets of each parameter are used to train the abnormality detection model, such as training the temperature abnormality model based on the temperature sample set and training the voltage abnormality model based on the voltage sample set. After training each model, a parameter abnormality analysis branch is constructed, thereby obtaining multiple parameter abnormality analysis branches. At the same time, after obtaining multiple sample video monitoring result sets and multiple sample video abnormal amplitude value sets, the multiple sample video monitoring result sets and multiple sample video abnormal amplitude value sets are divided into multiple sample subsets according to the location and abnormality type of the equipment, such as a motor subset, a conveyor belt subset, etc., which contain multiple groups of corresponding video monitoring results and video abnormal amplitude values. Subsequently, for each sample subset, a convolutional neural network is used to train the video anomaly detection model, tune the network structure and hyperparameters, learn the high-level features of the video image, and model the abnormal patterns of different parts. After each convolutional network-based video anomaly model training is completed, a video anomaly analysis branch is constructed, and an abnormal judgment can be made on the new monitoring image of the corresponding part, thereby obtaining multiple video anomaly analysis branches. Afterwards, for each of the multiple devices, the parameters that need to be monitored by the device are set, and the corresponding parameter anomaly analysis branches are extracted according to the parameters that need to be monitored; at the same time, the components that need to be monitored by video are set, and the corresponding video anomaly analysis is extracted according to the components that need to be monitored by video, and the extracted parameter anomaly analysis branches and video anomaly analysis are constructed as the monitoring anomaly analysis channel of the device. According to the above process, multiple monitoring anomaly analysis channels corresponding to multiple devices are obtained, and the multiple monitoring anomaly analysis channels are integrated to obtain a monitoring anomaly analyzer that monitors multiple devices simultaneously.

[0022] Afterwards, the obtained multiple parameter monitoring results and panoramic video monitoring results are extracted and input into the monitoring anomaly analyzer. The monitoring anomaly analyzer inputs the corresponding parameter monitoring results and video monitoring results into the corresponding monitoring anomaly analysis channel according to each device. The monitoring anomaly analysis channel analyzes the monitoring results according to the set parameter anomaly analysis branch and video anomaly analysis branch to determine whether the parameter monitoring and video monitoring of each device are abnormal. If an abnormal device appears, the parameter anomaly analysis branch and the video anomaly analysis branch respectively calculate the abnormal amplitude of the abnormal device in the parameter monitoring results and video monitoring results. If the parameter monitoring and video monitoring of a device detect an abnormality at the same time, its abnormal amplitude is the average of the parameter abnormal amplitude and the video abnormal amplitude.

[0023] According to the abnormal amplitude, the monitoring requirements of the multiple parameter monitors and the multiple video monitors are adjusted to obtain multiple adjusted parameter monitoring requirements and multiple adjusted video monitoring requirements, and the parameter monitoring unit and the video monitoring unit are controlled; Furthermore, the embodiment of the present application also includes: Obtain a set of sample abnormal amplitudes, and obtain a plurality of sample parameter monitoring frequency sets, a plurality of sample video monitoring resolution sets, and a plurality of sample video monitoring frame rate sets; The sample abnormal amplitude set, the plurality of sample parameter monitoring frequency sets, the plurality of sample video monitoring resolution sets and the plurality of sample video monitoring frame rate sets are used to construct a monitoring adjustment analyzer; According to the monitoring adjustment analyzer, monitoring adjustment analysis is performed based on the abnormal amplitude to obtain the multiple adjustment parameter monitoring requirements and multiple adjustment video monitoring requirements, wherein the parameter monitoring frequency within the adjustment parameter monitoring requirements corresponding to the abnormal device is greater than the parameter monitoring frequency within the adjustment parameter monitoring requirements corresponding to other devices.

[0024] In a feasible implementation, first, based on the historical logs, the abnormal amplitudes of the equipment under various abnormal conditions are extracted to form a set of sample abnormal amplitudes. At the same time, a sample set of parameter monitoring frequency, video monitoring resolution and frame rate corresponding to each abnormal state is obtained to obtain multiple sample parameter monitoring frequency sets, multiple sample video monitoring resolution sets and multiple sample video monitoring frame rate sets. Among them, the parameter monitoring frequency represents the frequency of device parameter detection; the video monitoring resolution and frame rate determine the clarity and smoothness of the video monitor operation. Then, using the obtained sample abnormal amplitude set, multiple sample parameter monitoring frequency sets, multiple sample video monitoring resolution sets and multiple sample video monitoring frame rate sets, a machine learning method is used to train the complex mapping relationship between abnormal level and parameter monitoring frequency, video monitoring resolution and frame rate, and construct a monitoring adjustment analyzer. During the training process, the larger the abnormal amplitude, the higher the parameter monitoring frequency, and the higher the video resolution and frame rate. The constructed monitoring adjustment analyzer can output the dynamically adjusted new monitoring strategy requirements according to the input real-time abnormal amplitude, including parameter detection frequency, video acquisition resolution, and video frame rate. In the event of equipment abnormality, it can control and optimize the monitoring plan and improve the monitoring accuracy and quality of abnormal equipment.

[0025] Subsequently, the abnormal amplitude of the acquired abnormal device is sequentially input into the constructed monitoring adjustment analyzer, and the monitoring adjustment analyzer outputs the adjusted parameter monitoring frequency, video monitoring resolution and frame rate and other requirements corresponding to the abnormal device, and obtains multiple adjustment parameter monitoring requirements and multiple adjustment video monitoring requirements. Among them, for the target device with abnormality, its parameter monitoring frequency is increased, and the video acquisition resolution and frame rate are increased to improve its monitoring performance, while some monitoring requirements of normal devices will be lowered to reduce the computing resources occupied, that is, the parameter monitoring frequency in the adjustment parameter monitoring requirements corresponding to the abnormal device is greater than the parameter monitoring frequency in the adjustment parameter monitoring requirements corresponding to other devices. Then, the parameter monitoring unit and the video monitoring unit are set according to the acquired multiple adjustment parameter monitoring requirements and multiple adjustment video monitoring requirements, that is, according to the output adjustment parameter monitoring frequency, the detection frequency of multiple parameter monitors in the parameter monitoring unit is controlled, the detection interval is increased or decreased, and according to the output adjustment video resolution and frame rate requirements, the acquisition and encoding parameters of multiple video monitors in the video monitoring unit are controlled to improve or reduce the acquisition quality. Finally, the control of the parameter monitoring unit and the video monitoring unit is completed, and the monitoring strategy is adjusted to achieve dynamic optimization of monitoring performance and improve the response speed and processing capability to abnormal conditions. In subsequent operations, the above adjustments and controls can be iterated according to the abnormal amplitude updated according to the equipment conditions, so that the monitoring equipment can dynamically match the changes in operation requirements.

[0026] According to the abnormal amplitude, analyzing and obtaining multiple deflection information of multiple normal video monitors other than the abnormal video monitor corresponding to the abnormal device to deflect and monitor the abnormal device, and controlling the multiple normal video monitors, wherein the deflection information includes the deflection amplitude information of the video monitor deflection to make the abnormal device fall into the monitoring frame; Furthermore, the embodiment of the present application also includes: Obtain a set of sample abnormal amplitude values, and obtain a set of sample deflection information of a position in a monitoring frame where the abnormal device can fall when multiple normal video monitors are deflected, wherein the size of the sample abnormal amplitude value is positively correlated with the degree to which the abnormal device is close to the center of the monitoring frame after deflection according to the sample deflection information; Using the sample abnormal amplitude set and multiple sample deflection information sets, a video monitoring deflection analyzer is constructed, wherein the video monitoring deflection analyzer includes multiple video monitoring deflection analysis branches corresponding to multiple normal video monitors; Based on the video monitoring deflection analyzer, multiple deflection information for deflection control of the multiple normal video monitors is analyzed and acquired according to the abnormal amplitude.

[0027] In a feasible implementation, first, two types of sample sets are obtained, namely, a sample abnormal amplitude set and a plurality of sample deflection information sets. Among them, the sample deflection information set represents the position distribution of abnormal devices in the monitoring image at different rotation angles. And there is a positive correlation between the size of the abnormal amplitude and the proximity of the abnormal device to the center of the monitoring frame after deflection, that is, the larger the abnormal amplitude, the closer the abnormal device is to the center of the picture after deflection; the smaller the abnormal amplitude, the closer it is to the edge of the picture. Then, using the abnormal amplitude set and the corresponding plurality of sample deflection information sets, a machine learning method is used to train the mapping relationship between the abnormal amplitude and the video pan-tilt rotation control, and obtain a plurality of video monitoring deflection analysis branches corresponding to a plurality of normal video monitors, each video monitoring deflection analysis branch is responsible for the deflection of the normal video monitor, and a plurality of video monitoring deflection analysis branches constitute a video monitoring deflection analyzer, so as to output the control information such as the rotation angle and direction of different normal video monitoring devices according to the abnormal amplitude, so as to realize the automatic response to abnormal situations.

[0028] After obtaining the abnormal amplitude, the obtained abnormal amplitude is input into the video monitoring deflection analyzer. The video monitoring deflection analyzer activates the corresponding video monitoring deflection analysis branch according to the abnormal device that generates the abnormal amplitude. The video monitoring deflection analysis branch outputs the deflection information of the video monitor that monitors the abnormal device according to the abnormal amplitude. By synchronously processing the abnormal amplitudes of all abnormal devices using the video monitoring deflection analysis branch, the deflection information of multiple video monitoring devices is obtained, including the corresponding deflection amplitude information that makes the abnormal device fall within the monitoring range of each video monitor screen. Subsequently, according to the obtained multiple deflection information, the pan-tilt system of the video monitor is controlled to rotate to a specified angle respectively, and the normal video monitor is controlled. For example, video monitor 1 needs to move 45 degrees to the right, and video monitor 2 needs to move 30 degrees to the left, so as to locate the abnormal device within the effective range of different monitoring screens.

[0029] Based on the control results of the multiple deflection information, multiple distortion degrees of the monitoring image of the abnormal device in multiple normal monitoring frames are obtained; Further, such as Figure 2 As shown, the embodiment of the present application also includes: Based on the multiple video monitors, multiple sets of sample deflection information are obtained, and the degree of distortion of the abnormal device in the monitoring frames of the multiple video monitors under different sample deflection information is obtained to obtain multiple sets of sample distortion degrees; Using the multiple sample deflection information sets and the multiple sample distortion degree sets, construct multiple distortion mapping branches to obtain a distortion analyzer; The distortion analyzer is used to perform mapping analysis on the multiple deflection information to obtain the multiple distortion degrees.

[0030] In a preferred embodiment, first, based on multiple video monitors in the monitoring area, the corresponding historical deflection information is searched to obtain multiple sample deflection information sets, and the distortion degree of the abnormal device corresponding to each sample deflection information in each video screen is recorded to obtain multiple sample distortion degree sets. Subsequently, for each video monitor, the corresponding sample deflection information set and sample distortion degree set are used to train a mapping relationship model between the deflection parameter and the distortion degree of the target abnormal device in the monitoring screen, and a distortion mapping branch is constructed. Then, the distortion mapping branches of all video monitors are integrated to form a component for image distortion detection and analysis, namely, a distortion analyzer, which realizes the distortion situation of the abnormal device image in the monitoring screen based on the deflection information of the video monitor, and outputs the distortion situation of the abnormal device image in the monitoring screen in real time, providing input conditions for optimization control. Afterwards, the current deflection information of the multiple video monitors obtained is input into the distortion analyzer, and the distortion analyzer activates the corresponding distortion mapping branch, and outputs the distortion degree of the image of the abnormal device in the corresponding monitoring screen after rotation adjustment in real time, and obtains multiple distortion degrees.

[0031] According to the abnormal amplitude and the multiple distortion degrees, multiple optimized distortion correction schemes for performing distortion correction on the multiple normal monitoring frames are optimized and acquired, and the distortion correction is performed on the multiple normal monitoring frames for monitoring.

[0032] Furthermore, the embodiment of the present application also includes: According to the abnormal amplitude, a distortion correction function is constructed as follows: ; Where dis is the correction fitness, M is the number of multiple normal video monitors and multiple optimized distortion correction schemes, is an example of correcting the normal monitoring frame using the i-th optimized distortion correction scheme, F is the abnormal amplitude, is the mean of historical abnormal amplitudes, is the degree of distortion after the i-th normal monitoring frame is corrected using the i-th optimized distortion correction scheme, is the distortion degree of the i-th normal monitoring frame before correction, and is the weight; According to the distortion correction function, the distortion correction schemes for performing distortion correction on the multiple normal monitoring frames are optimized to obtain the multiple optimized distortion correction schemes.

[0033] In a preferred embodiment, first, a distortion correction function is constructed as: ; Where dis is the correction fitness, M is the number of multiple normal video monitors and multiple optimized distortion correction schemes, is an example of correcting the normal monitoring frame using the i-th optimized distortion correction scheme, F is the abnormal amplitude, is the mean of historical abnormal amplitudes, is the degree of distortion after the i-th normal monitoring frame is corrected using the i-th optimized distortion correction scheme, is the distortion degree of the i-th normal monitoring frame before correction, and is the weight. The distortion correction function is used to evaluate and determine the best distortion correction solution.

[0034] Then, based on the distortion correction function, multiple candidate distortion correction schemes are evaluated and screened to obtain the current optimal correction strategy. Multiple distortion correction schemes corresponding to the abnormal amplitude of the abnormal equipment are substituted into the distortion correction function for calculation. The distortion correction scheme can adopt edge reconstruction, deformation inference, filtering and noise reduction, etc. Subsequently, the distortion correction scheme with the largest correction fitness, that is, the smallest degree of distortion after correction, is selected as the optimized distortion correction scheme. Multiple abnormal amplitudes are processed in turn to obtain multiple optimized distortion correction schemes.

[0035] After obtaining multiple optimized distortion correction schemes, based on the image processing algorithms corresponding to these schemes, multiple normal monitoring frames with obvious distortion detected in the video monitor are corrected, that is, the selected optimal image restoration strategy, including edge reconstruction, shape correction, contrast adjustment and other algorithms, is used to correct the normal monitoring images that are distorted by operations such as pan / tilt rotation. After completing the image distortion correction, the video monitoring equipment can continue to carry out subsequent monitoring and judgment such as abnormal state recognition and event analysis based on the optimized normal monitoring frames to ensure the accuracy of the monitoring and analysis process.

[0036] In summary, the smart inspection method combined with panoramic video surveillance provided by the embodiment of the present application has the following technical effects: Through the multiple parameter monitors in the parameter monitoring unit, the equipment parameters of multiple devices in the target area are monitored, and through the multiple video monitors in the video monitoring unit, panoramic video fusion monitoring is performed on multiple devices in the target area to obtain multiple parameter monitoring results and panoramic video monitoring results, providing a basis for subsequent judgment of whether the equipment is abnormal. According to the multiple parameter monitoring results, the equipment parameter abnormality analysis is performed, and according to the panoramic video monitoring results, the video monitoring abnormality analysis of multiple devices is performed. When the equipment parameter abnormality or video monitoring abnormality occurs, the abnormal device is obtained, and the abnormal amplitude is obtained according to the parameter monitoring results or video monitoring results of the abnormal device, providing a basis for subsequent monitoring strategy adjustment. According to the abnormal amplitude, the monitoring requirements of multiple parameter monitors and multiple video monitors are adjusted, and multiple adjustment parameter monitoring requirements and multiple adjustment video monitoring requirements are obtained. The parameter monitoring unit and the video monitoring unit are controlled to realize the dynamic optimization configuration of monitoring resources. According to the abnormal amplitude, analyze and obtain multiple deflection information of multiple normal video monitors other than the abnormal video monitor corresponding to the abnormal device for deflection monitoring of the abnormal device, and control multiple normal video monitors, wherein the deflection information includes the deflection amplitude information of the video monitor deflection to make the abnormal device fall into the monitoring frame, and concentrate the video monitors to monitor the abnormal device while taking into account the monitoring needs of normal equipment. Based on the control results of multiple deflection information, obtain multiple distortion degrees of the monitoring image of the abnormal device in multiple normal monitoring frames to provide a reference for distortion correction. According to the abnormal amplitude and multiple distortion degrees, optimize and obtain multiple optimized distortion correction schemes for distortion correction of multiple normal monitoring frames, perform distortion correction on multiple normal monitoring frames, and monitor to improve the overall monitoring and inspection quality.

[0037] Embodiment 2

[0038] Based on the same inventive concept as the smart inspection method combined with panoramic video monitoring in the aforementioned embodiment, Figure 3 As shown, an embodiment of the present application provides a smart inspection device combined with panoramic video monitoring, the device includes a parameter monitoring unit and a video monitoring unit, including: The monitoring result acquisition module 11 is used to monitor the device parameters of multiple devices in the target area through multiple parameter monitors in the parameter monitoring unit, and perform panoramic video fusion monitoring on multiple devices in the target area through multiple video monitors in the video monitoring unit to obtain multiple parameter monitoring results and panoramic video monitoring results; The abnormal amplitude analysis module 12 is used to perform device parameter abnormality analysis based on the multiple parameter monitoring results, and perform video monitoring abnormality analysis of multiple devices based on the panoramic video monitoring results. When device parameter abnormality or video monitoring abnormality occurs, the abnormal device is obtained, and the abnormal amplitude is obtained according to the parameter monitoring result or video monitoring result analysis of the abnormal device; A monitoring requirement adjustment module 13, used to adjust the monitoring requirements of the multiple parameter monitors and the multiple video monitors according to the abnormal amplitude, obtain multiple adjusted parameter monitoring requirements and multiple adjusted video monitoring requirements, and control the parameter monitoring unit and the video monitoring unit; The normal monitoring control module 14 is used to analyze and obtain multiple deflection information of multiple normal video monitors other than the abnormal video monitor corresponding to the abnormal device to deflect and monitor the abnormal device according to the abnormal amplitude, and control the multiple normal video monitors, wherein the deflection information includes the deflection amplitude information of the video monitor deflection to make the abnormal device fall into the monitoring frame; A distortion degree acquisition module 15, configured to acquire multiple distortion degrees of the monitoring image of the abnormal device in multiple normal monitoring frames based on the control results of the multiple deflection information; The monitoring distortion correction module 16 is used to optimize and obtain multiple optimized distortion correction schemes for performing distortion correction on the multiple normal monitoring frames according to the abnormal amplitude and the multiple distortion degrees, perform distortion correction on the multiple normal monitoring frames, and perform monitoring.

[0039] Furthermore, the abnormal amplitude analysis module 12 includes the following execution steps: Based on the multiple devices, construct a monitoring anomaly analyzer; Based on the monitoring anomaly analyzer, the device parameter anomaly analysis is performed on the multiple parameter monitoring results, and the panoramic video monitoring results are segmented and the video monitoring anomaly analysis is performed to obtain the abnormal device and abnormal amplitude.

[0040] Furthermore, the abnormal amplitude analysis module 12 further includes the following execution steps: Based on the monitoring data records of multiple devices, multiple sample parameter monitoring result sets are obtained, and multiple sample parameter abnormal amplitude value sets are obtained by analysis; Acquire a plurality of sample video monitoring result sets, and analyze and obtain a plurality of sample video abnormal amplitude value sets; Using the plurality of sample parameter monitoring result sets and the plurality of sample parameter abnormal amplitude value sets, constructing a plurality of parameter abnormality analysis branches; Using the multiple sample video monitoring result sets and the multiple sample video abnormal amplitude value sets, based on a convolutional neural network, construct multiple video abnormality analysis branches; Based on the multiple devices, the multiple parameter anomaly analysis branches and the multiple video anomaly analysis branches are divided and combined to obtain multiple monitoring anomaly analysis channels, and to obtain the monitoring anomaly analyzer.

[0041] Furthermore, the monitoring requirement adjustment module 13 includes the following execution steps: Obtain a set of sample abnormal amplitudes, and obtain a plurality of sample parameter monitoring frequency sets, a plurality of sample video monitoring resolution sets, and a plurality of sample video monitoring frame rate sets; The sample abnormal amplitude set, the plurality of sample parameter monitoring frequency sets, the plurality of sample video monitoring resolution sets and the plurality of sample video monitoring frame rate sets are used to construct a monitoring adjustment analyzer; According to the monitoring adjustment analyzer, monitoring adjustment analysis is performed based on the abnormal amplitude to obtain the multiple adjustment parameter monitoring requirements and multiple adjustment video monitoring requirements, wherein the parameter monitoring frequency within the adjustment parameter monitoring requirements corresponding to the abnormal device is greater than the parameter monitoring frequency within the adjustment parameter monitoring requirements corresponding to other devices.

[0042] Furthermore, the normal monitoring control module 14 includes the following execution steps: Obtain a set of sample abnormal amplitude values, and obtain a set of sample deflection information of a position in a monitoring frame where the abnormal device can fall when multiple normal video monitors are deflected, wherein the size of the sample abnormal amplitude value is positively correlated with the degree to which the abnormal device is close to the center of the monitoring frame after deflection according to the sample deflection information; Using the sample abnormal amplitude set and multiple sample deflection information sets, a video monitoring deflection analyzer is constructed, wherein the video monitoring deflection analyzer includes multiple video monitoring deflection analysis branches corresponding to multiple normal video monitors; Based on the video monitoring deflection analyzer, multiple deflection information for deflection control of the multiple normal video monitors is analyzed and acquired according to the abnormal amplitude.

[0043] Furthermore, the distortion degree acquisition module 15 includes the following execution steps: Based on the multiple video monitors, multiple sets of sample deflection information are obtained, and the degree of distortion of the abnormal device in the monitoring frames of the multiple video monitors under different sample deflection information is obtained to obtain multiple sets of sample distortion degrees; Using the multiple sample deflection information sets and the multiple sample distortion degree sets, construct multiple distortion mapping branches to obtain a distortion analyzer; The distortion analyzer is used to perform mapping analysis on the multiple deflection information to obtain the multiple distortion degrees.

[0044] Furthermore, the monitoring distortion correction module 16 includes the following execution steps: According to the abnormal amplitude, a distortion correction function is constructed as follows: ; Where dis is the correction fitness, M is the number of multiple normal video monitors and multiple optimized distortion correction schemes, is an example of correcting the normal monitoring frame using the i-th optimized distortion correction scheme, F is the abnormal amplitude, is the mean of historical abnormal amplitudes, is the degree of distortion after the i-th normal monitoring frame is corrected using the i-th optimized distortion correction scheme, is the distortion degree of the i-th normal monitoring frame before correction, and is the weight; According to the distortion correction function, the distortion correction schemes for performing distortion correction on the multiple normal monitoring frames are optimized to obtain the multiple optimized distortion correction schemes.

[0045] Any step of the method described above can be stored as a computer instruction or program in an unlimited computer memory, and can be called and recognized by an unlimited computer processor to implement any method in the embodiments of the present application, without any unnecessary restrictions.

[0046] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these modifications and variations.

Claims

1. A smart inspection method combined with panoramic video monitoring, characterized in that: The method is applied to a smart inspection device for panoramic video monitoring, the device comprising a parameter monitoring unit and a video monitoring unit, and the method comprises: The device parameters of multiple devices in the target area are monitored by multiple parameter monitors in the parameter monitoring unit, and the multiple devices in the target area are monitored by panoramic video fusion through multiple video monitors in the video monitoring unit to obtain multiple parameter monitoring results and panoramic video monitoring results; According to the multiple parameter monitoring results, perform device parameter abnormality analysis, and according to the panoramic video monitoring results, perform video monitoring abnormality analysis of multiple devices. When device parameter abnormality or video monitoring abnormality occurs, obtain the abnormal device, and obtain the abnormal amplitude according to the parameter monitoring result or video monitoring result analysis of the abnormal device; According to the abnormal amplitude, the monitoring requirements of the multiple parameter monitors and the multiple video monitors are adjusted to obtain multiple adjusted parameter monitoring requirements and multiple adjusted video monitoring requirements, and the parameter monitoring unit and the video monitoring unit are controlled; According to the abnormal amplitude, analyzing and obtaining multiple deflection information of multiple normal video monitors other than the abnormal video monitor corresponding to the abnormal device to deflect and monitor the abnormal device, and controlling the multiple normal video monitors, wherein the deflection information includes the deflection amplitude information of the video monitor deflection to make the abnormal device fall into the monitoring frame; Based on the control results of the multiple deflection information, multiple distortion degrees of the monitoring image of the abnormal device in multiple normal monitoring frames are obtained; According to the abnormal amplitude and the multiple distortion degrees, multiple optimized distortion correction schemes for performing distortion correction on the multiple normal monitoring frames are optimized and acquired, and the distortion correction is performed on the multiple normal monitoring frames for monitoring.

2. The method according to claim 1, characterized in that According to the multiple parameter monitoring results, device parameter abnormality analysis is performed, and according to the panoramic video monitoring results, video monitoring abnormality analysis of multiple devices is performed, including: Based on the multiple devices, construct a monitoring anomaly analyzer; Based on the monitoring anomaly analyzer, the device parameter anomaly analysis is performed on the multiple parameter monitoring results, and the panoramic video monitoring results are segmented and the video monitoring anomaly analysis is performed to obtain the abnormal device and abnormal amplitude.

3. The method according to claim 2, characterized in that Based on the multiple devices, a monitoring anomaly analyzer is constructed, including: Based on the monitoring data records of multiple devices, multiple sample parameter monitoring result sets are obtained, and multiple sample parameter abnormal amplitude value sets are obtained by analysis; Acquire a plurality of sample video monitoring result sets, and analyze and obtain a plurality of sample video abnormal amplitude value sets; Using the plurality of sample parameter monitoring result sets and the plurality of sample parameter abnormal amplitude value sets, constructing a plurality of parameter abnormality analysis branches; Using the multiple sample video monitoring result sets and the multiple sample video abnormal amplitude value sets, based on a convolutional neural network, construct multiple video abnormality analysis branches; Based on the multiple devices, the multiple parameter anomaly analysis branches and the multiple video anomaly analysis branches are divided and combined to obtain multiple monitoring anomaly analysis channels, and to obtain the monitoring anomaly analyzer.

4. The method according to claim 1, characterized in that: According to the abnormal amplitude, the monitoring requirements of the multiple parameter monitors and the multiple video monitors are adjusted to obtain multiple adjusted parameter monitoring requirements and multiple adjusted video monitoring requirements, including: Obtain a set of sample abnormal amplitudes, and obtain a plurality of sample parameter monitoring frequency sets, a plurality of sample video monitoring resolution sets, and a plurality of sample video monitoring frame rate sets; The sample abnormal amplitude set, the plurality of sample parameter monitoring frequency sets, the plurality of sample video monitoring resolution sets and the plurality of sample video monitoring frame rate sets are used to construct a monitoring adjustment analyzer; According to the monitoring adjustment analyzer, monitoring adjustment analysis is performed based on the abnormal amplitude to obtain the multiple adjustment parameter monitoring requirements and multiple adjustment video monitoring requirements, wherein the parameter monitoring frequency within the adjustment parameter monitoring requirements corresponding to the abnormal device is greater than the parameter monitoring frequency within the adjustment parameter monitoring requirements corresponding to other devices.

5. The method according to claim 1, characterized in that According to the abnormal amplitude, analyzing and obtaining multiple deflection information of the abnormal device by multiple normal video monitors other than the abnormal video monitor corresponding to the abnormal device for deflection monitoring, including: Obtain a set of sample abnormal amplitude values, and obtain a set of sample deflection information of a position in a monitoring frame where the abnormal device can fall when multiple normal video monitors are deflected, wherein the size of the sample abnormal amplitude value is positively correlated with the degree to which the abnormal device is close to the center of the monitoring frame after deflection according to the sample deflection information; Using the sample abnormal amplitude set and multiple sample deflection information sets, a video monitoring deflection analyzer is constructed, wherein the video monitoring deflection analyzer includes multiple video monitoring deflection analysis branches corresponding to multiple normal video monitors; Based on the video monitoring deflection analyzer, multiple deflection information for deflection control of the multiple normal video monitors is analyzed and acquired according to the abnormal amplitude.

6. The method according to claim 1, characterized in that Based on the control results of the multiple deflection information, multiple distortion degrees of the monitoring image of the abnormal device in multiple normal monitoring frames are obtained, including: Based on the multiple video monitors, multiple sets of sample deflection information are obtained, and the degree of distortion of the abnormal device in the monitoring frames of the multiple video monitors under different sample deflection information is obtained to obtain multiple sets of sample distortion degrees; Using the multiple sample deflection information sets and the multiple sample distortion degree sets, construct multiple distortion mapping branches to obtain a distortion analyzer; The distortion analyzer is used to perform mapping analysis on the multiple deflection information to obtain the multiple distortion degrees.

7. The method according to claim 1, characterized in that According to the abnormal amplitude and the multiple distortion degrees, multiple optimized distortion correction schemes for performing distortion correction on the multiple normal monitoring frames are optimized and obtained, including: According to the abnormal amplitude, a distortion correction function is constructed as follows: ； Where dis is the correction fitness, M is the number of multiple normal video monitors and multiple optimized distortion correction schemes, is an example of correcting the normal monitoring frame using the i-th optimized distortion correction scheme, F is the abnormal amplitude, is the mean of historical abnormal amplitudes, is the degree of distortion after the i-th normal monitoring frame is corrected using the i-th optimized distortion correction scheme, is the distortion degree of the i-th normal monitoring frame before correction, and is the weight; According to the distortion correction function, the distortion correction schemes for performing distortion correction on the multiple normal monitoring frames are optimized to obtain the multiple optimized distortion correction schemes.

8. A smart inspection device combined with panoramic video monitoring, characterized in that: A smart inspection method combined with panoramic video monitoring according to any one of claims 1 to 7 is used to implement the method, wherein the device comprises a parameter monitoring unit and a video monitoring unit, and the device comprises: A monitoring result acquisition module, the monitoring result acquisition module is used to monitor the device parameters of multiple devices in the target area through multiple parameter monitors in the parameter monitoring unit, and perform panoramic video fusion monitoring on multiple devices in the target area through multiple video monitors in the video monitoring unit to obtain multiple parameter monitoring results and panoramic video monitoring results; An abnormal amplitude analysis module, the abnormal amplitude analysis module is used to perform device parameter abnormality analysis based on the multiple parameter monitoring results, and perform video monitoring abnormality analysis of multiple devices based on the panoramic video monitoring results. When device parameter abnormality or video monitoring abnormality occurs, the abnormal device is obtained, and the abnormal amplitude is obtained according to the parameter monitoring result or video monitoring result analysis of the abnormal device; A monitoring requirement adjustment module, the monitoring requirement adjustment module is used to adjust the monitoring requirements of the multiple parameter monitors and the multiple video monitors according to the abnormal amplitude, obtain multiple adjusted parameter monitoring requirements and multiple adjusted video monitoring requirements, and control the parameter monitoring unit and the video monitoring unit; A normal monitoring control module, the normal monitoring control module is used to analyze and obtain multiple deflection information of multiple normal video monitors other than the abnormal video monitor corresponding to the abnormal device to deflect and monitor the abnormal device according to the abnormal amplitude, and control the multiple normal video monitors, wherein the deflection information includes the deflection amplitude information of the video monitor deflection to make the abnormal device fall into the monitoring frame; A distortion degree acquisition module, the distortion degree acquisition module is used to acquire multiple distortion degrees of the monitoring image of the abnormal device in multiple normal monitoring frames based on the control results of the multiple deflection information; A monitoring distortion correction module is used to optimize and obtain multiple optimized distortion correction schemes for performing distortion correction on the multiple normal monitoring frames according to the abnormal amplitude and the multiple distortion degrees, perform distortion correction on the multiple normal monitoring frames, and perform monitoring.

Images