Multi-system video monitoring fusion method and system

By performing grayscale processing and feature matching of multiple scene videos, combined with grid division technology, video fusion and optimization of target industrial equipment are achieved, the problem of lack of fusion and analysis capabilities in the existing technology is solved, and video is improved Clarity and monitoring efficiency.

CN119071647AActive Publication Date: 2024-12-03ZHONGZHEXIN TECH CONSULTING CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202411561987.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-05
Publication Date
2024-12-03
Estimated Expiration
2044-11-05

AI Technical Summary

Technical Problem

The prior art lacks the ability to fusion of multiple scene videos of active objects and post-fusion video analysis of active objects in scene videos, and cannot effectively observe the motion state of the active objects and judge the accuracy of the fusion information, and thus cannot optimize the video fusion process.

Method used

By adopting the multi-system video surveillance fusion method, the video captured by each surveillance camera is grayscaled and fused, the working information of the target industrial equipment is extracted, the clarity evaluation coefficient is analyzed, the video is displayed in real time, and early warning prompts are issued when the fault is faulty. Video feature matching and meshing technology obtains the best fusion line and optimizes the video fusion process.

Benefits of technology

The complete work video acquisition and optimization of the target industrial equipment has been achieved, the waste of human inspection resources has been reduced, the clarity and monitoring convenience of videos have been improved, the blur and ghosting problems in video fusion have been reduced, and the video quality has been ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071647A_ABST
    Figure CN119071647A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-system video monitoring fusion method and system, and relates to the technical field of image processing, and the method comprises the steps: processing videos of target industrial equipment at all angles, obtaining all effective working videos, fusing all the effective working videos, and obtaining a complete working video of the target industrial equipment. According to the invention, the image processing technology is applied to the field of industrial manufacturing, the video fusion process is optimized by comparing the working condition of the target industrial equipment extracted from the video with the actual working condition of the target industrial equipment, and the video fusion process is optimized, so that the clearer and more complete working video is obtained. A worker can directly acquire the working condition of the target industrial equipment through the fused complete working video, so that resource waste caused by frequent inspection of the worker is avoided, and economic loss caused by the fact that equipment faults cannot be found in time is also avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a multi-system video surveillance fusion method and system. Background Art

[0002] With the continuous development of computer technology and digital image processing technology, constructing high-resolution wide-angle video through video fusion has become a research hotspot in recent years. This technology can be applied to many fields and has brought great convenience to people's life and work. Therefore, this application proposes a multi-system video surveillance fusion method and system.

[0003] Prior art, such as the invention application patent with announcement number: CN112037264A, discloses a video fusion system and method. The video fusion system includes: a video acquisition module, which is used to acquire scene video information; an active object data statistics system, which is used to acquire active object information in the scene video information based on the scene video information; a data generation module, which is used to generate scene data based on each of the scene video information and / or active object information; a fusion calculation module, which is used to perform fusion calculation on the scene video information and the scene data, thereby generating fusion information, and the fusion information includes at least one scene data and one scene video information. The video fusion system of the present application can save costs, reduce failure points, and significantly improve overall reliability and scalability.

[0004] With respect to the above scheme, there are at least the following technical problems: 1. The current technology mainly fuses the information of active objects in scene videos, lacks the fusion of multiple scene videos of active objects, and lacks the analysis of the fused video of active objects in scene videos, making it impossible to observe the movement state of active objects in a concise and clear manner.

[0005] 2. The current technology lacks the comparison between the fusion information generated after the fusion of active objects and the actual information of the active objects in the scene, and is unable to judge the accuracy of the fusion information, and thus is unable to optimize the video fusion process. Summary of the invention

[0006] The purpose of this application is to provide a multi-system video surveillance fusion method and system to solve the problems existing in the background technology.

[0007] To solve the above technical problems, the present application adopts the following technical solutions: In the first aspect, the present application provides a multi-system video surveillance fusion method, step one, effective video acquisition: grayscale processing is performed on each working video of the target industrial equipment captured by each surveillance camera, and each effective working video is acquired.

[0008] Step 2: Effective video fusion: After fusing the effective working videos, obtain the complete working video of the target industrial equipment, and extract the working information of the target industrial equipment from it, and then analyze and obtain the clarity evaluation coefficient of the complete working video, so as to determine whether the complete working video needs to be optimized.

[0009] Step 3: Early warning: The obtained complete working video is displayed on the monitoring display screen in real time, and an early warning is issued when a failure of the target industrial equipment in the video is detected.

[0010] Preferably, the monitoring cameras include a long-range camera, a high-definition camera and an infrared camera; the long-range camera is used to obtain the appearance video of the target industrial equipment, the high-definition camera is used to obtain the operation video of the parts in the target industrial equipment; and the infrared camera is used to obtain the environment video of the target industrial equipment when it is working.

[0011] Preferably, the complete working picture of the target industrial equipment is obtained after the effective working videos are fused, and the specific fusion process is as follows: A1. A certain designated effective working video is preprocessed as a reference video for video feature extraction, and the next effective working video adjacent to it is preprocessed as a matching video for video feature extraction, the features extracted from the reference video are matched with the features extracted from the matching video, and the feature area after the matching is completed is recorded as the overlapping area.

[0012] A2. Compare the reference video and the matching video The grid is divided into two parts, and the eigenvalues ​​of each grid intersection are obtained. The eigenvalues ​​include data of 1 and -1. The eigenvalues ​​of the grid intersections passed by the outer contour of the overlapped area are recorded as 1. When the appearance contour of the overlapped area does not pass through the grid intersection, the eigenvalue of the grid intersection closest to it is recorded as 1. The grid intersections with eigenvalues ​​of 1 are connected to obtain the fusion line area of ​​the reference video and the matching video.

[0013] A3. Obtain the pixel difference between the reference video and the matching video at each grid intersection in the fusion line area, and compare it with the pixel difference threshold in the database. When the pixel value difference at a grid intersection in the fusion area is less than the pixel difference threshold in the database, the characteristic value of the grid intersection is recorded as 1, otherwise it is recorded as -1. Connect the grid intersections with a characteristic value of 1 as the best fusion line to splice the reference video and the matching video, and record the spliced ​​video as the matching video.

[0014] A4. Use the matching video as the reference video and repeat the above steps until a complete working video of the target industrial equipment is obtained.

[0015] Preferably, the operating information of the target industrial equipment includes the gear speed per unit time, the product output and the ambient temperature when the target industrial equipment is working stably.

[0016] Preferably, the video is optimized, and the specific optimization process is as follows: at least two optimal fusion lines are obtained from the overlapping area obtained after matching the reference video and the matching video. If a moving component in the target industrial equipment passes through a fusion line, the optimal fusion line is recorded as an invalid fusion line, and the fusion line is not used for video fusion.

[0017] If the moving parts in the target industrial equipment do not pass through the fusion line, the fusion line is recorded as a valid fusion line and the video fusion is performed using the fusion line.

[0018] If the moving parts in the target equipment industry do not cross any fusion line, the reference video and the matching video are fused based on each optimal fusion line, and the fused video with the highest definition is obtained as the matching video.

[0019] Preferably, an early warning is issued when a fault is detected in the target industrial equipment in the video, and the specific response operations after the early warning are as follows: when a fault is detected in the target industrial equipment in the video, the clarity of the video and the operation of the target industrial equipment are detected respectively. If the video is not clear, it is determined that a fault has occurred in the video fusion, and each valid video is reintegrated and optimized until the video clarity meets the requirements; if the video is clear, it is determined that a fault has occurred in the target industrial equipment, and relevant staff are dispatched to inspect and repair the target industrial equipment.

[0020] In a second aspect, the present application provides a multi-system video surveillance fusion system, including: an effective video acquisition module: grayscale processing is performed on the working videos of the target industrial equipment captured by each surveillance camera, and each effective working video is acquired.

[0021] Effective video fusion module: After fusing the effective working videos, the complete working video of the target industrial equipment is obtained, and the working information of the target industrial equipment is extracted from it, and then the clarity evaluation coefficient of the complete working video is analyzed to determine whether the complete working video needs to be optimized.

[0022] Early warning module: The complete working video is displayed on the monitoring display screen in real time, and an early warning is issued when a failure of the target industrial equipment in the video is detected.

[0023] Database module: used to store information in the effective video acquisition module, the effective video fusion module and the early warning prompt module.

[0024] The beneficial effects of the present application are: 1. A multi-system video surveillance fusion method and system provided by the present application processes videos from various angles of the target industrial equipment to obtain each valid working video, and then fuses each valid working video to obtain a complete working video of the target industrial equipment, and then compares the working conditions of the target industrial equipment extracted from the complete working video with the actual working conditions of the target industrial equipment, thereby optimizing the video fusion process to obtain a clearer complete working video. The present application applies image processing technology to the field of industrial manufacturing. The staff can directly obtain the working conditions of the target industrial equipment through the fused complete working video, avoiding the waste of resources caused by frequent inspections by the staff, and also avoiding the economic losses caused by the failure to timely discover equipment failures.

[0025] 2. This application obtains a complete working video of the target industrial equipment by fusing the effective working videos of the target industrial equipment, applies image processing technology to the field of industrial manufacturing, and realizes remote monitoring of the target industrial equipment, which greatly reduces the waste of resources caused by manual inspections. The working conditions of the target industrial equipment can be directly observed through the fused complete working video, which also greatly improves the convenience of the staff's daily work.

[0026] 3. This application analyzes and compares the working conditions of the target industrial equipment obtained on the monitoring display with the working conditions of the target industrial equipment actually detected, and then optimizes the fused video, which is beneficial to improve the video clarity and make it more convenient for staff to monitor the operation of the target industrial equipment.

[0027] 4. The present application realizes the splicing of each valid working video by obtaining the overlapping area of ​​each adjacent valid working video and obtaining multiple optimal fusion lines from the overlapping area. The optimal fusion line obtained by the minimum pixel difference can minimize the problems such as blur and ghosting at the video fusion caused by video pixels, thereby effectively ensuring the quality of the fused video. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 The present invention is a flowchart of the steps for implementing the present application method.

[0030] Figure 2 This is a schematic diagram of the system structure connection for this application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] Reference Figure 1 As shown, the present application provides a multi-system video surveillance fusion method in the first aspect, comprising the following steps: Step 1, effective video acquisition: grayscale processing is performed on each working video of the target industrial equipment captured by each surveillance camera, and each effective working video is acquired.

[0033] In a specific example, the monitoring cameras include a long-range camera, a high-definition camera and an infrared camera; the long-range camera is used to obtain the appearance video of the target industrial equipment, the high-definition camera is used to obtain the operation video of the parts in the target industrial equipment; and the infrared camera is used to obtain the environment video of the target industrial equipment when it is working.

[0034] It should be noted that the length, width and resolution of each working video shot by each camera are the same.

[0035] In a specific example, the grayscale processing is performed on each working video of the target industrial equipment captured by each monitoring camera. The specific processing process is as follows: the red, green and blue color channel values ​​in each working video are recorded as , and , is the number of each work video, , is the pixel coordinates in each working video, according to the calculation formula: Get pixel points Gray value at ,in , and Respectively represent the weight factor of the red channel value in the database in the grayscale image calculation, the weight factor of the green channel value in the grayscale image calculation, and the weight factor of the blue channel value in the grayscale image calculation. Respectively represent the weight factor of the red channel value in the database in the grayscale image calculation, the weight factor of the green channel value in the grayscale image calculation, and the weight factor of the blue channel value in the grayscale image calculation.

[0036] It should be noted that , , , .

[0037] It should be noted that the weight factors corresponding to the red channel values, the green channel values, and the blue channel values ​​in each grayscale image are obtained by factor analysis. First, the information of spatial path attenuation of the red channel values, green channel values, and blue channel values ​​in the grayscale image is condensed, and then the variance explanation rate after rotation is obtained, and the weight is obtained by dividing the cumulative variance explanation rate.

[0038] It should be noted that factor analysis is a well-known technology. It is a multivariate statistical analysis method that starts from studying the internal dependencies of variables and reduces some variables with intricate relationships to a few comprehensive factors; information concentration is expressed as calculating the median; the variance explanation rate is the amount of information extracted by the factor, and the variance explanation rate = characteristic root / total number of analysis items; the variance explanation rate after rotation is expressed as the variance explanation rate of the factor after maximum variance rotation.

[0039] In a specific example, the specific acquisition process of obtaining each effective working video is as follows: extracting the appearance contour of the target industrial equipment from each grayscale image, and recording the length and width of the appearance contour as and , according to the calculation formula The ratio of the number of pixels in the appearance contour of the target industrial equipment in each grayscale image to the total number of pixels in the corresponding complete grayscale image is obtained. ,in is the number of pixels within the appearance contour of the target industrial equipment in each grayscale image, , and Respectively represent the length and width of the grayscale image in the database, It is the correction factor corresponding to the number of pixels within the appearance contour of the target industrial equipment in the grayscale image in the database.

[0040] If the ratio of the number of pixels in the appearance contour of the target industrial equipment in a grayscale image to the total number of pixels in the corresponding complete grayscale image is within the interval If the grayscale image is inside, the working video corresponding to the grayscale image is recorded as a valid working video; otherwise, the working video corresponding to the outer grayscale image is recorded as an invalid working video.

[0041] It should be noted that the length and width of the grayscale image in the database are the length and width of each working video captured by each camera.

[0042] It should be noted that .

[0043] It should be noted that the factor correction method is a well-known technology. The correction factor corresponding to the signal strength evaluation coefficient of each channel per unit time is obtained by the correction factor analysis method. First, the original data of the signal strength of each channel per unit time is obtained, and the average value is taken after removing the maximum and minimum values. The correction factor is then obtained by dividing the original data of the signal strength of each channel per unit time by the median after removing the maximum and minimum values.

[0044] Step 2: Effective video fusion: After fusing the effective working videos, obtain the complete working video of the target industrial equipment, and extract the working information of the target industrial equipment from it, and then analyze and obtain the clarity evaluation coefficient of the complete working video, so as to determine whether the complete working video needs to be optimized.

[0045] In a specific example, the effective working videos are fused to obtain a complete working picture of the target industrial equipment. The specific fusion process is as follows: A1. A specified effective working video is preprocessed as a reference video for video feature extraction, and the next adjacent effective working video is preprocessed as a matching video for video feature extraction. The features extracted from the reference video are matched with the features extracted from the matching video, and the feature area after matching is recorded as the overlapping area. A2. The reference video and the matching video are preprocessed. The grid is divided into two parts, and the eigenvalues ​​of each grid intersection are obtained. The eigenvalues ​​include data of 1 and -1. The eigenvalues ​​of the grid intersections passed by the outer contour of the overlapped area are recorded as 1. When the appearance contour of the overlapped area does not pass through the grid intersection, the eigenvalue of the grid intersection closest to it is recorded as 1. The grid intersections with eigenvalues ​​of 1 are connected to obtain the fusion line area of ​​the reference video and the matching video.

[0046] A3. Obtain the pixel difference between the reference video and the matching video at each grid intersection in the fusion line area, and compare it with the pixel difference threshold in the database. When the pixel value difference at a grid intersection in the fusion area is less than the pixel difference threshold in the database, the characteristic value of the grid intersection is recorded as 1, otherwise it is recorded as -1. Connect the grid intersections with a characteristic value of 1 as the best fusion line to splice the reference video and the matching video, and record the spliced ​​video as the matching video.

[0047] A4. Use the matching video as the reference video and repeat the above steps until a complete working video of the target industrial equipment is obtained.

[0048] It should be noted that video preprocessing includes performing vignetting correction, distortion correction and light correction on the video, and decoding the corrected video.

[0049] It should be noted that video features include corner points, blocks or shadow areas. The value is defined by the staff. The larger it is, the more accurate the optimal fusion line is and the clearer the fused video is.

[0050] It should be noted that the pixel difference threshold in the database is the maximum pixel difference at the video fusion point that ensures that the target industrial equipment can be clearly observed after video fusion.

[0051] In a specific example, the operating information of the target industrial equipment includes the gear speed per unit time, the product output and the ambient temperature when the target industrial equipment is working stably.

[0052] It should be noted that the operating information of the target industrial equipment is any parameter related to the target industrial equipment when it is working, which can be gear speed, equipment pressure, or equipment flow, and therefore cannot be understood as a limitation to the present application.

[0053] In a specific example, the analysis obtains the clarity evaluation coefficient of the video, and then determines whether the video needs to be optimized. The specific analysis and judgment process is as follows: the gear speed per unit time, the product output and the ambient temperature of the target industrial equipment are recorded as , and , according to the calculation formula The video clarity evaluation coefficient is obtained ,in , and , respectively represent the standard value of gear speed per unit time, the standard value of product output and the standard value of ambient temperature when the target industrial equipment in the database is working stably. , and , which respectively represent the weight factor corresponding to the gear speed per unit time when the target industrial equipment in the data is working stably, the weight factor corresponding to the product output, and the weight factor corresponding to the ambient temperature.

[0054] When the video clarity evaluation coefficient is in the range If the video is clear, no optimization is needed; otherwise, the video is not clear and needs to be optimized. , , , It should be noted that the weight factors corresponding to the gear speed per unit time when the target industrial equipment is working, the weight factors corresponding to the product output and the weight factors corresponding to the ambient temperature are obtained by factor analysis. First, the information of the spatial path attenuation of the gear speed per unit time, the product output and the ambient temperature when the target industrial equipment is working is concentrated, and then the variance explanation rate after rotation is obtained, and the weight is obtained by dividing the cumulative variance explanation rate.

[0055] It should be noted that the standard value of gear speed per unit time when the target industrial equipment in the database is working is expressed as the gear speed per unit time when the target industrial equipment is actually measured and working stably; the standard value of actual product output is expressed as the product output produced per unit time when the target industrial equipment is working stably; and the standard value of ambient temperature is expressed as the ambient temperature per unit time when the target industrial equipment is working stably.

[0056] It should be noted that the database includes the weight factor of the red channel value in the grayscale image calculation, the weight factor of the green channel value in the grayscale image calculation, and the weight factor of the blue channel value in the grayscale image calculation; it also includes the length and width of the grayscale image; it includes the correction factor corresponding to the number of pixels within the appearance contour of the target industrial equipment in the grayscale image; it also includes the standard value of the gear speed per unit time when the target industrial equipment is working, the standard value of the actual product output and the standard value of the ambient temperature; it also includes the weight factor corresponding to the gear speed per unit time when the target industrial equipment is working, the weight factor corresponding to the product output and the ambient temperature weight factor.

[0057] In a specific example, the video is optimized, and the specific optimization process is as follows: at least two optimal fusion lines are obtained from the overlapping area obtained after matching the reference video and the matching video. If a moving part in the target industrial equipment passes through a fusion line, the optimal fusion line is recorded as an invalid fusion line, and the fusion line is not used for video fusion.

[0058] If the moving parts in the target industrial equipment do not pass through the fusion line, the fusion line is recorded as a valid fusion line and the video fusion is performed using the fusion line.

[0059] If the moving parts in the target equipment industry do not cross any fusion line, the reference video and the matching video are fused based on each optimal fusion line, and the fused video with the highest definition is obtained as the matching video.

[0060] It should be noted that when the moving parts in the target industrial equipment cross the fusion line, the continuous movement of the parts will cause the pixels at the fusion line to change continuously, thereby affecting the video fusion effect.

[0061] Step 3: Early warning: The obtained complete working video is displayed on the monitoring display screen in real time, and an early warning is issued when a failure of the target industrial equipment in the video is detected.

[0062] It should be noted that the complete working video displayed on the monitoring display screen includes, in addition to the video, also working information of the target industrial equipment extracted from the video, such as gear speed, product output of the target industrial equipment, and ambient temperature.

[0063] In a specific example, an early warning is issued when a fault is detected in the target industrial equipment in the video. The specific response operations after the early warning are as follows: when a fault is detected in the target industrial equipment in the video, the clarity of the video and the operation of the target industrial equipment are detected respectively. If the video is not clear, it is determined that a fault has occurred in the video fusion, and each valid video is reintegrated and optimized until the video clarity meets the requirements; if the video is clear, it is determined that a fault has occurred in the target industrial equipment, and relevant staff are dispatched to inspect and repair the target industrial equipment.

[0064] It should be noted that the method for detecting the clarity of a video is to calculate a clarity evaluation coefficient of the video and determine whether the video is clear according to the interval of the clarity evaluation coefficient.

[0065] Reference Figure 2 As shown, the present application provides a multi-system video surveillance fusion system in a second aspect, including: an effective video acquisition module: grayscale processing is performed on the working videos of the target industrial equipment captured by each surveillance camera, and each effective working video is acquired.

[0066] Effective video fusion module: After fusing the effective working videos, the complete working video of the target industrial equipment is obtained, and the working information of the target industrial equipment is extracted from it, and then the clarity evaluation coefficient of the complete working video is analyzed to determine whether the complete working video needs to be optimized.

[0067] Early warning module: The complete working video is displayed in real time on the monitoring display screen, and an early warning is issued when a failure of the target industrial equipment in the video is detected.

[0068] Database module: used to store information in the effective video acquisition module, the effective video fusion module and the early warning prompt module.

[0069] The present application provides a multi-system video surveillance fusion method and system, which processes videos from various angles of the target industrial equipment to obtain each valid working video, and then fuses each valid working video to obtain a complete working video of the target industrial equipment, and then compares the working conditions of the target industrial equipment extracted from the complete working video with the actual working conditions of the target industrial equipment, and then optimizes the video fusion process to obtain a clearer complete working video. The present application applies image processing technology to the field of industrial manufacturing, and the staff can directly obtain the working conditions of the target industrial equipment through the fused complete working video, avoiding the waste of resources caused by frequent inspections by the staff, and also avoiding the economic losses caused by the failure to discover equipment failures in time.

[0070] The above contents are merely examples and explanations of the concept of the present application. The technicians in this technical field may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined in the present application, they should all fall within the protection scope of the present application.

Claims

1. A multi-system video surveillance fusion method, characterized in that: include: Step 1: Acquire valid videos: grayscale each working video of the target industrial equipment captured by each surveillance camera, and acquire each valid working video; Step 2: Effective video fusion: After fusing the effective working videos, the complete working video of the target industrial equipment is obtained, and the working information of the target industrial equipment is extracted from it, and then the clarity evaluation coefficient of the complete working video is obtained by analysis, so as to determine whether the complete working video needs to be optimized; Step 3: Early warning: The obtained complete working video is displayed on the monitoring display screen in real time, and an early warning is issued when a failure of the target industrial equipment in the video is detected.

2. A multi-system video surveillance fusion method according to claim 1, characterized in that: The monitoring cameras include long-range cameras, high-definition cameras and infrared cameras; the long-range cameras are used to monitor the appearance of the target industrial equipment, and the high-definition cameras are used to monitor the operation of the components in the target industrial equipment; Use infrared cameras to monitor the environmental video of the target industrial equipment while it is working.

3. A multi-system video surveillance fusion method according to claim 2, characterized in that: The grayscale processing is performed on each working video of the target industrial equipment captured by each monitoring camera. The specific processing process is as follows: The red, green and blue color channel values ​​in each working video are recorded as , and , is the number of each work video, , is the pixel coordinate, according to the calculation formula: Get the pixel points in each working video Gray value at ,in , and They respectively represent the weight factor of the red channel value in the database in the grayscale image calculation, the weight factor of the green channel value in the grayscale image calculation, and the weight factor of the blue channel value in the grayscale image calculation.

4. The multi-system video surveillance fusion method according to claim 3 is characterized in that: The specific acquisition process of obtaining each valid working video is as follows: The appearance contour of the target industrial equipment is extracted from each grayscale image, and the length and width of the appearance contour are recorded as and , according to the calculation formula Obtain the ratio of the number of pixels in the appearance contour of the target industrial equipment in each grayscale image to the total number of pixels in its corresponding complete grayscale image ,in is the number of pixels within the appearance contour of the target industrial equipment in each grayscale image, , and Respectively represent the length and width of the grayscale image in the database, is a correction factor corresponding to the number of pixels within the appearance contour of the target industrial equipment in the grayscale image in the database; If the ratio of the number of pixels in the appearance contour of the target industrial equipment in a grayscale image to the total number of pixels in the corresponding complete grayscale image is within the interval If the grayscale image is within the range of 0, the working video corresponding to the grayscale image is recorded as a valid working video; otherwise, the working video corresponding to the grayscale image is recorded as an invalid working video.

5. A multi-system video surveillance fusion method according to claim 4, characterized in that: After the effective working videos are fused, the complete working picture of the target industrial equipment is obtained. The specific fusion process is as follows: A1. Preprocess a designated valid working video as a reference video for video feature extraction, and preprocess the next adjacent valid working video as a matching video for video feature extraction, match the features extracted from the reference video with the features extracted from the matching video, and record the feature area after matching as the overlap area; A2. Compare the reference video and the matching video The grid is divided into two parts, and the characteristic value of each grid intersection is obtained. The characteristic value includes data of 1 and -1. The characteristic value of the grid intersection passed by the outer contour of the overlapped area is recorded as 1. When the appearance contour of the overlapped area does not pass through the grid intersection, the characteristic value of the grid intersection closest to it is recorded as 1. The grid intersections with characteristic values ​​of 1 are connected to obtain the fusion line area of ​​the reference video and the matching video. A3, obtaining the pixel difference between the reference video and the matching video at each grid intersection in the fusion line area, and comparing it with the pixel difference threshold in the database. When the pixel value difference at a grid intersection in the fusion area is less than the pixel difference threshold in the database, the characteristic value of the grid intersection is recorded as 1, otherwise it is recorded as -1. The grid intersections with characteristic values ​​of 1 are connected as the best fusion line to splice the reference video and the matching video, and the spliced ​​video is recorded as the matching video; A4. Use the matching video as the reference video and repeat the above steps until a complete working video of the target industrial equipment is obtained.

6. A multi-system video surveillance fusion method according to claim 5, characterized in that: The operating information of the target industrial equipment includes the gear speed per unit time, the product output and the ambient temperature when the target industrial equipment is working stably.

7. A multi-system video surveillance fusion method according to claim 6, characterized in that: The analysis obtains the video clarity evaluation coefficient, and then determines whether the video needs to be optimized. The specific analysis and judgment process is as follows: The gear speed per unit time, product output and ambient temperature of the target industrial equipment when it is working stably are recorded as , and , according to the calculation formula Get the video clarity evaluation coefficient ,in , and They represent the gear speed per unit time, the actual product output and the ambient temperature of the target industrial equipment in the database when it is working stably. , and They respectively represent the gear speed per unit time, the product output and the ambient temperature when the target industrial equipment in the data is working stably; When the video clarity evaluation coefficient is in the range If the video is clear, no optimization is needed; otherwise, the video is not clear and needs to be optimized.

8. The multi-system video surveillance fusion method according to claim 7, characterized in that: The video is optimized, and the specific optimization process is as follows: At least two optimal fusion lines are obtained from the overlapped area obtained after matching the reference video and the matching video. If a moving component in the target industrial equipment passes through a fusion line, the optimal fusion line is recorded as an invalid fusion line and is not used for video fusion. If the moving parts in the target industrial equipment do not pass through the fusion line, the fusion line is recorded as a valid fusion line and used for video fusion; If the moving parts in the target equipment industry do not cross any fusion line, the reference video and the matching video are fused based on each optimal fusion line, and the fused video with the highest definition is obtained as the matching video.

9. The multi-system video surveillance fusion method according to claim 8, characterized in that: When a fault is detected in the target industrial equipment in the video, an early warning is issued, and the specific response operations after the early warning are as follows: When a malfunction of the target industrial equipment in the video is detected, the video clarity and the operation of the target industrial equipment are tested respectively. If the video is not clear, it is determined that the video fusion has failed, and the valid videos are reintegrated and optimized until the video clarity meets the requirements. If the video is clear, it is determined that the target industrial equipment has failed, and relevant staff are dispatched to inspect and repair the target industrial equipment.

10. A multi-system video surveillance fusion system, used to execute a multi-video surveillance fusion method as described in any one of claims 1 to 9, characterized in that: include: Effective video acquisition module: grayscale processing is performed on each working video of the target industrial equipment captured by each surveillance camera, and each effective working video is obtained; Effective video fusion module: After fusing the effective working videos, the complete working video of the target industrial equipment is obtained, and the working information of the target industrial equipment is extracted from it, and then the clarity evaluation coefficient of the complete working video is analyzed to determine whether the complete working video needs to be optimized; Early warning module: Display the complete working video in real time on the monitoring display screen, and issue an early warning when a failure of the target industrial equipment in the video is detected; Database module: used to store information in the effective video acquisition module, the effective video fusion module and the early warning prompt module.

Citation Information

Patent Citations

  • Video fusion system and method

    CN112037264A

  • Method for screening matching pairs of feature points to splice images

    CN102819835A

  • Image splicing method for background homogeneous processing, storage medium and terminal

    CN112037134A

  • Three-dimensional video fusion splicing method

    CN112669205A

  • Improved GMS feature matching image splicing method

    CN113706384A