A visual intelligence-based audio and video monitoring method and system

Abstract

The application is suitable for the technical field of image recognition, and provides an audio and video monitoring method and system based on visual intelligence, which comprises the following steps: obtaining a historical video frame and a current video frame, extracting a first abnormal area between the current video frame and the historical video frame; if the number of pixels of the first abnormal area is greater than a preset number, extracting a plurality of subsequent video frames; respectively extracting a second abnormal area between each of the plurality of subsequent video frames and the historical video frame; determining whether an abnormal condition exists in an EMC test device according to the spatial position relationship between the first abnormal area and the plurality of second abnormal areas; if it is determined that the abnormal condition exists, sending an abnormal prompt information to a control terminal; and the abnormal prompt information is used for prompting a user to handle the abnormal condition. The above technical solution significantly improves the detection efficiency and accuracy of the abnormal condition.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image recognition, and particularly relates to an audio and video monitoring method and system based on visual intelligence. BACKGROUND

[0002] With the rapid development of electronic devices, electromagnetic compatibility (EMC) testing has become an important link for evaluating whether the electronic devices meet the relevant electromagnetic environment requirements. During the EMC testing process, abnormal phenomena may occur in the devices due to electromagnetic interference and other reasons, such as circuit overload, short circuit, etc., thereby causing safety hazards such as device fire or smoke. Therefore, how to monitor the EMC testing process in real time and effectively, especially to discover and respond to potential fire or smoke and other abnormal conditions in a timely manner during the testing process, has become a technical problem to be solved.

[0003] The abnormality detection method based on video monitoring is a new trend of safety monitoring. By acquiring and analyzing the visual information of the EMC testing site in real time, the safety hazards such as fire and smoke can be effectively detected, and an alarm can be sent in a timely manner. However, the existing visual monitoring methods mostly focus on static images or simple image processing techniques, lack in-depth analysis of dynamic changes between video frames, and are difficult to accurately and quickly discover potential abnormal conditions in complex and dynamically changing environments. SUMMARY

[0004] Therefore, the embodiments of the present application provide an audio and video monitoring method and system based on visual intelligence to solve the technical problem that the existing visual monitoring methods are difficult to accurately and quickly discover potential abnormal conditions in complex and dynamically changing environments.

[0005] The first aspect of the embodiments of the present application provides an audio and video monitoring method based on visual intelligence, which comprises:

[0006] acquiring a historical video frame and a current video frame, and extracting a first abnormal area between the current video frame and the historical video frame; the historical video frame refers to a video frame without an abnormal area;

[0007] if the number of pixels of the first abnormal area is greater than a preset number, extracting a plurality of subsequent video frames; the plurality of subsequent video frames refer to video frames corresponding to a plurality of sampling points after the current video frame;

[0008] extracting a second abnormal area between each of the plurality of subsequent video frames and the historical video frame;

[0009] determining whether an abnormal condition exists in the EMC testing device according to the spatial position relationship between the first abnormal area and the plurality of second abnormal areas; the abnormal condition includes smoke or open fire;

[0010] If it is determined that the abnormal condition exists, an abnormal prompt information is sent to a control terminal; the abnormal prompt information is used to prompt a user to handle the abnormal condition.

[0011] Further, the step of obtaining the historical video frame and the current video frame, and extracting a first abnormal region between the current video frame and the historical video frame comprises:

[0012] obtaining a historical video frame and a current video frame;

[0013] extracting an abnormal pixel point in the historical video frame and the current video frame; the abnormal pixel point refers to a pixel point whose difference between the same position in the historical video frame and the current video frame exceeds a first threshold value;

[0014] a set of continuous abnormal pixel points whose number exceeds a second threshold value is taken as a to-be-recognized region;

[0015] performing a gray scale processing on the to-be-recognized region to obtain a gray scale image;

[0016] extracting a gray scale texture feature in the gray scale image;

[0017] obtaining a preset sample feature corresponding to the open fire image, and calculating a similarity between the gray scale texture feature and the preset sample feature;

[0018] if the similarity is greater than a third threshold value, it is determined that the to-be-recognized region is a first abnormal region;

[0019] if the similarity is not greater than the third threshold value, it is determined whether the to-be-recognized region is the first abnormal region based on a smoke feature.

[0020] Further, the step of extracting the gray scale texture feature in the gray scale image comprises:

[0021] obtaining a neighboring region around each pixel point in the gray scale image; the neighboring region is a rectangular region with the pixel point as the center;

[0022] extracting an edge pixel point corresponding to four sides of the neighboring region;

[0023] calculating a gray scale difference between a plurality of edge pixel points;

[0024] counting a first number of gray scale difference values greater than a sixth threshold value;

[0025] if the first number exceeds a preset value, a value of the pixel point is set as a first flag value;

[0026] if the first number does not exceed the preset value, the value of the pixel point is set as a second flag value;

[0027] The binarized image composed of the first flag value and the second flag value is taken as the gray-scale texture feature.

[0028] Further, if the similarity is not greater than the third threshold value, the step of determining whether the to-be-identified region is the first abnormal region based on the smoke feature includes:

[0029] If the similarity is not greater than the third threshold value, a pixel difference value of the to-be-identified region at the same pixel position in the historical video frame and the current video frame is extracted.

[0030] A second number of differences between corresponding pixel difference values of adjacent pixel points that are less than a fourth threshold value is counted.

[0031] The second number is divided by a pixel number of the to-be-identified region to obtain a first proportion.

[0032] If the first proportion exceeds a preset proportion, it is determined that the to-be-identified region is the first abnormal region.

[0033] If the first proportion does not exceed the preset proportion, average pixel information of the to-be-identified region is extracted.

[0034] If a difference between the average pixel information and smoke sample pixel information is less than a fifth threshold value, it is determined that the to-be-identified region is the first abnormal region.

[0035] If the difference between the average pixel information and the smoke sample pixel information is not less than the fifth threshold value, it is determined that the to-be-identified region is a non-abnormal region.

[0036] Further, the step of determining whether the EMC test equipment has an abnormal condition according to a spatial position relationship between the first abnormal region and a plurality of the second abnormal regions includes:

[0037] A largest region of the first abnormal region and the plurality of the second abnormal regions is extracted.

[0038] A pixel point overlap proportion between each abnormal region and the largest region is calculated.

[0039] A third number of the pixel point overlap proportions that are greater than a preset proportion is counted.

[0040] If the third number is greater than a seventh threshold value, it is determined that the EMC test equipment has an abnormal condition.

[0041] If the third number is not greater than the seventh threshold value, it is determined that the EMC test equipment does not have an abnormal condition.

[0042] Further, the step of calculating the pixel point overlap proportion between each abnormal region and the largest region includes:

[0043] counting a third number of pixels that overlap between the abnormal area and the maximum area;

[0044] dividing the third number by the number of pixels in the abnormal area to obtain a pixel overlap ratio.

[0045] Further, after the step of obtaining a historical video frame and a current video frame, and extracting a first abnormal area between the current video frame and the historical video frame, the method further comprises:

[0046] If the number of pixels in the first abnormal area is not greater than a preset number, then the steps of obtaining a historical video frame and a current video frame, extracting a first abnormal area between the current video frame and the historical video frame, and subsequent steps are executed in a loop.

[0047] A second aspect of the embodiment of the application provides an audio and video monitoring device based on visual intelligence, comprising:

[0048] an obtaining unit configured to obtain a historical video frame and a current video frame, and extract a first abnormal area between the current video frame and the historical video frame; the historical video frame refers to a video frame without an abnormal area;

[0049] a first extracting unit configured to, if the number of pixels in the first abnormal area is greater than a preset number, extract a plurality of subsequent video frames; the plurality of subsequent video frames refer to video frames corresponding to a plurality of sampling points after the current video frame;

[0050] a second extracting unit configured to extract a second abnormal area between each of the plurality of subsequent video frames and the historical video frame;

[0051] a determining unit configured to determine whether an EMC test device has an abnormal condition according to a spatial position relationship between the first abnormal area and the plurality of second abnormal areas; the abnormal condition includes smoke or open flame;

[0052] a sending unit configured to, if it is determined that the abnormal condition exists, send an abnormal prompt information to a control terminal; the abnormal prompt information is used to prompt a user to handle the abnormal condition.

[0053] A third aspect of the embodiment of the application provides an audio and video monitoring system based on visual intelligence, the audio and video monitoring system comprising a device to be tested, an EMC test device, and a visual monitoring system:

[0054] the visual monitoring system is configured to obtain a historical video frame and a current video frame, and extract a first abnormal area between the current video frame and the historical video frame; the historical video frame refers to a video frame without an abnormal area;

[0055] The visual monitoring system is used for extracting a plurality of subsequent video frames if the pixel number of the first abnormal region is greater than a preset number, wherein the plurality of subsequent video frames refer to video frames corresponding to a plurality of sampling points located after the current video frame;

[0056] The visual monitoring system is used for extracting a second abnormal region between each of the plurality of subsequent video frames and the historical video frame;

[0057] The visual monitoring system is used for determining whether an abnormal condition exists in the EMC test equipment according to a spatial position relationship between the first abnormal region and the plurality of second abnormal regions, wherein the abnormal condition includes smoke or open flame;

[0058] The visual monitoring system is used for sending an abnormal prompt information to a control terminal if it is determined that the abnormal condition exists, wherein the abnormal prompt information is used for prompting a user to handle the abnormal condition.

[0059] A fourth aspect of the embodiment of the present application provides a terminal device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the audio and video monitoring method based on visual intelligence in the first aspect when executing the computer program.

[0060] A fifth aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the steps in the audio and video monitoring method based on visual intelligence in the first aspect.

[0061] Compared with the prior art, the embodiment of the present application has the beneficial effects that: by comparing the historical video frame and the current video frame, the first abnormal area is accurately extracted, and the significant abnormal change area is screened out by comparing the pixel quantity. The method can efficiently and in real time identify the abnormal area in the video frame, and provide reliable data support for subsequent abnormal condition analysis. When the pixel quantity of the first abnormal area exceeds the preset threshold, the method further introduces the analysis of multiple subsequent video frames, extracts the second abnormal area frame by frame, and analyzes according to the spatial position relationship of the abnormal areas. Through continuous comparison of multiple frame data, the probability of single frame misjudgment is effectively reduced, and the monitoring accuracy of abnormal conditions in a dynamic and complex environment is improved. By using the spatial position relationship between the multiple subsequent video frames and the historical video frame, the method can comprehensively analyze the occurrence time and position of abnormal conditions such as fire and smoke, and avoid the limitations of the traditional static image method that is difficult to accurately determine abnormal events. When an abnormal condition (such as smoke or open fire) is found, the system can immediately send an abnormal prompt information to the control terminal, and notify the user in real time to take emergency handling measures. This function greatly improves the safety in the test process, responds to abnormal conditions in time, effectively prevents potential accidents, and ensures the safety of test equipment and personnel. The detection efficiency and accuracy of abnormal conditions are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or related technical descriptions will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0063] Figure 1 A schematic flow chart of an audio and video monitoring method based on visual intelligence provided by the present application is shown;

[0064] Figure 2 A schematic diagram of an audio and video monitoring device based on visual intelligence provided by an embodiment of the present application is shown;

[0065] Figure 3 A schematic diagram of a terminal device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0066] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, technologies, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0067] The embodiments of the present application provide a visual intelligence-based audio and video monitoring method and system to solve the technical problem that the existing visual monitoring method is difficult to accurately and quickly find potential abnormal situations in a complex and dynamic environment.

[0068] Firstly, the present application provides a visual intelligence-based audio and video monitoring method. Please refer to Figure 1 , Figure 1 The present application provides a visual intelligence-based audio and video monitoring method. As shown in Figure 1 The visual intelligence-based audio and video monitoring method can include the following steps:

[0069] Step 101: Acquire historical video frames and current video frames, and extract a first abnormal area between the current video frames and the historical video frames; the historical video frames refer to video frames without abnormal areas before the current time point, i.e. video frames in a normal state;

[0070] During the test, the camera will shoot continuous video frames. The historical video frames refer to the video frames shot before the current time point without abnormal phenomena, i.e. the video frames in a normal state. The current video frames refer to the latest video frames shot at the current time, which are used for comparison and analysis with the historical video frames.

[0071] By comparing the current video frames and the historical video frames, the system will extract the "first abnormal area" between them, i.e. the area in the video that is different from the historical video frames, which may be smoke, fire or other abnormal image areas.

[0072] Specifically, step 101 specifically includes steps 1011 to 1018:

[0073] Step 1011: Acquire historical video frames and current video frames;

[0074] Step 1012: Extract abnormal pixel points in the historical video frames and the current video frames; the abnormal pixel points refer to the pixel points with a difference value between the pixel points at the same position in the historical video frames and the current video frames exceeding a first threshold value;

[0075] The abnormal pixel points refer to the pixel points at the same position in the current video frames and the historical video frames, whose value difference exceeds the preset "first threshold value".

[0076] For example, each video frame is composed of a large number of pixels, each pixel has its color (or grayscale) value. If the value of a certain pixel in the historical video frame is very different from the value in the current video frame (exceeding a preset "first threshold"), this pixel point is considered to be an "abnormal pixel point". This method is used to capture the significant changes between the current frame and the historical frame, which usually represents the appearance of a new, abnormal phenomenon (such as smoke or fire) in the image.

[0077] Step 1013: If the number of abnormal pixel points exceeds a second threshold and the set of continuous abnormal pixel points is a "to-be-identified region";

[0078] After obtaining the abnormal pixel points, the system will check the number of these abnormal pixel points. If the number of abnormal pixel points exceeds a second preset threshold, and these abnormal pixel points are continuous (i.e. they are located in the same region of the video image), these abnormal pixel points will be defined as a "to-be-identified region".

[0079] The set of continuous abnormal pixel points is more likely to represent a local, concentrated abnormal phenomenon, such as a flame or smoke, while the abnormal difference of a single pixel point may be noise or meaningless change.

[0080] Step 1014: Perform grayscale processing on the to-be-identified region to obtain a grayscale image;

[0081] After determining the to-be-identified region, the system will perform grayscale processing on these regions. Grayscale processing converts color images into grayscale images, simplifying image information and making it easier for subsequent feature extraction.

[0082] The grayscale image removes color information and only retains grayscale information, which is particularly helpful for analyzing the texture, shape and other features in the image.

[0083] Step 1015: Extract grayscale texture features in the grayscale image;

[0084] Grayscale texture features refer to the patterns or structures of grayscale level changes in the image. These features can reflect the texture and shape of objects in the image. For example, phenomena such as flames and smoke have specific texture features, which may be different from static backgrounds or objects in normal states.

[0085] Specifically, step 1015 specifically includes steps A1 to A7:

[0086] Step A1: Obtain the adjacent region around each pixel point in the grayscale image; the adjacent region is a rectangular region centered on the pixel point;

[0087] For each pixel point, the system will extract its surrounding adjacent region. Here, the adjacent region refers to a rectangular region within a certain range centered on the pixel point. The size of the adjacent region is usually set according to the characteristics and needs of the image, such as selecting a rectangular region of different sizes such as 16x16, 64x64, etc. The selection of the adjacent region is very important for subsequent texture feature extraction and analysis, as it determines the local range of analysis. By selecting the surrounding adjacent region, the local environmental changes of the pixel point can be captured, thereby better identifying the texture patterns in the image (such as flames, smoke, etc.).

[0088] Step A2: Extracting the edge pixel points corresponding to the four edges of the adjacent region;

[0089] Once the adjacent region is determined, the system will then extract edge pixel points from the four edges (top, bottom, left, right) of the region. The edge pixel points refer to the pixel points located on the four edges of the adjacent region.

[0090] Step A3: Calculating the gray level difference between multiple edge pixel points;

[0091] Next, the system will calculate the gray level difference between these edge pixel points. The gray level difference refers to the difference between the gray level values of two edge pixel points, reflecting the degree of gray level change in the region. If the gray level difference is large, it means that there is a strong texture change in the region, which may correspond to certain significant features in the image (such as the edge of a flame or the boundary of smoke).

[0092] Step A4: Counting the first number of gray level differences greater than the sixth threshold value;

[0093] The system will count the cases where the gray level difference is greater than the sixth threshold value. The sixth threshold value is a preset value used to determine which pixel points with large gray level differences are considered significant. The first number refers to the number of pixel points whose gray level difference exceeds the sixth threshold value. If there are a large number of significant gray level changes in the adjacent region, it may mean that the texture of the region is complex, which may correspond to abnormal phenomena such as flames, smoke, etc. It can be understood that for regions with obvious texture features, the differences between multiple edge pixel points are large, and the number of edge pixel points with large differences is large. For regions with no obvious texture features, the gray level values of multiple edge pixel points are uniform, i.e., the differences are not large.

[0094] Step A5: If the first number exceeds a preset value, set the value of the pixel point to a first flag value;

[0095] If the first number (i.e. the number of pixel points with a gray difference greater than the sixth threshold value) exceeds a preset threshold value, the system considers that the region where the pixel point is located has obvious texture features, and therefore sets the value of the pixel point as a first flag value. The first flag value usually represents that the region has strong texture features (such as the features of fire or other abnormal phenomena).

[0096] Step A6: If the first number does not exceed the preset value, set the value of the pixel point as a second flag value.

[0097] If the first number does not reach the preset threshold value, it means that the gray difference of the region is small, and there may be no significant texture change. Therefore, the system sets the value of the pixel point as a second flag value. The second flag value indicates that the region where the pixel point is located has relatively flat texture or no obvious abnormalities.

[0098] Step A7: Take the binary image composed of the first flag value and the second flag value as the gray texture feature.

[0099] Finally, the system will generate a binary image by combining the flag values (first flag value and second flag value) of the pixel points. In this image, the value of the pixel point is either the first flag value or the second flag value, forming obvious contrast (usually 0 and 1 or two different flag values). This binary image will serve as the gray texture feature, reflecting the texture changes of each region in the image. The binary processing helps to simplify the image information and highlight the texture features, providing effective input for subsequent abnormality recognition (such as fire or smoke).

[0100] In the embodiments corresponding to steps A1 to A7, the texture features of the image are extracted by analyzing the surrounding adjacent regions of each pixel point in the gray image. By comparing the gray difference of the edge pixel points and counting the number of significantly changed pixel points, the system can determine the texture features of the region. According to the degree of significance of the texture features, the system gives each pixel point a different flag value (first flag value or second flag value), and finally forms a binary image. This binary image reflects the texture changes in the image, serving as the basis for subsequent judgment of abnormal conditions such as fire or smoke.

[0101] Step 1016: Obtain the preset sample features corresponding to the open fire image, and calculate the similarity between the gray texture features and the preset sample features;

[0102] The preset sample features refer to the "open fire" features extracted from previously collected image samples. These sample features represent the typical texture features of fire images, which will be used by the system as a standard to compare with the to-be-identified region in the current video frame. The calculation of the similarity between binary images is a traditional technique and will not be described here.

[0103] Step 1017: If the similarity is greater than the third threshold, determining that the to-be-identified region is a first abnormal region.

[0104] If the similarity of the gray texture feature of the to-be-identified region and the preset flame feature exceeds the third preset threshold, the system will judge that the region is a first abnormal region, that is, the region may have an open fire or a fire source.

[0105] This judgment is based on accurate matching of flame image features, ensuring that only regions that truly conform to the flame feature are marked as abnormal.

[0106] Step 1018: If the similarity is not greater than the third threshold, determining whether the to-be-identified region is a first abnormal region based on a smoke feature.

[0107] If the similarity of the current to-be-identified region and the flame image feature is not sufficient to exceed the third threshold, the system will analyze whether the region has the characteristics of smoke.

[0108] The smoke feature is different from the flame feature, and is usually characterized by a more blurred and diffuse image pattern, so the system uses different feature extraction techniques to determine whether the region is a smoke region.

[0109] Specifically, step 1018 specifically includes steps B1 to B7:

[0110] Step B1: If the similarity is not greater than the third threshold, extracting pixel difference values of the to-be-identified region at the same pixel position in the historical video frame and the current video frame.

[0111] When the similarity (i.e., the similarity of the gray texture features of the current frame and the historical frame) does not reach the preset third threshold, the system considers that the to-be-identified region may not be a flame region, so further analysis is needed by other features to determine whether the region is smoke.

[0112] The pixel difference value refers to the difference in gray value of two pixels at the same position in the historical video frame and the current video frame. This difference can reflect the dynamic change of the region, especially the unstable phenomenon of smoke.

[0113] Step B2: Counting a second number of adjacent pixel point corresponding pixel difference values that are less than a fourth threshold.

[0114] The system will count the difference in pixel difference values between adjacent pixel points in the to-be-identified region, that is, the difference in difference between adjacent pixel points in the historical and current frames. If this difference is less than the preset fourth threshold, it is considered that this part of the change is small, and it may be a smooth transition background or a relatively blurred region such as smoke.

[0115] The second quantity refers to the number of adjacent pixel points that meet the condition (i.e., the difference is less than the fourth threshold value). This quantity reflects the degree of smooth change in the region.

[0116] Step B3: Divide the second quantity by the number of pixels in the region to be identified to obtain a first proportion;

[0117] The first proportion is obtained by dividing the second quantity (the number of pixels meeting the condition) by the total number of pixels in the region to be identified. This step aims to calculate the proportion of smooth pixel points in the entire region, reflecting the change pattern of the region.

[0118] If this proportion is high, it means that the pixel change in the region is relatively smooth, which may represent a phenomenon such as smoke that is blurred and has no obvious boundary.

[0119] Step B4: If the first proportion exceeds a preset proportion, determine the region to be identified as a first abnormal region;

[0120] If the first proportion exceeds a preset proportion threshold, it means that most of the pixel points in the region change smoothly and continuously, which may represent an abnormal phenomenon such as smoke that has no obvious boundary.

[0121] In this case, the system determines that the region to be identified is a first abnormal region, i.e., it may be smoke or other similar abnormal conditions.

[0122] Step B5: If the first proportion does not exceed the preset proportion, extract the average pixel information of the region to be identified;

[0123] If the first proportion does not exceed the preset proportion threshold, it means that the pixel change in the region is relatively intense, which does not conform to the characteristics of smoke. At this time, the system will extract the average pixel information of the region.

[0124] The average pixel information refers to the average value of all pixel values in the region, which can generally reflect the overall brightness or color distribution of the region. For thick smoke, the average pixel information usually presents white color. Therefore, whether it is a smoke region can be judged based on the average pixel information.

[0125] Step B6: If the difference between the average pixel information and the smoke sample pixel information is less than a fifth threshold value, determine the region to be identified as a first abnormal region;

[0126] This step compares the average pixel information of the region to be identified with the preset smoke sample pixel information. If the difference between the two is less than the fifth threshold value, it means that the overall pixel information of the region to be identified is very similar to the smoke sample, which may be a smoke region. The smoke sample pixel information refers to the pixel information corresponding to the pre-stored thick fog image.

[0127] It is worth noting that steps B2 to B4 are suitable for cases with low smoke concentration, while steps B5 and B6 are suitable for cases with high smoke concentration.

[0128] Step B7: If the difference between the average pixel information and the smoke sample pixel information is not less than the fifth threshold value, it is determined that the to-be-identified region is a non-abnormal region.

[0129] If the difference between the average pixel information and the smoke sample pixel information is greater than or equal to the fifth threshold value, it means that the pixel information of the to-be-identified region has a large difference with the smoke sample, and the system considers that the region does not conform to the characteristics of smoke. Therefore, the system judges that the region is a non-abnormal region, that is, it is considered that no abnormal phenomenon occurs in the region.

[0130] In the embodiments corresponding to steps B1 to B7, when the similarity does not reach the third threshold value, the system judges whether the region has the characteristics of smoke by calculating the pixel difference, counting the difference between adjacent pixel points, and calculating the proportion of flat pixel points in the region. If the proportion is high, it means that the region changes gently and may be smoke; otherwise, the system will further extract the average pixel information of the region and compare it with the smoke sample. If the difference is small, it is considered that the region is a smoke abnormal region; if the difference is large, it is considered that the region is a non-abnormal region. This process helps to judge the abnormal situation in the image through multi-level analysis of pixel difference and sample comparison.

[0131] In the corresponding embodiments of steps 1011 to 1018, by comparing the pixel points at the same position in the historical video frame and the current video frame, the abnormal pixel points with a difference exceeding a first threshold value are extracted. This step can accurately identify the image area that changes in the continuous time period, effectively avoiding false detection caused by environmental changes or background interference. By setting the difference threshold value, it is ensured that only the pixel points with significant changes are marked as abnormal, which helps to filter out irrelevant noise information and improves the detection accuracy. The continuity of the abnormal pixel points is judged, and the set of continuous and second threshold value exceeding abnormal pixel points is taken as the to-be-identified area. This step can effectively filter out the continuously changing area, rule out transient or accidental interference, and enhance the stability of the detection algorithm. Through threshold control, false judgments caused by small-range changes in a short time can be avoided, thereby improving the reliability of the region identification. By calculating the similarity with the pre-set open flame sample features, the flame and other types of abnormalities such as smoke can be efficiently distinguished. This step compares the similarity between the gray texture features and the pre-set open flame features to determine whether the to-be-identified area is an open flame, ensuring the accuracy of the fire alarm. When the open flame feature similarity is high, the fire event can be identified and alarmed in time, which has the advantage of fast response. In the case where the open flame feature similarity threshold is not reached, the method will further judge based on the smoke feature. Through the analysis of the smoke feature, the abnormal condition caused by the smoke can be effectively identified. This processing logic ensures that the system can not only identify the fire, but also monitor the possible fire precursor such as smoke leakage, thereby improving the comprehensiveness and accuracy of the system. The multi-level abnormal region identification and judgment mechanism of the above embodiments combines pixel difference, texture features, and sample comparison, and can efficiently and accurately identify potential fires or smoke abnormalities in complex environments. Through the setting of multiple thresholds, interference can be filtered without missing, thereby improving the accuracy and response speed in the detection process.

[0132] Step 102: If the number of pixels in the first abnormal area is greater than a preset number, a plurality of subsequent video frames are extracted; the plurality of subsequent video frames refer to video frames corresponding to a plurality of sampling points after the current video frame;

[0133] If there are enough pixel points in the first abnormal area (i.e., the size or significance of the abnormal area exceeds a predetermined threshold), the system considers that the current abnormal phenomenon may have a higher risk. Therefore, the system will further acquire a plurality of subsequent video frames to continuously monitor the abnormal phenomenon. The subsequent video frames refer to frames after the current video frame (i.e., frames later in time) for continued analysis of whether the abnormal area is continuous or intensified.

[0134] As an optional embodiment of the present application, after step 101, further comprising: if the number of pixels of the first abnormal region is not greater than a preset number, then repeatedly performing the steps of obtaining the historical video frame and the current video frame, extracting the first abnormal region between the current video frame and the historical video frame, and the subsequent steps (i.e. the steps after step 101 are repeatedly performed).

[0135] Step 103: Extracting a second abnormal region between each of the subsequent video frames and the historical video frame;

[0136] In these subsequent video frames, the system continues to find the "second abnormal region" between the historical video frame. The calculation logic of the second abnormal region is consistent with that of the first abnormal region, which is not limited here.

[0137] Step 104: According to the spatial position relationship between the first abnormal region and the plurality of second abnormal regions, determining whether the EMC test equipment has an abnormal condition; the abnormal condition includes smoke or open flame;

[0138] The system analyzes the spatial position relationship between the first abnormal region and the second abnormal region in the subsequent video frames. For example, whether the abnormal region appears in a similar position and presents a gradually expanding trend, which may indicate the occurrence of an abnormal phenomenon such as a fire source or smoke. The abnormal condition mainly refers to the appearance of smoke or open flame, which may mean that the equipment has overheating, short circuit or other dangerous situations during the EMC test.

[0139] It is worth noting that since the judgment of the abnormal region is based on the difference in pixel points, there is a certain limitation and error, so in order to improve the recognition accuracy, the spatial position relationship between the difference in pixel points and the abnormal region is used to jointly determine whether an abnormality occurs. The spatial position relationship is based on the dynamic change characteristics of open flame or smoke to determine whether an abnormality occurs, and the difference in pixel points is based on a single video frame to determine the abnormal region.

[0140] Specifically, step 104 specifically includes steps 1041 to 1045:

[0141] Step 1041: Extracting the largest region in the first abnormal region and the plurality of second abnormal regions;

[0142] First, the system extracts the largest region from the plurality of second abnormal regions and compares it with the first abnormal region. The "largest region" here refers to the region with the largest pixel area among all second abnormal regions. The purpose of extracting the largest region is to find the most significant region among the plurality of abnormal regions, which occupies the largest area, because this region may contain more significant abnormal features and have a greater impact on the judgment result.

[0143] Step 1042: Calculate the pixel point overlap ratio between each abnormal region and the maximum region;

[0144] For each abnormal region (including the first abnormal region and other second abnormal regions), the system will calculate the pixel point overlap ratio between the region and the maximum region.

[0145] The pixel point overlap ratio refers to how many pixel points between the current abnormal region and the maximum region are overlapping, and the proportion of the overlapping part to the current abnormal region. This helps to measure the spatial relationship between different abnormal regions and the maximum region.

[0146] The way to calculate the overlap ratio is usually to calculate the number of overlapping pixel points between regions, and compare it with the total number of pixel points in the region. For example, if the number of overlapping pixels between a region and the maximum region accounts for a high proportion of the total number of pixels in the region, it means that the relationship between this region and the maximum region is close, which may represent the concentrated area of abnormal phenomena.

[0147] Specifically, step 1042 specifically includes steps C1 to C2:

[0148] Step C1: Count the third number of overlapping pixel points between the abnormal region and the maximum region;

[0149] The third number refers to how many pixel points between the abnormal region and the maximum region are overlapping. Simply put, overlapping pixel points refer to pixel points that have the same pixel position and belong to both regions.

[0150] For example, if there are two rectangular regions A and B, and part of their regions overlap, then the overlapping pixel points are the pixel points that exist in both regions. Counting the number of overlapping pixel points is to calculate the number of such common pixel points.

[0151] Suppose that the abnormal region A and the maximum region B are both sets of pixel points. The number of overlapping pixel points is the size of the intersection of set A and set B (i.e. the number of pixel points they have in common).

[0152] This can be achieved by calculating the intersection of the two regions, such as using the intersection operation of sets to find the pixel points that belong to both regions.

[0153] Step C2: Divide the third number by the number of pixel points in the abnormal region to obtain the pixel point overlap ratio.

[0154] The pixel point overlap ratio refers to the proportion of the above-mentioned overlapping pixel points (i.e., the third number) in the current abnormal area. The pixel point overlap ratio represents the degree of overlap between the current abnormal area and the maximum area. If this ratio is high, it means that the spatial overlap between the abnormal area and the maximum area is large, which may represent a closer association or a more significant abnormal phenomenon.

[0155] In the embodiment corresponding to step C1 to step C2, through these two steps, the system can calculate the pixel point overlap ratio of each abnormal area and the maximum area. This overlap ratio measures the degree of spatial association between the abnormal area and the maximum area, which can help determine whether there is strong spatial overlap between different areas, thereby reflecting the abnormal relationship between the areas.

[0156] Step 1043: Counting a third number of pixel point overlap ratios greater than a preset ratio;

[0157] The system will count the pixel point overlap ratios of all abnormal areas and the maximum area, and find those areas with an overlap ratio greater than a preset ratio.

[0158] The third number refers to the statistical result of the number of areas that meet the overlap ratio condition (i.e., the area with an overlap ratio greater than a preset ratio). This statistical quantity can reflect how many areas have a high spatial overlap relationship with the maximum area. Areas with a large overlap ratio usually indicate a high concentration of abnormal areas.

[0159] Step 1044: If the third number is greater than a seventh threshold value, it is determined that the EMC test equipment has an abnormal condition;

[0160] Since the distribution of open flames or smoke in dynamic video presents a situation of large and small or gradually expanding, there will be some overlapping areas and changes, so the third number corresponding to the overlap ratio can be used to determine that the EMC test equipment has an abnormal condition (i.e., open flames or smoke).

[0161] If the third number (i.e., the number of abnormal areas that meet the overlap ratio condition) exceeds the seventh threshold value, the system considers that the abnormal characteristics of these areas are more obvious and concentrated, and therefore determines that the EMC test equipment has an abnormal condition.

[0162] Step 1045: If the third number is not greater than the seventh threshold value, it is determined that the EMC test equipment does not have an abnormal condition.

[0163] If the third number does not exceed the seventh threshold value, it means that the overlap relationship between the abnormal area and the maximum area is weak, which may be an occasional or scattered abnormal phenomenon, and the overall concentration of abnormalities is low, which does not meet the characteristics of open flames or smoke.

[0164] In the embodiments corresponding to steps 1041 to 1045, the maximum abnormal area is extracted, and the overlap ratio of the area with other abnormal areas is calculated; the number of areas with an overlap ratio greater than a preset ratio is counted; if the number of areas meeting the condition is large (i.e., exceeds a seventh threshold), it is considered that the device may have a serious abnormality, otherwise, it is considered that the device is in a normal state. Through this method, the system can find out whether there is a widespread and coherent abnormal phenomenon in multiple abnormal areas, so as to make a reasonable judgment on the abnormal state of the EMC test device.

[0165] Step 105: If it is determined that the abnormal condition exists, sending an abnormal prompt information to the control terminal; the abnormal prompt information is used to prompt the user to handle the abnormal condition.

[0166] Once the system confirms the presence of smoke or open fire and other abnormal conditions, it will immediately send an alarm information to the control terminal (such as the monitoring interface of the test personnel or the alarm system) to notify the relevant personnel to pay attention and take appropriate measures. The prompt information can be a visual or audio alarm to attract the attention of the test personnel and remind them to intervene and handle the abnormality in time to prevent safety accidents.

[0167] In the embodiments corresponding to steps 101 to 104, by comparing the historical video frame and the current video frame, the first abnormal area is accurately extracted, and the significant abnormal change area is screened out by comparing the pixel number. This method can efficiently and in real time identify the abnormal area in the video frame, providing reliable data support for subsequent abnormal condition analysis. When the pixel number of the first abnormal area exceeds a preset threshold, the method further introduces the analysis of multiple subsequent video frames, extracts the second abnormal area frame by frame, and analyzes the spatial position relationship of these abnormal areas. Through continuous comparison of multiple frame data, the probability of single frame misjudgment is effectively reduced, and the monitoring accuracy of abnormal conditions in dynamic and complex environments is improved. By using the spatial position relationship between multiple subsequent video frames and historical video frames, the method can comprehensively analyze the timing and location of abnormal conditions such as fire and smoke, avoiding the limitations of traditional static image methods that are difficult to accurately judge abnormal events. When an abnormal condition (such as smoke or open fire) is found, the system can immediately send an abnormal prompt information to the control terminal to notify the user to take emergency handling measures in real time. This function greatly improves the safety of the test process, responds to abnormal conditions in time, effectively prevents potential accidents, and ensures the safety of test equipment and personnel. The detection efficiency and accuracy of abnormal conditions are significantly improved.

[0168] As Figure 2 The application provides an audio and video monitoring device based on visual intelligence, please see Figure 2 , Figure 2A schematic diagram of an audio and video monitoring device based on visual intelligence is shown in the figure. Figure 2 The audio and video monitoring device based on visual intelligence comprises:

[0169] The acquisition unit 21 is configured to acquire a historical video frame and a current video frame, and extract a first abnormal area between the current video frame and the historical video frame; the historical video frame refers to a video frame without an abnormal area.

[0170] The first extraction unit 22 is configured to extract a plurality of subsequent video frames if the number of pixels of the first abnormal area is greater than a preset number; the plurality of subsequent video frames refer to video frames corresponding to a plurality of sampling points after the current video frame.

[0171] The second extraction unit 23 is configured to extract a second abnormal area between each of the plurality of subsequent video frames and the historical video frame.

[0172] The determination unit 24 is configured to determine whether an EMC test device has an abnormal condition according to the spatial position relationship between the first abnormal area and the plurality of second abnormal areas; the abnormal condition includes smoke or open flame.

[0173] The sending unit 25 is configured to send an abnormal prompt information to a control terminal if it is determined that the abnormal condition exists; the abnormal prompt information is used to prompt a user to handle the abnormal condition.

[0174] The audio and video monitoring device based on visual intelligence provided by the present application can accurately extract a first abnormal area by comparing a historical video frame and a current video frame, and screen out a significant abnormal change area by comparing the number of pixels. This method can efficiently and in real time identify the abnormal area in the video frame, and provide reliable data support for subsequent abnormal condition analysis. When the number of pixels of the first abnormal area exceeds a preset threshold, the method further introduces analysis of a plurality of subsequent video frames, extracts a second abnormal area frame by frame, and analyzes the spatial position relationship of the abnormal areas. Through continuous comparison of multiple frames of data, the probability of single frame misjudgment is effectively reduced, and the monitoring accuracy of abnormal conditions in a dynamic and complex environment is improved. By using the spatial position relationship between the plurality of subsequent video frames and the historical video frame, the method can comprehensively analyze the occurrence time and position of abnormal conditions such as fire and smoke, avoiding the limitations of traditional static image methods that are difficult to accurately determine abnormal events. When an abnormal condition (such as smoke or open flame) is found, the system can immediately send an abnormal prompt information to a control terminal, and notify the user in real time to take emergency measures. This function greatly improves the safety during the test, responds to abnormal conditions in a timely manner, effectively prevents potential accidents, and ensures the safety of the test equipment and personnel. The detection efficiency and accuracy of abnormal conditions are significantly improved.

[0175] Figure 3 This is a schematic diagram of a terminal device provided in an embodiment of the present invention. Figure 3 As shown, a terminal device 3 in this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a visual intelligence-based audio and video monitoring program. When the processor 30 executes the computer program 32, it implements the steps in the various embodiments of the visual intelligence-based audio and video monitoring method described above, for example... Figure 1 Steps 101 to 105 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each unit in the above-described device embodiments, for example... Figure 2 The function of the unit shown.

[0176] For example, the computer program 32 can be divided into one or more units, which are stored in the memory 31 and executed by the processor 30 to complete the present invention. The one or more units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 32 in the terminal device 3. For example, the specific functions of each unit of the computer program 32 can be divided as follows:

[0177] The acquisition unit is used to acquire historical video frames and the current video frame, and extract a first abnormal region between the current video frame and the historical video frames; the historical video frame refers to a video frame that does not have an abnormal region.

[0178] The first extraction unit is used to extract multiple subsequent video frames if the number of pixels in the first abnormal region is greater than a preset number; the multiple subsequent video frames refer to the video frames corresponding to multiple sampling points located after the current video frame.

[0179] The second extraction unit is used to extract the second abnormal regions between the multiple subsequent video frames and the historical video frames respectively;

[0180] The determining unit is configured to determine whether there is an abnormal condition in the EMC testing equipment based on the spatial relationship between the first abnormal area and multiple second abnormal areas; the abnormal condition includes smoke or open flame.

[0181] The sending unit is used to send an abnormality prompt message to the control terminal if the abnormality is determined to exist; the abnormality prompt message is used to prompt the user to handle the abnormality.

[0182] The terminal device includes, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that... Figure 3The terminal device 3 is only an example and does not constitute a limitation on the terminal device 3, and can include more or fewer components, or combine certain components, or different components, for example, the terminal device can also include an input / output device, a network access device, a bus, etc.

[0183] The processor 30 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0184] The memory 31 can be an internal storage unit of the terminal device 3, for example, a hard disk or a memory of the terminal device 3. The memory 31 can also be an external storage device of the terminal device 3, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 31 can include both the internal storage unit and the external storage device of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the terminal device 3. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0185] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0186] It should be noted that the information interaction, execution process, etc. between the above devices / units, since the same concept as the method embodiment of the present application, the specific functions and the technical effects brought by it, specific can refer to the method embodiment part, here will not be repeated.

[0187] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or software functional unit. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0188] The embodiment of the present application also provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps in each method embodiment.

[0189] The embodiment of the present application provides a computer program product, when the computer program product runs on a mobile terminal, so that the mobile terminal executes the steps in each method embodiment.

[0190] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the above-mentioned embodiment methods, which can be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium, and the computer program can realize the steps in each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form, etc. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, such as U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0191] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0192] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software mode depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0193] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0194] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, which can be located in one place or distributed on a plurality of network units.

[0195] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of described features, integers, steps, operations, elements, and / or components, but does not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0196] It should also be understood that the term "and / or" as used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0197] As used in the specification and the appended claims of the present application, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to monitoring" depending on the context. Similarly, the phrase "if it is determined" or "if it is monitored [that a described condition or event] can be interpreted depending on the context as meaning "upon determining" or "in response to determining" or "upon monitoring [that a described condition or event]" or "in response to monitoring [that a described condition or event]".

[0198] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0199] In the present application, the expression "one embodiment" or "some embodiments" or the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the expressions "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the present specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0200] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A visual intelligence-based audio-video monitoring method, characterized in that, The audio and video monitoring method based on visual intelligence comprises: acquiring a historical video frame and a current video frame, and extracting a first abnormal area between the current video frame and the historical video frame; the historical video frame refers to a video frame without an abnormal area; if the number of pixels of the first abnormal area is greater than a preset number, extracting a plurality of subsequent video frames; the plurality of subsequent video frames refer to video frames corresponding to a plurality of sampling points after the current video frame; respectively extracting a second abnormal area between the plurality of subsequent video frames and the historical video frame; determining whether an abnormal condition exists for EMC test equipment according to the spatial positional relationship between the first abnormal area and the plurality of second abnormal areas; the abnormal condition includes smoke or open flame; if it is determined that the abnormal condition exists, sending an abnormal prompt information to a control terminal; the abnormal prompt information is used to prompt a user to handle the abnormal condition; wherein the acquiring a historical video frame and a current video frame, and extracting a first abnormal area between the current video frame and the historical video frame comprises: acquiring a historical video frame and a current video frame; extracting an abnormal pixel point in the historical video frame and the current video frame; the abnormal pixel point refers to a pixel point whose difference between the same position in the historical video frame and the current video frame exceeds a first threshold value; taking an abnormal pixel point set with a number of abnormal pixel points exceeding a second threshold value and being continuous as a to-be-recognized area; performing gray scale processing on the to-be-recognized area to obtain a gray scale image; extracting a gray scale texture feature in the gray scale image; acquiring a preset sample feature corresponding to an open flame image, and calculating a similarity between the gray scale texture feature and the preset sample feature; if the similarity is greater than a third threshold value, determining that the to-be-recognized area is a first abnormal area; if the similarity is not greater than the third threshold value, determining whether the to-be-recognized area is a first abnormal area based on a smoke feature; wherein the extracting a gray scale texture feature in the gray scale image comprises: acquiring an adjacent area around each pixel point in the gray scale image; the adjacent area is a rectangular area with the pixel point as the center; extracting an edge pixel point corresponding to four sides of the adjacent area; calculating a gray scale difference value between a plurality of edge pixel points; counting a first number of gray scale difference values greater than a sixth threshold value; if the first number exceeds a preset value, setting a value of the pixel point as a first flag value; if the first number does not exceed the preset value, setting the value of the pixel point as a second flag value; taking a binary image composed of the first flag value and the second flag value as the gray scale texture feature; wherein the if the similarity is not greater than the third threshold value, determining whether the to-be-recognized area is a first abnormal area based on a smoke feature comprises: step B1: if the similarity is not greater than the third threshold value, extracting a pixel difference value of the to-be-recognized area at the same pixel position in the historical video frame and the current video frame; step B2: counting a second number of pixel difference values between adjacent pixel points less than a fourth threshold value; step B3: dividing the second number by the number of pixels of the to-be-recognized area to obtain a first proportion; Step B4: If the first proportion exceeds a preset proportion, the to-be-identified region is determined as a first abnormal region; Step B5: If the first proportion does not exceed the preset proportion, average pixel information of the to-be-identified region is extracted; Step B6: If a difference between the average pixel information and smoke sample pixel information is less than a fifth threshold, the to-be-identified region is determined as the first abnormal region; wherein, steps B2 to B4 are used to judge a case that smoke concentration is low, and steps B5 and B6 are used to judge a case that smoke concentration is high; Step B7: If the difference between the average pixel information and the smoke sample pixel information is not less than the fifth threshold, the to-be-identified region is determined as a non-abnormal region.

2. The visual intelligence based audio-video surveillance method as claimed in claim 1, wherein, The step of determining whether the EMC test equipment has an abnormal condition according to the spatial position relationship between the first abnormal region and the plurality of second abnormal regions comprises: extracting a maximum region among the first abnormal region and the plurality of second abnormal regions; calculating a pixel point overlapping proportion between each abnormal region and the maximum region; counting a third number of pixel point overlapping proportions greater than a preset proportion; if the third number is greater than a seventh threshold, determining that the EMC test equipment has an abnormal condition; if the third number is not greater than the seventh threshold, determining that the EMC test equipment does not have an abnormal condition.

3. The visual intelligence based audio-video surveillance method as claimed in claim 2, wherein, The step of calculating the pixel point overlapping proportion between each abnormal region and the maximum region comprises: counting a third number of overlapping pixel points between the abnormal region and the maximum region; dividing the third number by a number of pixel points in the abnormal region to obtain the pixel point overlapping proportion.

4. The visual intelligence based audio-video surveillance method as claimed in claim 1, wherein, After the step of obtaining historical video frames and current video frames, extracting a first abnormal region between the current video frames and the historical video frames, further comprising: if a pixel number of the first abnormal region is not greater than a preset number, cyclically executing the step of obtaining historical video frames and current video frames, extracting a first abnormal region between the current video frames and the historical video frames, and subsequent steps.

5. A visual intelligence based audio-video surveillance system, characterized in that, The audio and video monitoring system comprises a to-be-tested equipment, an EMC test equipment and a visual monitoring system: The visual monitoring system is used to obtain historical video frames and current video frames, and extract a first abnormal region between the current video frames and the historical video frames; the historical video frames refer to video frames without abnormal regions; The visual monitoring system is used to extract a plurality of subsequent video frames if a pixel number of the first abnormal region is greater than a preset number; the plurality of subsequent video frames refer to video frames corresponding to a plurality of sampling points after the current video frames; The visual monitoring system is used to extract a second abnormal region between a plurality of the subsequent video frames and the historical video frames, respectively; The visual monitoring system is used to determine whether the EMC test equipment has an abnormal condition according to a spatial position relationship between the first abnormal region and the plurality of second abnormal regions; The abnormal condition comprises smoke or open flame; The visual monitoring system is used to send abnormal prompt information to a control terminal if it is determined that the abnormal condition exists; The visual monitoring system is used to send abnormal prompt information to a control terminal if it is determined that the abnormal condition exists; The abnormal prompt information is used for prompting the user to handle the abnormal condition; The method comprises the following steps: Obtain historical video frames and current video frames; Extract abnormal pixel points in the historical video frames and the current video frames; the abnormal pixel points refer to pixel points whose difference between the same positions in the historical video frames and the current video frames exceeds a first threshold value; If the number of abnormal pixel points exceeds a second threshold value and the abnormal pixel points are continuous, the abnormal pixel points are taken as a to-be-identified region; Perform gray processing on the to-be-identified region to obtain a gray image; Extract a gray texture feature in the gray image; Obtain a preset sample feature corresponding to the open fire image, and calculate the similarity between the gray texture feature and the preset sample feature; If the similarity is greater than a third threshold value, it is determined that the to-be-identified region is a first abnormal region; If the similarity is not greater than the third threshold value, it is determined whether the to-be-identified region is the first abnormal region based on a smoke feature; The method comprises the following steps: Obtain an adjacent region around each pixel point in the gray image; the adjacent region is a rectangular region with the pixel point as the center; Extract edge pixel points corresponding to four sides of the adjacent region; Calculate the gray difference between a plurality of edge pixel points; Count a first number of gray difference values greater than a sixth threshold value; If the first number exceeds a preset value, the value of the pixel point is set as a first flag value; If the first number does not exceed the preset value, the value of the pixel point is set as a second flag value; A binary image composed of the first flag value and the second flag value is taken as the gray texture feature; If the similarity is not greater than the third threshold value, it is determined whether the to-be-identified region is the first abnormal region based on a smoke feature; Step B1: if the similarity is not greater than the third threshold value, extract the pixel difference of the to-be-identified region at the same pixel position in the historical video frames and the current video frames; Step B2: count a second number of pixel difference values between adjacent pixel points that are less than a fourth threshold value; Step B3: divide the second number by the number of pixels in the to-be-identified region to obtain a first proportion; Step B4: if the first proportion exceeds a preset proportion, it is determined that the to-be-identified region is the first abnormal region; Step B5: if the first proportion does not exceed the preset proportion, extract average pixel information of the to-be-identified region; Step B6: if the difference between the average pixel information and smoke sample pixel information is less than a fifth threshold value, it is determined that the to-be-identified region is the first abnormal region; steps B2 to B4 are used to judge the case that the smoke concentration is low, and steps B5 and B6 are used to judge the case that the smoke concentration is high; Step B7: if the difference between the average pixel information and the smoke sample pixel information is not less than the fifth threshold value, it is determined that the to-be-identified region is a non-abnormal region.

6. A terminal device, characterized by comprising: The terminal device comprises a memory, a processor, and a visual intelligence-based audio and video monitoring program stored in the memory and executable on the processor, and the visual intelligence-based audio and video monitoring program is configured to implement the steps in the visual intelligence-based audio and video monitoring method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program, when executed by a processor, implements the steps in the visual intelligence-based audio and video monitoring method according to any one of claims 1 to 4.

Images