Safety monitoring system and monitoring method based on machine vision

Abstract

The invention relates to the field of network security monitoring and image processing, in particular to a security monitoring system and monitoring method based on machine vision. Comprising the following steps: collecting visual image data of server hardware in real time, carrying out multi-frequency domain decomposition on the visual image data, filtering and recombining obtained signals of different frequency domains, and generating a denoised reconstructed image; global feature representation is obtained based on a deep network, a space-time diagram is constructed, behavior analysis is carried out, and comprehensive space-time feature representation is obtained; and carrying out risk assessment after carrying out fusion processing on the comprehensive spatial-temporal feature representation and sensor data, and determining an early warning strategy. The problem that noise interference cannot be effectively separated and suppressed, so that a detection result is misreported or missed to report is solved; complex abnormal behaviors or dangerous operations are difficult to identify effectively; the multi-modal fusion with the data of the sensor is not carried out, so that the risk assessment is not comprehensive enough; and an early warning mechanism lacks flexibility, the response speed lags behind or response measures are not accurate enough.

Description

Technical Field

[0001] The present invention relates to the field of network security monitoring and image processing, and in particular to a machine vision-based security monitoring system and a monitoring method. Background Art

[0002] With the rapid development of data centers and cloud computing technologies, server hardware security monitoring has become increasingly important. In data centers, server hardware monitoring is an important part of ensuring the stable operation of the system. In the past, server hardware monitoring mainly relied on manual inspection and basic hardware monitoring tools, which had problems such as untimely monitoring and high false alarm rates. With the development of technology, some server hardware security monitoring systems based on machine vision have been applied to actual scenarios.

[0003] However, the existing server hardware security monitoring systems based on machine vision still have some shortcomings. First, when processing images in the complex environment inside the server cabinet, these systems often find it difficult to effectively separate and suppress interference from multiple noise sources such as server equipment operation and light changes inside the cabinet, resulting in hardware status detection results being easily affected by noise, resulting in false positives or missed positives. Second, in terms of server hardware anomaly detection and maintenance operation analysis, existing systems find it difficult to effectively identify complex hardware failure modes or potentially dangerous maintenance operations, especially in multi-server, high-density deployment scenarios, where the system performance often fails to meet actual needs. Furthermore, many systems fail to fully utilize data from different sensors inside the server for multi-modal fusion, resulting in an incomplete risk assessment of the server's operating status, which may ignore some potential hardware failures or safety hazards. The early warning mechanism of existing systems often lacks flexibility, resulting in a delayed response speed to server abnormalities or inaccurate response measures in actual applications, ultimately affecting the safety operation of the data center. Summary of the invention

[0004] The present invention provides a safety monitoring system and a monitoring method based on machine vision to solve the problems that the interference from various noise sources such as equipment operation and ambient light changes cannot be effectively separated and suppressed, resulting in the detection results being easily affected by noise, causing false alarms or missed alarms; it is difficult to effectively identify complex abnormal behaviors or dangerous operations, and the data from different sensors cannot be fully utilized for multimodal fusion, resulting in incomplete risk assessment and possible neglect of potential dangerous factors in the environment; the early warning mechanism lacks flexibility, which may lead to delayed response speed or inaccurate response measures in actual applications, ultimately affecting safety assurance.

[0005] A machine vision-based safety monitoring system and monitoring method of the present invention specifically include the following technical solutions:

[0006] A safety monitoring method based on machine vision comprises the following steps:

[0007] S1. Collect visual image data of server hardware in real time, perform multi-frequency domain decomposition on the visual image data through generalized Fourier-Bessel transform to obtain signals in different frequency domains; perform adaptive nonlinear filtering on the signals in different frequency domains to obtain frequency domain signals after filtering; generate a denoised reconstructed image based on the frequency domain signals after filtering;

[0008] S2. Based on the denoised reconstructed image, a global feature representation is obtained through a deep network of high-order polynomial nonlinear superposition; based on the global feature representation, a spatiotemporal graph is constructed, and behavioral analysis is performed to obtain a comprehensive spatiotemporal feature representation; the comprehensive spatiotemporal feature representation is fused with the sensor data to obtain fused fuzzy decision data; based on the fused fuzzy decision data, risk assessment is performed to generate an early warning strategy.

[0009] Preferably, the S1 specifically includes:

[0010] By introducing nonlinear transformation, adaptive nonlinear filtering is performed on signals in different frequency domains to obtain frequency domain signals after filtering.

[0011] Preferably, the S1 specifically includes:

[0012] The filtered frequency domain signals are recombined through weighted fusion to obtain the denoised reconstructed image.

[0013] Preferably, the S2 specifically includes:

[0014] Based on the denoised reconstructed image, a deep network with high-order polynomial nonlinear superposition is introduced to extract the feature representation of each layer of the deep network, and the feature representation of all layers of the deep network is weightedly fused to obtain the global feature representation.

[0015] Preferably, the S2 specifically includes:

[0016] Based on the global feature representation, a space-time graph is constructed; the high-order Bessel function and Laplace operator joint transformation are introduced to analyze the space-time graph and obtain the space-time feature representation; based on the space-time feature representation, a convolution operation is performed to aggregate the space-time feature information at different times and obtain a comprehensive space-time feature representation.

[0017] Preferably, the S2 specifically includes:

[0018] Through fuzzy logic and convolution integral, the comprehensive spatiotemporal feature representation and sensor data are fused to obtain the fused fuzzy decision data.

[0019] Preferably, the S2 specifically includes:

[0020] Based on the fused fuzzy decision data, risk assessment is performed to obtain risk assessment results; based on the risk assessment results, an early warning strategy is generated through nonlinear mapping and fuzzy reasoning models.

[0021] A safety monitoring system based on machine vision includes the following parts:

[0022] Image acquisition module, data preprocessing module, target detection module, behavior analysis module, early warning response module, database;

[0023] An image acquisition module captures the visual image data of the server hardware in real time and transmits the collected visual image data to the data preprocessing module;

[0024] The data preprocessing module performs multi-frequency domain decomposition of visual image data through generalized Fourier-Bessel transform to obtain signals in different frequency domains; performs adaptive nonlinear filtering on the signals in different frequency domains to obtain frequency domain signals after filtering; reconstructs the image through weighted fusion based on the frequency domain signals after filtering to generate a denoised reconstructed image; and transmits the denoised reconstructed image to the target detection module and database;

[0025] The target detection module is based on a deep network with high-order polynomial nonlinear superposition. It extracts the feature representation of each layer of the deep network from the denoised reconstructed image, and performs weighted fusion on the feature representation of all layers of the deep network to obtain the global feature representation; the global feature representation is transmitted to the behavior analysis module and the database;

[0026] The behavior analysis module uses a high-order Bessel transform network to construct a spatiotemporal graph based on the global feature representation, extracts the spatiotemporal feature representation, aggregates the spatiotemporal features through convolution operations, and forms a comprehensive spatiotemporal feature representation; the comprehensive spatiotemporal feature representation is transmitted to the early warning response module and the database;

[0027] The early warning response module fuses the comprehensive spatiotemporal feature representation with the sensor data to obtain the fused fuzzy decision data; based on the fused fuzzy decision data, it conducts risk assessment to obtain the risk assessment results; based on the risk assessment results, it generates early warning strategies through nonlinear mapping and fuzzy reasoning models;

[0028] The database is used to store data transmitted by the data preprocessing module, target detection module, and behavior analysis module.

[0029] The beneficial effects of the technical solution of the present invention are:

[0030] 1. Through the combination of generalized Fourier-Bessel transform and adaptive nonlinear filtering, the environmental noise is effectively separated and suppressed, while retaining the key visual image features. The adaptive nonlinear filtering process not only takes into account the characteristics of local noise, but also enhances the contrast of visual image signals through nonlinear transformation, thereby improving the accuracy and robustness of visual image processing and ensuring the reliability of subsequent target detection;

[0031] 2. Use a deep network with high-order polynomial nonlinear superposition to perform multi-level feature extraction and fusion on the denoised reconstructed image, effectively capturing the complex features in the denoised reconstructed image, especially the high-order features; the superposition of multiple layers of features and the generation of global features enable the machine vision system to accurately detect potential dangerous targets or abnormal equipment status;

[0032] 3. By constructing a spatiotemporal graph and combining high-order Bessel functions and Laplace operator joint transforms, the machine vision system can accurately capture the behavior patterns and relationships of target objects in the spatiotemporal dimension; the aggregation of spatiotemporal features further enhances the ability of the machine vision system to analyze target behaviors and identify abnormal operations, and can effectively detect potential security risks;

[0033] 4. The present invention not only relies on the data of the machine vision system, but also combines the data from various sensors (such as temperature, pressure, vibration, etc.), and fuses these multimodal data through fuzzy logic and convolution integration to form unified risk assessment data; it improves the comprehensive judgment ability of the machine vision system on the environmental status, making the risk assessment more comprehensive and accurate;

[0034] 5. Based on the fused fuzzy decision data, the machine vision system realizes dynamic risk assessment through second-order derivative analysis, and triggers different levels of early warning response mechanisms according to the risk assessment results. The machine vision system sets multiple early warning thresholds and can flexibly control different early warning levels, from low-level early warnings (such as sound alarms) to high-level early warnings (such as automatic shutdowns), ensuring that the machine vision system can respond promptly and accurately in various risk situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a structural diagram of a safety monitoring system based on machine vision according to the present invention;

[0036] Figure 2 This is a flow chart of a safety monitoring method based on machine vision according to the present invention;

[0037] Figure 3 A topological diagram of a server visual monitoring scene according to the present invention;

[0038] Figure 4This is a design diagram of the machine vision-based security monitoring system architecture described in the present invention. DETAILED DESCRIPTION

[0039] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0041] The following is a detailed description of a specific scheme of a machine vision-based safety monitoring system and monitoring method provided by the present invention in conjunction with the accompanying drawings.

[0042] See attached Figure 1 , which shows a structure diagram of a machine vision-based safety monitoring system provided by an embodiment of the present invention, the system includes the following parts:

[0043] Image acquisition module, data preprocessing module, target detection module, behavior analysis module, early warning response module, database;

[0044] The image acquisition module captures the visual image data of the server hardware in real time through the camera and transmits the captured image data to the data preprocessing module;

[0045] The data preprocessing module uses the generalized Fourier-Bessel transform to perform multi-frequency domain decomposition on the visual image data to obtain signals in different frequency domains; the signals in different frequency domains are subjected to noise suppression and feature enhancement through adaptive nonlinear filters, and finally the image data is reconstructed through weighted fusion to obtain a denoised reconstructed image; the denoised reconstructed image is transmitted to the target detection module and database;

[0046] The target detection module, based on a deep network of high-order polynomial nonlinear superposition, extracts multi-layer features from the denoised reconstructed image and generates a global feature representation for identifying potential dangerous targets or abnormal equipment status; the global feature representation is transmitted to the behavior analysis module and database;

[0047] The behavior analysis module uses a high-order Bessel transform network to construct a spatiotemporal graph based on global feature representation, extracts spatiotemporal feature representation, aggregates spatiotemporal features through convolution operations, and forms a comprehensive spatiotemporal feature representation, which is used to analyze the target's behavior pattern and identify dangerous behaviors or abnormal operations; the comprehensive spatiotemporal feature representation is transmitted to the early warning response module and database;

[0048] The early warning response module fuses the comprehensive spatiotemporal feature representation with other sensor data to obtain fused fuzzy decision data; based on the fused fuzzy decision data, it performs risk assessment to obtain risk assessment results; based on the risk assessment results, it generates early warning signals through nonlinear mapping and fuzzy reasoning models to trigger corresponding safety response measures. The machine vision system sets multiple early warning thresholds, and each early warning threshold corresponds to a different level of early warning or response mechanism.

[0049] The database is used to store data transmitted by the data preprocessing module, target detection module, and behavior analysis module.

[0050] See attached Figure 2 , which shows a flow chart of a safety monitoring method based on machine vision provided by an embodiment of the present invention, the method comprising the following steps:

[0051] S1. Collect visual image data of server hardware in real time, perform multi-frequency domain decomposition on the visual image data through generalized Fourier-Bessel transform to obtain signals in different frequency domains; perform adaptive nonlinear filtering on the signals in different frequency domains to obtain frequency domain signals after filtering; generate a denoised reconstructed image based on the frequency domain signals after filtering;

[0052] See attached Figure 3 and attached Figure 4 ,The client captures the visual image data of the server hardware through the ,image acquisition module, and transmits it to the server side. The server side includes a ,data preprocessing module, a target detection module, a behavior analysis module, and an early ,warning response module. The captured visual image data is stored in a standard ,format and transmitted to the data preprocessing module in real time through a high-speed data ,transmission interface (such as GigE or USB3.0), and storage management is performed in the server ,cabinet to ensure the integrity and availability of the data at any time.

[0053] The collected visual image data contains rich environmental information, but also contains a lot of complex noise, which may come from equipment operation, ambient light changes and other uncertain factors. Therefore, preprocessing is required to improve the accuracy of subsequent analysis. The data preprocessing module in the server-side elastic computing device uses the generalized Fourier-Bessel transform to decompose the visual image data in multiple frequency domains. The generalized Fourier-Bessel transform formula is:

[0054]

[0055] Among them, F i (x, y) represents the value of the visual image signal in the ith frequency domain, which is the frequency domain component of the input visual image data after the generalized Fourier-Bessel transform, that is, the signal in the ith frequency domain; I(x, y) is the grayscale value of the input visual image data at the spatial coordinates (x, y); m and n are the spatial position offsets of the visual image data; ψi(m, n) represents the basis function of the ith frequency domain; is a complex exponential term, which represents the contribution of different frequency components in Fourier transform, ω m，n is the angular frequency, j is the ordinal unit; J v (ar) is the Bessel function, which represents the amplitude distribution in the frequency domain, v is the order of the Bessel function, α is the parameter for adjusting the frequency amplitude, and r is the spatial distance

[0056] After completing the multi-frequency domain decomposition of the visual image data, the data preprocessing module also performs multi-frequency domain decomposition on the signals F i (x, y) is processed by adaptive nonlinear filtering to suppress noise and enhance key features. The adaptive nonlinear filter not only considers the characteristics of local noise, but also introduces nonlinear transformation to further optimize the processing effect. The specific formula of adaptive nonlinear filtering is:

[0057]

[0058] Among them, F′ i (x, y) represents the visual image signal in the i-th frequency domain after the adaptive nonlinear filtering process, that is, the frequency domain signal after the filtering process; represents the estimated value of the noise of the visual image signal at position (x, y) in the ith frequency domain, obtained by statistically analyzing the grayscale changes in the local area of ​​the visual image signal; i It is the adjustment parameter of the adaptive nonlinear filter, which is used to control the intensity of the filter to ensure that useful signals are retained while suppressing noise; the nonlinear function tanh(γ·F i (x, y)) further enhances the contrast of the visual image signal in the ith frequency domain, where γ is a nonlinear coefficient used to adjust the intensity of the enhancement of the visual image signal in the ith frequency domain; ω is the frequency component in the frequency domain, used to describe the distribution of the visual image signal in the ith frequency domain in the frequency domain.

[0059] The frequency domain signals after filtering are recombined, and the results of different frequency domains are weighted and fused to form a denoised reconstructed image. The formula is:

[0060]

[0061] Where I′(x, y) is the value of the reconstructed image at the spatial coordinate (x, y) after denoising; α i is the weighting coefficient of the ith frequency domain, indicating the importance of the ith frequency domain in the final image reconstruction; Represents the total number of frequency domains; cos(βω) is the cosine function used for frequency domain fusion, β is the parameter for adjusting the frequency of the cosine function, controlling the role of the frequency domain components in the reconstructed image, and ω is the frequency component in the frequency domain. The above-mentioned denoised reconstructed image I′(x, y) is obtained, in which the noise has been effectively suppressed while retaining important image features.

[0062] S2. Based on the denoised reconstructed image, a global feature representation is obtained through a deep network of high-order polynomial nonlinear superposition; based on the global feature representation, a spatiotemporal graph is constructed, and behavioral analysis is performed to obtain a comprehensive spatiotemporal feature representation; the comprehensive spatiotemporal feature representation is fused with the sensor data to obtain fused fuzzy decision data; based on the fused fuzzy decision data, risk assessment is performed to generate an early warning strategy.

[0063] The target detection module takes the denoised reconstructed image as input and uses a deep network with high-order polynomial nonlinear superposition to perform target detection, accurately identifying potential dangerous targets or abnormal equipment status from the denoised reconstructed image.

[0064] Specifically, a multi-layer feature representation is extracted through a deep network with high-order polynomial nonlinear superposition. The formula is as follows:

[0065]

[0066] Among them, G l is the feature representation of the l-th layer of the deep network; Y l represents the input feature of the lth layer of the deep network; Y1 = I'(x, y) represents the reconstructed image after denoising, which is the input feature of the first layer of the deep network; after multiple layers of nonlinear transformation The generated feature representation will be used as the input feature of the l+1th layer deep network; is the q-th power of the input feature of the l-th layer deep network, V l is the weight matrix of the l-th layer deep network, b l is the bias term of the l-th layer deep network, q represents the order of the polynomial, and is obtained by taking the q-order derivative of a specific dimension in the input feature of the l-th layer deep network. The calculation of can further capture the high-order features in the reconstructed image after denoising. z is a specific dimension (such as spatial coordinates) in the input feature of the l-th layer of the deep network; p is the highest order of the polynomial, that is, the depth of the extracted features, which determines the nonlinear expression ability of the deep network; is the superposition coefficient, which controls the influence of the feature representation of the previous layer of deep network on the feature representation of the current layer of deep network. Each layer of deep network feature representation G l It will be gradually superimposed in multiple layers to form a more expressive global feature representation.

[0067] The generation of global feature representation is achieved through the following formula:

[0068]

[0069] Among them, G global is the global feature representation; δ l is the fusion weight of the feature representation of each layer of the deep network; L represents the total number of layers of the deep network with high-order polynomial nonlinear superposition. By weighted fusion of the feature representations of all layers of the deep network and integrating them over the time dimension r, a global feature representation that integrates the information of each layer of the deep network is obtained.

[0070] The behavior analysis module is based on the global feature representation G global The purpose of constructing the space-time graph G is to capture the relationship and behavior patterns of the target objects in time and space through the structured representation of the graph, and identify dangerous behaviors or abnormal operations. The specific implementation steps are as follows:

[0071] Global feature representation G global Each detection target in the deep network, that is, the feature representation of each layer of the deep network, is a node in the spatiotemporal graph. The feature vector of the node comes from a specific subset of the global feature representation, which is used to represent the spatial position, speed, and appearance characteristics of the detection target; the edge between nodes represents the spatiotemporal relationship between different detection targets. The edge features are calculated by the feature vectors of two nodes. Common calculation methods include Euclidean distance, cosine similarity, etc. The weight of the edge depends on the physical distance and relative speed between the detection targets, that is, the natural logarithm is used as the base, and the exponent is the Euclidean distance between the two nodes divided by the adjustment parameter used to control the decay speed of the edge weight; the spatiotemporal graphs at different times are connected through the time dimension to form a time series graph. The node at each moment is connected to its node at the next moment through an edge to ensure continuity in time and space.

[0072] After constructing the space-time graph G, the space-time graph is analyzed. In order to better capture the space-time characteristics, a joint transformation of high-order Bessel function and Laplace operator is introduced. The formula is as follows:

[0073]

[0074] H0=σ(W0·G global +b0)

[0075] Among them, Ht+1 It is the spatiotemporal feature representation of the next moment, combining the second-order derivative features of the spatiotemporal graph and the spatiotemporal feature representation of the current moment H t ; H0 is the initial spatiotemporal feature representation; is the Laplace operator, which is used to extract the local second-order derivative features of the spatiotemporal graph and enhance the representation of spatiotemporal dependencies; J k is the kth order Bessel function; represents the highest order of Bessel function; is the weight corresponding to the k-th order Bessel function, which controls the contribution of different order spatiotemporal features in the spatiotemporal graph; λ and They are the adjustment coefficient and frequency parameter, which are used to adjust the amplitude and frequency of the changes in spatiotemporal features in the time dimension; cosh is the hyperbolic cosine function, which further enhances the expressiveness of spatiotemporal features; W0 is the weight matrix, which is used to perform a linear transformation on the global feature representation; b0 is the bias vector, which is added to the result of the linear transformation and is used to adjust the center position of the spatiotemporal feature representation to avoid over-reliance on zero values.

[0076] The extracted spatiotemporal feature representations are aggregated through convolution operations to form a comprehensive spatiotemporal feature representation, whose formula is:

[0077]

[0078] Among them, H final It is a comprehensive spatiotemporal feature representation; is the weighting coefficient of the time dimension; W c is the convolution kernel, which aggregates the spatiotemporal feature information at different times by performing convolution operations on the spatiotemporal feature representation, and finally forms a comprehensive spatiotemporal feature representation; T is the length of the time series, that is, the total number of time steps.

[0079] The comprehensive spatiotemporal features are represented as a comprehensive video stream that can reflect the current risk status. The server displays the processed video stream through the front end. Operators can view real-time data, adjust security monitoring system parameters, and view historical alarm records through the configuration page.

[0080] The early warning response module takes the comprehensive spatiotemporal feature representation generated by the behavior analysis module as input and performs fusion processing with the data collected by other sensors (such as temperature sensors, pressure sensors, vibration sensors, humidity sensors, etc.). Other sensor data are used to supplement the image data of the machine vision system and provide more dimensional data to help the security monitoring system more accurately assess the environmental status and potential safety risks. The fusion processing is achieved through fuzzy logic and convolution integral, and its formula is:

[0081]

[0082] Among them, Df is the fused fuzzy decision data, which serves as the basic data for risk assessment; u is the index of other sensor data; N is the total number of other sensor data; is the membership function of the fuzzy set, indicating that The calculated membership degree, It is from S u The eigenvalues ​​extracted from u is the u-th sensor data matrix, which is obtained by convolution operation and fusion of comprehensive spatiotemporal features; dω is the integral of the frequency variable, indicating that the sensor data is fully processed in the frequency domain. The integration operation ensures that all frequency domain components in the sensor data are effectively processed.

[0083] The risk assessment of the fused fuzzy decision data is performed as follows:

[0084]

[0085] Where R(t) is the risk level at the current time t, which represents the risk value obtained by evaluating the input data at the current time, that is, the risk assessment result; M is the total number of risk assessment factors; c is the index of the risk assessment factor; It represents the second-order derivative of the fused fuzzy decision data in the time dimension, reflecting the dynamic changes of the fused fuzzy decision data over time; is the weight parameter in fuzzy reasoning; R(t-1) is the risk level at the previous moment.

[0086] According to the risk assessment results, the early warning strategy is determined through nonlinear mapping and fuzzy reasoning model. The formula is as follows:

[0087]

[0088] Among them, D output (t) is the decision output signal at time t; θ is the decision weight, which indicates the influence of the risk assessment result in the early warning decision; τ is the standardization coefficient, which is used to adjust the scale of the risk level R(t) to ensure the smoothness of the decision output signal. The final output D output (t) is the early warning signal of the alarm system, which is used to trigger corresponding safety measures, such as alarm, notification or automatic shutdown.

[0089] The alarm system sets multiple warning thresholds, each of which corresponds to a different level of warning or response mechanism. 。utput(t) When a certain warning threshold is reached or exceeded, the alarm system will trigger the corresponding warning mechanism. According to the triggered warning mechanism, the alarm system decides what warning measures to initiate. For example, the warning levels are divided into three levels: low, medium and high, corresponding to different risk levels. Low-level warning: may trigger a sound alarm or display a warning message on the monitoring screen to prompt the operator to pay attention; medium-level warning: the alarm system may send a text message or email to notify the safety manager to further investigate the potential risk; high-level warning: the alarm system may immediately execute emergency measures such as automatic shutdown and safety isolation to prevent possible major accidents.

[0090] This enables flexible control of different warning levels, ensuring that the machine vision system can respond promptly and accurately in various risk situations, thereby ensuring the security of the network environment.

[0091] In summary, a safety monitoring system and monitoring method based on machine vision have been completed.

[0092] The order of the embodiments of the invention is for description only and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0093] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0094] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A safety monitoring method based on machine vision, characterized in that: The following steps are involved: S1, real-time collection of visual image data from server hardware, multi-frequency domain decomposition of the visual image data through generalized Fourier-Bessel transform, to obtain signals in different frequency domains; Performing adaptive nonlinear filtering on signals in different frequency domains to obtain frequency domain signals after filtering; Based on the frequency domain signal after filtering, a denoised reconstructed image is generated; S2, based on the denoised reconstructed image, a global feature representation is obtained through a deep network with high-order polynomial nonlinear superposition; Based on the global feature representation, a spatiotemporal graph is constructed and behavior analysis is performed to obtain a comprehensive spatiotemporal feature representation; the comprehensive spatiotemporal feature representation is fused with the sensor data to obtain fused fuzzy decision data; Based on the fused fuzzy decision data, risk assessment is carried out and early warning strategies are generated.

2. A safety monitoring method based on machine vision according to claim 1, characterized in that: The S1 specifically includes: By introducing nonlinear transformation, adaptive nonlinear filtering is performed on signals in different frequency domains to obtain frequency domain signals after filtering.

3. A safety monitoring method based on machine vision according to claim 2, characterized in that: In the S1, it also includes: The filtered frequency domain signals are recombined through weighted fusion to obtain the denoised reconstructed image.

4. The method for safety monitoring based on machine vision according to claim 1, characterized in that: The S2 specifically includes: Based on the denoised reconstructed image, a deep network with high-order polynomial nonlinear superposition is introduced to extract the feature representation of each layer of the deep network, and the feature representation of all layers of the deep network is weightedly fused to obtain the global feature representation.

5. The method for safety monitoring based on machine vision according to claim 4, characterized in that: In the S2, it also includes: Based on the global feature representation, a space-time graph is constructed; the high-order Bessel function and Laplace operator joint transformation are introduced to analyze the space-time graph and obtain the space-time feature representation; based on the space-time feature representation, a convolution operation is performed to aggregate the space-time feature information at different times and obtain a comprehensive space-time feature representation.

6. A safety monitoring method based on machine vision according to claim 5, characterized in that: In the S2, it also includes: Through fuzzy logic and convolution integral, the comprehensive spatiotemporal feature representation and sensor data are fused to obtain the fused fuzzy decision data.

7. A safety monitoring method based on machine vision according to claim 6, characterized in that: In the S2, it also includes: Based on the fused fuzzy decision data, risk assessment is performed to obtain risk assessment results; based on the risk assessment results, an early warning strategy is generated through nonlinear mapping and fuzzy reasoning models.

8. A machine vision-based safety monitoring system, applied to the machine vision-based safety monitoring method as claimed in claim 1, characterized in that: Includes the following parts: Image acquisition module, data preprocessing module, target detection module, behavior analysis module, early warning response module, database; An image acquisition module captures the visual image data of the server hardware in real time and transmits the collected visual image data to the data preprocessing module; The data preprocessing module performs multi-frequency domain decomposition on the visual image data through generalized Fourier-Bessel transform to obtain signals in different frequency domains; Performing adaptive nonlinear filtering on signals in different frequency domains to obtain frequency domain signals after filtering; reconstructing images through weighted fusion based on the frequency domain signals after filtering to generate denoised reconstructed images; transmitting the denoised reconstructed images to the target detection module and the database; The target detection module is based on a deep network with high-order polynomial nonlinear superposition. It extracts the feature representation of each layer of the deep network from the denoised reconstructed image, and performs weighted fusion on the feature representation of all layers of the deep network to obtain the global feature representation; the global feature representation is transmitted to the behavior analysis module and the database; The behavior analysis module uses a high-order Bessel transform network to construct a spatiotemporal graph based on the global feature representation, extracts the spatiotemporal feature representation, aggregates the spatiotemporal features through convolution operations, and forms a comprehensive spatiotemporal feature representation; the comprehensive spatiotemporal feature representation is transmitted to the early warning response module and the database; Database, used to store data transmitted by the data preprocessing module, target detection module, and behavior analysis module; The early warning response module fuses the comprehensive spatiotemporal feature representation with the sensor data to obtain the fused fuzzy decision data; Based on the fused fuzzy decision data, risk assessment is carried out to obtain risk assessment results; based on the risk assessment results, an early warning strategy is generated through nonlinear mapping and fuzzy reasoning models.

Images