A warehouse safety monitoring method and system based on visual detection

Abstract

This invention discloses a vision-based warehouse safety monitoring method and system, relating to the field of spatial monitoring technology. The method includes: installing spatial markers in the warehouse and acquiring high-frequency warehouse images using industrial cameras; converting pixel coordinates in the warehouse images to camera coordinates based on camera parameters and establishing a ray model using different camera perspectives; optimizing the ray model and outputting spatial coordinates based on multiple camera perspectives; defining structural units based on the spatial coordinates and calculating their areas; analyzing deformation parameters and calculating deformation coefficients based on area transformations; calculating the deformation offset of points based on the relative positions of markers and deformation coefficients; constructing multimodal parameter time series data for each spatial marker and extracting frequency domain features for correlation analysis; constructing an early warning model, training it, and analyzing and outputting the warehouse risk probability; and determining the warehouse risk level based on the warehouse risk probability. This invention significantly improves the accuracy of spatial measurement and the sensitivity of deformation detection.

Description

Technical Field

[0001] This invention relates to the field of spatial monitoring technology, and in particular to a warehouse safety monitoring method and system based on visual inspection. Background Technology

[0002] With the continuous improvement of industrial automation and the rapid development of smart warehousing technology, vision-based warehouse monitoring systems have become one of the important means to achieve safe and intelligent management. Traditional warehouse monitoring technology mainly relies on the combination of simple video surveillance and sensors, lacking the ability to accurately measure and analyze the dynamic changes of warehouse space in real time. In recent years, with the widespread application of high-performance industrial cameras and deep learning algorithms, warehouse monitoring technology based on image processing and spatial information analysis has gradually become a research hotspot. These technologies effectively improve the accuracy of spatial positioning and the real-time performance of monitoring by collecting high-frequency image information and combining geometric calibration and multi-view fusion algorithms. Especially in scenarios such as automated warehouses, hazardous materials storage, and management of valuables, accurate spatial positioning and deformation detection are the core of ensuring safe operation. However, existing vision-based warehouse monitoring systems still have shortcomings. For problems such as material deformation, stacking offset, and spatial structure deformation inside the warehouse space, existing technologies mostly use simple image difference or two-dimensional deformation detection algorithms, which are difficult to achieve high-precision deformation analysis and early warning. Summary of the Invention

[0003] In view of the aforementioned existing problems, the present invention is proposed.

[0004] Therefore, this invention provides a warehouse safety monitoring method and system based on visual inspection, which solves the problem that existing technologies are unable to achieve high-precision deformation analysis and early warning.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] In a first aspect, the present invention provides a warehouse safety monitoring method based on visual detection, comprising,

[0007] Spatial markers are installed in the warehouse storage and high-frequency warehouse images are acquired using industrial cameras. The pixel coordinates in the warehouse images are converted into camera coordinates according to the camera parameters. A ray model is established using different camera perspectives. The ray model is optimized and spatial coordinates are output by combining multiple camera perspectives.

[0008] Structural units are defined and their areas are calculated based on spatial coordinates. Deformation parameters are analyzed and deformation coefficients are calculated based on area transformation. Deformation offset of points is calculated based on the relative positions of marked points and deformation coefficients.

[0009] Multimodal parameter time series data for each spatial marker point are constructed and frequency domain features are extracted for correlation analysis. An early warning model is constructed, trained, and analyzed to output the probability of warehousing risk. The warehousing risk level is determined based on the probability of warehousing risk.

[0010] As a preferred embodiment of the vision-based warehouse safety monitoring method of the present invention, the steps of installing spatial markers in the warehouse and using industrial cameras to acquire high-frequency warehouse images refer to dividing the warehouse into a grid, installing spatial markers at each grid point, assigning a label to each spatial marker, configuring multiple high-speed industrial cameras, unifying the acquisition time of high-frequency warehouse images by the cameras, and calibrating the internal parameters of each camera through high-precision calibration to obtain the intrinsic parameter matrix of each camera. And external references;

[0011] The high-frequency warehouse images were filtered using Gaussian filtering and then enhanced with adaptive histogram equalization to improve local image contrast.

[0012] Masking is performed on the enhanced warehouse image based on a set pixel threshold;

[0013] Perform connectivity analysis on the masked warehouse image to obtain the centroid coordinates of the target region.

[0014] As a preferred embodiment of the visual detection-based warehouse safety monitoring method of the present invention, the step of converting the pixel coordinates in the warehouse image into camera coordinates according to camera parameters and establishing a ray model using different camera perspectives, optimizing the ray model and combining multiple camera perspectives to output spatial coordinates refers to converting the centroid coordinates of the target area to the camera coordinate system according to the intrinsic parameter matrix of each camera.

[0015] In different camera viewpoints, the spatial direction vector of the same spatial marker point is calculated by matching.

[0016] A ray model corresponding to the camera's viewpoint is established based on the spatial direction vector of the camera's viewpoint and the camera's extrinsic parameters;

[0017] Each pair of cameras is paired with another camera. The intersection of the ray models of each pair is calculated to form an overdetermined system. The least squares method is then used to find the optimal spatial points of each camera pair. ;

[0018] Iterate through all camera pairs and integrate the optimal spatial points obtained from all camera pairs to obtain the final spatial position. .

[0019] As a preferred embodiment of the visual inspection-based warehouse safety monitoring method of the present invention, the following steps are taken: defining structural units based on spatial coordinates and calculating their areas; analyzing deformation parameters and calculating deformation coefficients based on area transformations; calculating the deformation offset of points based on the relative positions of marker points and deformation coefficients to obtain the final spatial coordinates of each spatial marker point; forming structural units between spatial marker points; each structural unit being a triangular face composed of three points; traversing all spatial marker points to form structural units; and measuring and recording the initial area of ​​each triangular face. Using the initial coordinates of the vertices as a reference, and based on the real-time final spatial coordinates of the spatial markers, the real-time area of ​​each structural unit is calculated using the vector cross product method. ;

[0020] The absolute value of the difference between the real-time area and the initial area is used as the deformation parameter of the structural unit. The ratio of the deformation parameter to the initial area is used as the deformation coefficient. ;

[0021] Calculate the initial center point of the structural element based on the initial coordinates of its vertices. ;

[0022] Based on the real-time final spatial coordinates of spatial markers, the relative position of each point within the structural unit to the initial center point is calculated. ;

[0023] All structural elements containing spatial markers are statistically analyzed, and the deformation offset of the spatial markers is calculated based on their relative positions and deformation coefficients. .

[0024] As a preferred embodiment of the visual detection-based warehouse safety monitoring method of the present invention, the step of constructing multimodal parameter time series data for each spatial marker point and extracting frequency domain features for correlation analysis refers to constructing multimodal parameter time series data by sequentially analyzing the real-time final spatial coordinates and spatial offsets of the spatial marker points. ;

[0025] The multimodal parameter time series data were standardized using mean-standard deviation, and frequency domain features were extracted using width Fourier transform. ;

[0026] For each spatial marker, frequency domain features are extracted, and the correlation coefficient between spatial markers is calculated using the covariance among them. ;

[0027] Spatial markers with correlation coefficients greater than a set association threshold are recorded as related point pairs. All related point pairs of spatial markers are counted to form a spatial association graph. Multimodal parameter time series data and frequency domain features are added to each spatial marker as the marker feature vector in the spatial association graph.

[0028] As a preferred embodiment of the visual detection-based warehouse safety monitoring method of the present invention, the following steps are included: constructing an early warning model, training and analyzing the output warehouse risk probability, and determining the warehouse risk level based on the warehouse risk probability. This involves using a convolutional neural network to construct the early warning model and collecting a training dataset for training the early warning model. The input of the early warning model is a spatial correlation graph, and the output is the risk probability of spatial marker points. After inputting the spatial correlation graph into the early warning model, the warehouse risk probability is obtained. The warehouse risk level is then determined based on the warehouse risk probability and a set risk threshold. The warehouse risk level includes four levels: blue, yellow, orange, and red. Different countermeasures are taken for different early warning levels.

[0029] As a preferred embodiment of the visual detection-based warehouse safety monitoring method of the present invention, the method involves taking countermeasures based on the warehouse risk level, forming a response record, and periodically reassessing the warehouse risk level.

[0030] Secondly, the present invention provides a warehouse safety monitoring system based on visual detection, comprising:

[0031] The spatial monitoring module is used to install spatial markers in warehouse storage and use industrial cameras to collect high-frequency warehouse storage images. Based on the camera parameters, the pixel coordinates in the warehouse images are converted into camera coordinates, and a ray model is established using different camera perspectives. The ray model is optimized and spatial coordinates are output by combining multiple camera perspectives.

[0032] The deformation analysis module is used to define structural elements based on spatial coordinates and calculate the area, analyze deformation parameters and calculate deformation coefficients based on area transformation, and calculate the deformation offset of points based on the relative position of marker points and deformation coefficients.

[0033] The risk assessment module is used to construct multimodal parameter time series data for each spatial marker point, extract frequency domain features for correlation analysis, construct an early warning model for training, analyze and output the probability of warehousing risk, and determine the warehousing risk level based on the probability of warehousing risk.

[0034] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein the computer program, when executed by the processor, implements any step of the warehouse security monitoring method based on visual detection as described in the first aspect of the present invention.

[0035] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the warehouse security monitoring method based on visual detection as described in the first aspect of the present invention.

[0036] The beneficial effects of this invention are as follows: By installing spatial markers and combining them with high-frequency image acquisition from industrial cameras, this invention utilizes camera parameters to transform the spatial coordinates of pixels, establishes a multi-view ray model, and optimizes model parameters to improve the accuracy of spatial positioning. Simultaneously, it constructs spatial structural units, calculates area transformation and deformation parameters in real time, and establishes a warehouse risk early warning model by combining multimodal time-series frequency domain characteristics. This enables dynamic monitoring and risk assessment of warehouse space deformation, significantly improving the accuracy of spatial measurement and the sensitivity of deformation detection, and meeting the needs of high-precision early warning and warehouse risk management. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of the warehouse safety monitoring method based on visual detection in Example 1.

[0039] Figure 2 This is a structural diagram of the warehouse safety monitoring system based on visual detection in Example 1. Detailed Implementation

[0040] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0041] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0042] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0043] Example 1, referring to Figure 1and Figure 2 This is the first embodiment of the present invention, which provides a warehouse safety monitoring method based on visual detection, including the following steps:

[0044] S1. Install spatial markers in the warehouse and use industrial cameras to collect high-frequency warehouse images. Convert the pixel coordinates in the warehouse images into camera coordinates according to the camera parameters and establish a ray model using different camera perspectives. Optimize the ray model and output spatial coordinates by combining multiple camera perspectives.

[0045] Specifically, installing spatial markers in the warehouse and using industrial cameras to acquire high-frequency warehouse images involves dividing the warehouse into a grid, installing spatial markers at each grid point, assigning a label to each spatial marker, configuring multiple high-speed industrial cameras, standardizing the acquisition time of the cameras for high-frequency warehouse images, and calibrating the internal parameters of each camera through high-precision calibration to obtain the intrinsic parameter matrix of each camera. And external references:

[0046]

[0047] in and For camera focal length, and The location of the camera's principal point. Pixel skew;

[0048] The extrinsic parameters include the rotation matrix. Translation vector ;

[0049] The acquired high-frequency warehouse images were filtered using a Gaussian filter, and adaptive histogram equalization was used to enhance local image contrast. The parameters were set to 8x8 pixel blocks, and the cropping threshold was limited to 2.0.

[0050]

[0051] in For enhanced warehouse images, This is the filtered warehouse image;

[0052] Masking is performed on the enhanced warehouse image based on a set pixel threshold:

[0053]

[0054] in pixel coordinates The mask, if The number of pixels is greater than the pixel threshold. The value is 1 if it is 1, otherwise it is 0.

[0055] Perform connectivity analysis on the masked warehouse image to obtain the centroid coordinates of the target region:

[0056]

[0057] in and Let i be the centroid coordinates of the target region i. The total number of pixels in target region i.

[0058] By precisely calibrating the intrinsic and extrinsic parameters of multiple high-speed industrial cameras, a unified transformation relationship between 3D space and pixel coordinates was established. This joint multi-camera parameter calibration method enables the accurate superposition and fusion of images captured by different cameras, overcoming spatial positioning deviations caused by parameter errors in traditional multi-camera systems. High-precision parameter calibration ensures the accuracy of subsequent spatial reconstruction, facilitating precise monitoring of stored materials and spatial conditions, and providing a reliable spatial basis for deformation detection and risk assessment. Simultaneously, it significantly reduces error propagation caused by scene changes, ensuring the robustness and stability of the monitoring system. After image acquisition, Gaussian filtering is applied to remove noise and enhance the edge features of the target area; combined with adaptive histogram equalization to improve local contrast, it is beneficial for detail capture and subsequent region analysis. This combination utilizes the dual advantages of smoothing and enhancement, effectively improving... Improving image quality directly enhances the reliability of subsequent masking operations and connected component analysis, resulting in more accurate target region segmentation and supporting finer-grained spatial deformation analysis. Simultaneously, this method effectively reduces the impact of lighting changes in the scene on detection, ensuring stable continuous monitoring of the warehouse environment. By setting a threshold, pixel binarization is quickly achieved, effectively filtering out background interference and extracting the target region. Connected component analysis accurately identifies key structures or material stacking areas in the space. This scheme combines the efficiency of threshold segmentation with the spatial consistency of connected component analysis, strengthening regional targeting. Accurate acquisition of the centroid coordinates of the target region provides key parameters for subsequent spatial positioning and target tracking. This step not only saves computational resources but also enables rapid and stable target detection in dynamically changing warehouse environments, providing accurate data support for deformation monitoring and risk assessment.

[0059] Furthermore, based on camera parameters, the pixel coordinates in the warehouse image are converted into camera coordinates, and a ray model is established using different camera perspectives. The ray model is optimized, and the spatial coordinates are output based on the multi-camera perspective. The centroid coordinates of the target area are transformed into the camera coordinate system based on the intrinsic parameter matrix of each camera.

[0060]

[0061] in These are the normalized coordinates in the k-th camera coordinate system. Let be the intrinsic parameter inverse matrix of the k-th camera;

[0062] Calculate the spatial direction vector of the same spatial marker point from different camera perspectives:

[0063]

[0064] in and Let be the spatial direction vectors of the k-th camera and the j-th camera, respectively, satisfying... , , and These are the rotation matrices for the k-th camera and the j-th camera, respectively;

[0065] Establish a ray model corresponding to the camera's viewpoint based on the spatial direction vector of the camera's viewpoint and the camera's extrinsic parameters:

[0066]

[0067] in Let be the ray model representing the viewpoint of the k-th camera. The coordinates of the spatial marker points output by the ray model. , Let be the spatial center coordinates of the k-th camera, i.e., the camera's position in the world coordinate system. For parameters, ≥0;

[0068] Each pair of cameras is paired with another camera. The intersection of the ray models of each pair is calculated to form an overdetermined system. The least squares method is then used to find the optimal spatial points of each camera pair. :

[0069]

[0070] in and The ray parameter is a variable introduced by solving the equation, representing the position of the ray in space;

[0071] By solving the above least squares problem, the optimal spatial point of the m-th camera pair can be obtained. for:

[0072]

[0073] in and It is a coefficient matrix;

[0074] The and Defined in the following way:

[0075]

[0076]

[0077] in It is a 3x3 identity matrix. This is a transpose operation;

[0078] Iterate through all camera pairs and integrate the optimal spatial points obtained from all camera pairs to obtain the final spatial position. :

[0079]

[0080] in is the weighting coefficient, and M is the number of camera pairs.

[0081] By utilizing the inverse of the intrinsic parameter matrix, pixels in the acquired warehouse images are precisely mapped to the camera's own spatial coordinate system, providing a unified projection basis for subsequent spatial positioning. This process avoids errors caused by single-pixel information and is particularly important in multi-camera fusion. It converts two-dimensional information in the image into three-dimensional spatial information, making the benchmark for subsequent multi-view fusion and spatial reconstruction more accurate. This effectively solves the problem of insufficient spatial positioning accuracy caused by traditional image processing, providing a technical foundation for high-precision monitoring of warehouse spaces. Matching the spatial direction vectors of targets detected by different cameras reflects the association of multi-view information based on spatial geometric relationships. The matching operation ensures that the same spatial target in different views is correctly matched, overcoming the matching difficulties caused by viewpoint changes. This operation enhances the geometric consistency of the spatial positional relationship of the target, improves the robustness of data fusion from different views, and significantly improves the accuracy of multi-view spatial target reconstruction, laying a solid foundation for subsequent solutions to ray intersections and spatial positioning. By establishing the corresponding ray model through the spatial direction vector and spatial center position of each camera view, the spatial information from multiple views is... This model transforms information into spatial paths, emphasizing the role of multi-angle data in spatial point reconstruction and enhancing the system's geometric robustness. It utilizes multi-view spatial information for error suppression, providing the optimal solution for precise spatial point positioning. Simultaneously, it fully leverages the geometric redundancy of different perspectives, significantly improving the reliability and accuracy of spatial point positioning. This is a crucial innovation in resolving spatial positioning ambiguity within the system. The least squares method is used to optimize the intersection points of each pair of camera rays to obtain the optimal spatial position. This is a significant innovation in using statistical knowledge to handle multi-source measurement errors. The introduction of an overdetermined system ensures that the spatial position is not only unique but also statistically stable, resolving the positioning deviation problem caused by the superposition of errors from multiple camera intersections. This ensures high-precision spatial point positioning and provides a reliable benchmark for subsequent spatial dynamic monitoring and deformation analysis. Weighting coefficients are introduced to weightedly fuse spatial points from different camera pairs, integrating multi-view spatial point information to form a more reliable spatial position. The final fused spatial point has higher accuracy and robustness, significantly improving the reliability of spatial deformation and material displacement analysis in warehouse monitoring, thereby enhancing the incidence and accuracy of safety warnings.

[0082] S2. Define structural units based on spatial coordinates and calculate the area. Analyze deformation parameters and calculate deformation coefficients based on area transformation. Calculate the deformation offset of points based on the relative positions of marker points and deformation coefficients.

[0083] Specifically, structural units are defined and their areas are calculated based on spatial coordinates. Deformation parameters are analyzed and deformation coefficients are calculated based on area transformations. The deformation offset of each point is calculated based on its relative position and deformation coefficient to obtain the final spatial coordinates of each spatial marker. These spatial markers are then grouped into structural units, each consisting of a triangular facet composed of three points. This process is repeated for all spatial markers forming structural units, and the initial area of ​​each triangular facet is measured and recorded. Using the initial coordinates of the vertices as a reference, and based on the real-time final spatial coordinates of the spatial markers, the real-time area of ​​each structural unit is calculated using the vector cross product method. :

[0084]

[0085] in , as well as The three vertices of the structural unit;

[0086] Calculating the real-time area using vector cross product ensures the accuracy and real-time nature of deformation monitoring, especially when the structural state changes rapidly. This operation fully utilizes spatial geometric properties, automates and mathematically standardizes data processing, and provides a highly sensitive indicator for detecting minute deformations. Using the difference in area change as a deformation parameter enables rapid quantification of spatial structures. Combined with threshold judgment, it can detect deformation anomalies early, improve the sensitivity and timeliness of monitoring, and provide a scientific early warning basis for storage security.

[0087] The absolute value of the difference between the real-time area and the initial area is used as the deformation parameter of the structural unit. The ratio of the deformation parameter to the initial area is used as the deformation coefficient. ;

[0088] By defining triangular faces using the coordinates of spatial points, a localized model of spatial deformation can be constructed intuitively and efficiently. This approach fully utilizes the spatial relationships between spatial points. Establishing structural units is the foundation for deformation monitoring. Using the measured initial area as a benchmark provides an absolute reference point for deviations in subsequent coordinate transformations. This not only improves the sensitivity of local deformation but also provides a reliable basis for subsequent area change analysis. It achieves spatial quantification of micro-deformation in storage space areas. It is not only suitable for monitoring overall deformation trends but also for locating severely deformed local areas, providing a scientific basis for early warning. At the same time, area, as a sensitive indicator of local deformation, is more representative than single-point offset and better reflects the overall stress changes of the structure.

[0089] Calculate the initial center point of the structural element based on the initial coordinates of its vertices. :

[0090]

[0091] in Let the initial coordinates be the i-th vertex of the structural unit;

[0092] Based on the real-time final spatial coordinates of spatial markers, the relative position of each point within the structural unit to the initial center point is calculated. :

[0093]

[0094] in The real-time final spatial coordinates of the spatial marker points;

[0095] All structural elements containing spatial markers are statistically analyzed, and the deformation offset of the spatial markers is calculated based on their relative positions and deformation coefficients. :

[0096]

[0097] Where W is the number of structural units containing spatial marker point i.

[0098] By converting the area change ratio into a deformation coefficient and fully integrating the geometric changes of the spatial structure, a unified scale parameter is provided, thereby enabling a unified assessment of the deformation degree in different spatial regions. This integrated parameterization method simplifies the deformation statistics and analysis process and enhances the consistency of deformation monitoring indicators. By utilizing the offset of a spatial point relative to the center of the structural unit, combined with the deformation coefficient, the direction and magnitude of the offset are determined, enabling dynamic tracking of the spatial position of the marker point. This operation makes spatial micro-changes quantifiable and traceable, achieving spatial positioning of spatial point deformation, going beyond the simple abstraction of point offset, and improving the richness of spatial information in spatial deformation monitoring. It not only monitors the magnitude of the offset but also analyzes the offset direction and local deformation trend, providing rich parameters for deformation early warning and risk assessment, and improving the accuracy of early warning and the depth of detail in spatial identification.

[0099] S3. Construct multimodal parameter time series data for each spatial marker point and extract frequency domain features for correlation analysis. Construct an early warning model, train it, and analyze and output the probability of warehousing risk. Determine the warehousing risk level based on the probability of warehousing risk.

[0100] Specifically, constructing multimodal parameter time series data for each spatial marker point and extracting frequency domain features for correlation analysis involves constructing multimodal parameter time series data from the real-time final spatial coordinates and spatial offsets of the spatial marker points in chronological order. :

[0101]

[0102] The real-time spatial coordinates and spatial offsets of spatial markers are serialized into multimodal parameters over time, laying the foundation for capturing dynamic deformation features. This combination of multimodal and multi-time point features makes full use of the gradually accumulated information of spatial changes, which helps to detect small and gradually evolving deformation trends, achieve a comprehensive perception of the deformation process, and provide rich dynamic information for subsequent frequency domain feature analysis. The continuous parameter sequence significantly enhances the depth of analysis, enabling the sensitive detection of small deformations or local changes, and providing a highly sensitive technical guarantee for spatial state monitoring in complex environments.

[0103] The multimodal parameter time series data were standardized using mean-standard deviation, and frequency domain features were extracted using width Fourier transform. :

[0104]

[0105] in For Fourier transform, multimodal parameter time series data are transformed. It can be taken as a continuous parameter value over a period of time;

[0106] For each spatial marker, frequency domain features are extracted, and the correlation coefficient between spatial markers is calculated using the covariance among them. :

[0107]

[0108] in Let i be the covariance of spatial markers i and j. and Let i and j be the standard deviations of the spatial markers.

[0109] Spatial markers with correlation coefficients greater than a set association threshold are recorded as related point pairs. All related point pairs of spatial markers are counted to form a spatial association graph. Multimodal parameter time series data and frequency domain features are added to each spatial marker as the marker feature vector in the spatial association graph.

[0110] By calculating the correlation coefficients of spatial marker points, the dynamic synchronization and coordination relationships between points can be revealed, which can be used to detect abnormal changes in spatial relationships. This is a significant innovation in the construction of a three-dimensional monitoring system based on point sets. This analytical method can identify the dynamic characteristics of potential spatial structural imbalances or local cracks and deformation accumulations, surpassing traditional single-point or average value analysis. It can accurately identify potential abnormal coordination or synergistic imbalances between spatial elements, reveal potential crack tendencies or local deformation accumulations in advance, and provide multi-level and multi-angle support for early warning, thereby improving the comprehensiveness of spatial monitoring and the reliability of early warning.

[0111] Furthermore, an early warning model is constructed, trained, and its output warehousing risk probability is analyzed. The warehousing risk level is determined based on this probability. This involves using a convolutional neural network to build the early warning model and collecting a training dataset for training. The input to the early warning model is a spatial correlation graph, and the output is the risk probability of spatially marked points. The spatial correlation graph is input into the early warning model to obtain the warehousing risk probability. The warehousing risk level is then determined based on this probability and a set risk threshold. The warehousing risk level includes four warning levels: blue, yellow, orange, and red. Different response measures are taken for different warning levels.

[0112] The blue alert indicates a slight deformation of the storage structure and a slight abnormal fluctuation in a single parameter, but the trend is not obvious. It is recorded and displayed on the management platform to alert maintenance personnel.

[0113] The yellow warning indicates moderate deformation of the warehouse structure, continuous deviation of a single parameter from the baseline, or slight multi-factor abnormal coupling. It automatically generates a report and notifies the operation and maintenance team leader via APP / SMS, and suggests arranging planned inspections.

[0114] The orange alert indicates that the warehouse structure is significantly deformed, key parameters are seriously exceeding the threshold, or there is strong multi-factor abnormal coupling, indicating that the structural safety is being substantially affected. The system immediately notifies the safety manager and the operation and maintenance team through SMS, telephone and other means, requiring emergency on-site investigation and handling.

[0115] The red alert indicates that the warehouse structure is severely deformed, indicating an extremely high risk of collapse or a major accident. The system will immediately activate the highest level alarm (audio and visual alarm), send emergency text messages to all relevant responsible persons, and automatically link with the warehouse control system to immediately stop the operation of stacker cranes and other equipment in the area and forcibly evacuate personnel to maximize the protection of life and property.

[0116] Furthermore, response records are generated after taking countermeasures based on the warehousing risk level, and the warehousing risk level is reassessed periodically.

[0117] This embodiment also provides a warehouse safety monitoring system based on visual detection, including:

[0118] The spatial monitoring module is used to install spatial markers in warehouse storage and use industrial cameras to collect high-frequency warehouse storage images. Based on the camera parameters, the pixel coordinates in the warehouse images are converted into camera coordinates, and a ray model is established using different camera perspectives. The ray model is optimized and spatial coordinates are output by combining multiple camera perspectives.

[0119] The deformation analysis module is used to define structural elements based on spatial coordinates and calculate the area, analyze deformation parameters and calculate deformation coefficients based on area transformation, and calculate the deformation offset of points based on the relative position of marker points and deformation coefficients.

[0120] The risk assessment module is used to construct multimodal parameter time series data for each spatial marker point, extract frequency domain features for correlation analysis, construct an early warning model for training, analyze and output the probability of warehousing risk, and determine the warehousing risk level based on the probability of warehousing risk.

[0121] This embodiment also provides a computer device applicable to the warehouse safety monitoring method based on visual inspection, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the warehouse safety monitoring method based on visual inspection as proposed in the above embodiment.

[0122] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0123] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the warehouse security monitoring method based on visual detection as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0124] In summary, this invention, by installing spatial markers and combining high-frequency image acquisition from industrial cameras, utilizes camera parameters to transform the spatial coordinates of pixels, establishes a multi-view ray model, and optimizes model parameters to improve spatial positioning accuracy. Simultaneously, it constructs spatial structural units, calculates area transformation and deformation parameters in real time, and combines multimodal time-series frequency domain characteristics to establish a warehouse risk early warning model. This enables dynamic monitoring and risk assessment of warehouse space deformation, significantly improving spatial measurement accuracy and deformation detection sensitivity, thus meeting the needs of high-precision early warning and warehouse risk management.

[0125] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A warehouse security monitoring method based on visual detection, characterized in that: include, Spatial markers are installed in the warehouse storage and high-frequency warehouse images are acquired using industrial cameras. The pixel coordinates in the warehouse images are converted into camera coordinates according to the camera parameters. A ray model is established using different camera perspectives. The ray model is optimized and spatial coordinates are output by combining multiple camera perspectives. Structural units are defined and their areas are calculated based on spatial coordinates. Deformation parameters are analyzed and deformation coefficients are calculated based on area transformation. Deformation offset of points is calculated based on the relative positions of marked points and deformation coefficients. Construct multimodal parameter time series data for each spatial marker point and extract frequency domain features for correlation analysis. Build an early warning model, train it, and analyze the output warehouse risk probability. Determine the warehouse risk level based on the warehouse risk probability. The process involves converting pixel coordinates in the warehouse image into camera coordinates based on camera parameters, establishing a ray model using different camera perspectives, optimizing the ray model, and combining multiple camera perspectives to output spatial coordinates. This process involves converting the centroid coordinates of the target area to the camera coordinate system based on the intrinsic parameter matrix of each camera. In different camera viewpoints, the spatial direction vector of the same spatial marker point is calculated by matching. A ray model corresponding to the camera's viewpoint is established based on the spatial direction vector of the camera's viewpoint and the camera's extrinsic parameters; Each two cameras form a camera pair, the ray models of each pair of cameras are intersected to form an over-determined system, and the least square method is used to solve the optimal space point of the camera pair ; traversing all camera pairs, integrating the optimal spatial points obtained by all camera pairs to obtain a final spatial position ; The process of defining structural units based on spatial coordinates and calculating their areas, analyzing deformation parameters and calculating deformation coefficients based on area transformations, and calculating the deformation offset of points based on the relative positions of marker points and deformation coefficients specifically involves: obtaining the final spatial coordinates of each spatial marker point, forming structural units between the spatial marker points, with each structural unit being a triangular face composed of three points, traversing all spatial marker points to form structural units, and measuring and recording the initial area of ​​each triangular face. Using the initial coordinates of the vertices as a reference, and based on the real-time final spatial coordinates of the spatial markers, the real-time area of ​​each structural unit is calculated using the vector cross product method. ; The absolute value of the difference between the real-time area and the initial area is used as the deformation parameter of the structural unit. The ratio of the deformation parameter to the initial area is used as the deformation coefficient. ; Calculate the initial center point of the structural element based on the initial coordinates of its vertices. ; Based on the real-time final spatial coordinates of spatial markers, the relative position of each point within the structural unit to the initial center point is calculated. ; All structural elements containing spatial markers are statistically analyzed, and the deformation offset of the spatial markers is calculated based on their relative positions and deformation coefficients. ; The specific steps for constructing multimodal parameter time series data for each spatial marker point and extracting frequency domain features for correlation analysis are as follows: Multimodal parameter time series data are constructed by sequentially analyzing the real-time final spatial coordinates and spatial offsets of the spatial marker points. ; The multimodal parameter time series data were standardized using mean-standard deviation, and frequency domain features were extracted using width Fourier transform. ; For each spatial marker, frequency domain features are extracted, and the correlation coefficient between spatial markers is calculated using the covariance among them. ; Spatial markers with correlation coefficients greater than a set association threshold are recorded as related point pairs. All related point pairs of spatial markers are counted to form a spatial association graph. Multimodal parameter time series data and frequency domain features are added to each spatial marker as the marker feature vector in the spatial association graph.

2. The warehouse safety monitoring method based on visual detection as described in claim 1, characterized in that: The process of installing spatial markers in the warehouse and using industrial cameras to acquire high-frequency warehouse images involves dividing the warehouse into a grid, installing spatial markers at each grid point, assigning a label to each spatial marker, configuring multiple high-speed industrial cameras, standardizing the acquisition time of the cameras for high-frequency warehouse images, and calibrating the internal parameters of each camera through high-precision calibration to obtain the intrinsic parameter matrix of each camera. And external references; The high-frequency warehouse images were filtered using Gaussian filtering and then enhanced with adaptive histogram equalization to improve local image contrast. Masking is performed on the enhanced warehouse image based on a set pixel threshold; Perform connectivity analysis on the masked warehouse image to obtain the centroid coordinates of the target region.

3. The warehouse safety monitoring method based on visual detection as described in claim 2, characterized in that: The process of constructing an early warning model, training it, and analyzing the output of the warehouse risk probability, and then determining the warehouse risk level based on the warehouse risk probability, involves using a convolutional neural network to construct the early warning model and collecting a training dataset for training. The input of the early warning model is a spatial correlation graph, and the output is the risk probability of spatially marked points. After inputting the spatial correlation graph into the early warning model, the warehouse risk probability is obtained, and the warehouse risk level is determined based on the warehouse risk probability and a set risk threshold. The warehouse risk level includes four levels: blue, yellow, orange, and red. Different response measures are taken for different early warning levels.

4. The warehouse safety monitoring method based on visual detection as described in claim 3, characterized in that: After determining the risk level of warehousing, take corresponding measures and create a response record, and reassess the risk level of warehousing periodically.

5. A warehouse safety monitoring system based on visual inspection, based on the warehouse safety monitoring method based on visual inspection as described in any one of claims 1 to 4, characterized in that: include, The spatial monitoring module is used to install spatial markers in warehouse storage and use industrial cameras to collect high-frequency warehouse storage images. Based on the camera parameters, the pixel coordinates in the warehouse images are converted into camera coordinates, and a ray model is established using different camera perspectives. The ray model is optimized and spatial coordinates are output by combining multiple camera perspectives. The deformation analysis module is used to define structural elements based on spatial coordinates and calculate the area, analyze deformation parameters and calculate deformation coefficients based on area transformation, and calculate the deformation offset of points based on the relative position of marker points and deformation coefficients. The risk assessment module is used to construct multimodal parameter time series data for each spatial marker point, extract frequency domain features for correlation analysis, construct an early warning model for training, analyze and output the probability of warehousing risk, and determine the warehousing risk level based on the probability of warehousing risk.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the warehouse safety monitoring method based on visual detection as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the warehouse safety monitoring method based on visual detection as described in any one of claims 1 to 4.

Images