A state detection system for an archival storage device

Abstract

This invention relates to a status detection system for an archival storage device, comprising: one or more image acquisition devices mounted above the archival storage device for acquiring a top view of the archival storage device; an image processing unit for preprocessing the top view of the archival storage device acquired by the image acquisition device to obtain an image to be detected; and an archival storage device opening status detection unit for obtaining the opening status of the archival storage device based on the image to be detected obtained by the image processing unit and a trained archival storage device opening position detection model.

Description

Technical Field

[0001] This invention relates to the field of archival storage device opening detection technology, and more particularly to an archival storage device status detection system. Background Technology

[0002] Archive storage devices have advantages such as compact structure, small space occupation, and large material storage capacity, and are widely used in warehouses across various industries as important material storage equipment. However, during the operation of archive storage devices, problems such as large cabinet area, unfixed opening positions, and obstruction caused by the height of the cabinets make it difficult to conduct real-time video monitoring of the opening positions of archive storage devices, making it difficult to achieve full-process monitoring of personnel behavior in the warehouse.

[0003] To address this issue, in recent years, solutions have emerged that utilize movable guide rails to drive a monitoring camera with a small field of view directly above the opening. The core technology enabling this solution is the real-time detection of the opening position of the file storage device. Obtaining high-precision opening position while controlling hardware costs is crucial for successful engineering implementation.

[0004] Existing methods for detecting the open status of archival storage devices primarily rely on magnetic induction sensors, ultrasonic sensors, infrared sensors, or mechanical sensors. While these methods can detect the open status with high accuracy, they require the installation of sensors and wiring on each cabinet of the storage device, resulting in high costs and difficulties in maintenance. Furthermore, the non-fixed position of the archival storage device cabinets, which can move along guide rails, increases the difficulty of wiring installation and significantly impacts the lifespan of the wiring. Specifically, the following problems exist:

[0005] 1. Each file storage unit cabinet needs to be equipped with a sensor, which is not only costly but also increases the probability of malfunctions; 2. Each sensor also needs to be wired. Since the file storage unit cabinet moves along the guide rail, the wiring is not only difficult to install, but the frequent stretching and bending will also seriously affect the service life of the wires. Summary of the Invention

[0006] Based on the above analysis, the present invention aims to provide a status detection system for an archive storage device to solve the problems of high hardware cost and poor maintainability of existing archive storage device opening status detection systems.

[0007] On one hand, embodiments of the present invention provide a status detection system for an archive storage device, comprising:

[0008] One or more image acquisition devices are installed above the archive storage device for acquiring a top view of the archive storage device;

[0009] The image processing unit is used to preprocess the top view of the file storage device acquired by the image acquisition device to obtain the image to be detected;

[0010] The document storage device opening state detection unit is used to obtain the opening state of the document storage device based on the image to be detected obtained by the image processing unit and a trained document storage device opening position detection model.

[0011] Based on a further improvement of the above technical solution, the image processing unit preprocesses the top view of the file storage device acquired by the image acquisition device to obtain the image to be detected, including:

[0012] In the top view of the archive storage device, multiple images with a width of W and a length of L are extracted at equal intervals in a direction perpendicular to the moving direction of the archive storage device as images to be detected, wherein the long side of each image to be detected is parallel to the moving direction of the archive storage device.

[0013] Furthermore, the image processing unit also includes an image stitching unit; the image stitching unit is used to stitch together the images acquired by each image acquisition device to obtain a complete top view of the archive storage device when there is more than one image acquisition device.

[0014] Furthermore, the image stitching unit uses the following steps to stitch together the images acquired by each image acquisition device to obtain a complete top view of the archive storage device:

[0015] The SIFT algorithm is used to extract feature points from each image;

[0016] The fast nearest neighbor algorithm is used to match feature points between two adjacent images to obtain the initial correspondence between feature points in adjacent images.

[0017] The random sampling consistency algorithm is used to remove erroneous correspondences, and the homography matrix between adjacent images is calculated;

[0018] The stitched image is obtained by mapping different images onto the same image based on the homography matrix between adjacent images.

[0019] Furthermore, the file storage device opening location detection model is an improved ShuffleNet model, which includes:

[0020] The first convolutional layer is used to extract shallow features from the input data and output a shallow feature map. The first convolutional layer is a one-dimensional convolutional layer.

[0021] Max pooling layers are used to reduce the dimensionality of shallow feature maps;

[0022] Multiple ShuffleNet modules are used to extract deep semantic features from the dimensionality-reduced shallow feature maps by employing pointwise group convolution and channel shuffling.

[0023] The second convolutional layer is used to reduce the channel dimension of deep semantic features and output the detection results.

[0024] Furthermore, the loss of the file storage device opening location detection model is calculated using the following formula:

[0025]

[0026]

[0027] Loss=Point_Loss+Confidence_Loss

[0028] Where S represents the length of the features output by the model; k i This represents the position of the edge point within the i-th interval predicted by the model. c represents the actual edge point position within the i-th interval; i This represents the confidence level that there are edge points in the i-th interval; This indicates whether there is actually a target point in the i-th interval; the value is 1 if there is, and 0 otherwise. λ obj , λ noobj and λ point This represents the weighting coefficient.

[0029] Furthermore, the file storage device opening status detection unit includes an edge point detection unit and an opening position calculation unit;

[0030] The edge point detection unit is used to input each image to be detected into the file storage device opening position detection model to obtain the edge points at the opening of the file storage device in each image to be detected.

[0031] The opening position calculation unit is used to calculate the opening position of the archive storage device based on the edge points at the opening of the archive storage device in each image to be detected.

[0032] Furthermore, the opening position calculation unit calculates the opening position of the file storage device in the following manner, including:

[0033] Based on coordinate projection transformation, calculate the coordinates of the bottom of the cabinet corresponding to the edge point of the opening of the archive storage device in each image to be detected;

[0034] Based on the standard calibration board, the camera is calibrated on the ground plane to obtain the camera's internal parameters, external parameters, and distortion coefficients; based on the camera's internal parameters, external parameters, and distortion coefficients, the coordinates of the bottom of the cabinet at the opening of the archive storage device are converted into actual position data.

[0035] Furthermore, the coordinates (x, y) of the bottom of the cabinet at the opening of the file storage device are calculated using the following formula. * ,y * ):

[0036]

[0037] Where (x,y) are the coordinates of the edge point of the opening of the file storage device to be detected, (x * ,y * (x) represents the coordinates of the bottom of the cabinet at the opening of the file storage device. c ,y c ) represents the coordinates of the image center point, H represents the camera mounting height, and h represents the height of the file storage cabinet.

[0038] Furthermore, the coordinates of the bottom of the cabinet at the opening of the archival storage device are converted into actual position data using the following formula based on the camera's internal parameters, external parameters, and distortion coefficients:

[0039] According to the formula: Convert the pixel coordinates of the edge points to image coordinates (x, y, z); where (u, v) represents the pixel coordinates of the edge points, dx and dy represent the actual physical size of the pixels; and (u0, v0) represents the pixel coordinates of the image center point.

[0040] According to the formula Calculate the camera coordinates (X) of the edge points c ,Y c Z c ); where f represents the camera focal length;

[0041] According to the formula Calculate the world coordinates (X) of the edge point w ,Y w Z w ), where R represents the rotation matrix and T represents the translation matrix.

[0042] Compared with existing technologies, this invention uses a low-cost area-scan industrial camera to acquire a top view of the archival storage device, and achieves high-precision detection of the opening status of the archival storage device through a trained archival storage device opening position detection model. While achieving detection results comparable to traditional methods using magnetic induction sensors, ultrasonic sensors, infrared sensors, or mechanical sensors, it features low cost and simple maintenance. This invention can accurately identify the position of the opening of the archival storage device, and uses a lightweight convolutional neural network for real-time detection. It has no special requirements for computer configuration and does not increase computer hardware costs, making it of great application value.

[0043] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0044] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0045] Figure 1 This is a block diagram of the status detection system of the file storage device according to an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of an image acquisition scene according to an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of sample image extraction according to an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of the lightweight convolutional neural network model structure according to an embodiment of the present invention. Detailed Implementation

[0049] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0050] One specific embodiment of the present invention discloses a status detection system for an archive storage device, such as... Figure 1 As shown, it includes:

[0051] One or more image acquisition devices are installed above the archive storage device for acquiring a top view of the archive storage device;

[0052] The image processing unit is used to preprocess the top view of the file storage device acquired by the image acquisition device to obtain the image to be detected;

[0053] The document storage device opening state detection unit is used to obtain the opening state of the document storage device based on the image to be detected obtained by the image processing unit and a trained document storage device opening position detection model.

[0054] During implementation, an industrial area scan camera is installed above the archive storage device to obtain a top view of the archive storage device.

[0055] This application uses a low-cost area-scan industrial camera to acquire a top view of an archival storage device. A trained model for detecting the opening position of the archival storage device achieves high-precision detection of the opening's state. While achieving comparable detection results to traditional methods using magnetic induction sensors, ultrasonic sensors, infrared sensors, or mechanical sensors, it offers the advantages of low cost and simple maintenance. This invention can accurately identify the location of the opening in an archival storage device. Furthermore, by using a lightweight convolutional neural network, it enables real-time detection without special computer configuration requirements or additional hardware costs, making it highly valuable for application.

[0056] Specifically, the image processing unit preprocesses the top view of the file storage device acquired by the image acquisition device to obtain the image to be detected, including:

[0057] In the top view of the archive storage device, multiple images with a width of W and a length of L are extracted at equal intervals in a direction perpendicular to the moving direction of the archive storage device as images to be detected, wherein the long side of each image to be detected is parallel to the moving direction of the archive storage device.

[0058] In practice, the image width can be 1 pixel, and the sample image is an L×1 image, such as... Figure 3 As shown in the figure, each line represents an L×1 dimensional sample image, and the interval between the lines is fixed.

[0059] To facilitate calculations, when installing an industrial area scan camera, one of the camera's coordinate axes can be aligned parallel to the direction of movement of the archive storage device, thereby facilitating the extraction of sample images.

[0060] When implementing, such as Figure 2 As shown, if the shooting range of one industrial area scan camera is insufficient to cover the area of ​​an archive storage device, multiple industrial area scan cameras can be used to capture a top view of the archive storage device. Therefore, the image processing unit also includes an image stitching unit. The image stitching unit is used to stitch together the images captured by each image acquisition device to obtain a complete top view of the archive storage device when there is more than one image acquisition device.

[0061] The image stitching unit uses the following steps to stitch together the images acquired by each image acquisition device to obtain a complete top view of the archive storage device:

[0062] S21. Use the SIFT algorithm to extract feature points from each image;

[0063] The SIFT algorithm describes the detected feature points in an image using a 128-dimensional feature vector. Therefore, after an image is processed by the SIFT algorithm, each feature point is represented as a 128-dimensional feature description vector.

[0064] S22. Use the fast nearest neighbor algorithm to match feature points between two adjacent images to obtain the initial correspondence between feature points in adjacent images;

[0065] After identifying feature points, matching feature points not only requires the closest distance between their feature description vectors but also the ability to be distinguished from other feature points. Therefore, the ratio of the distance between the nearest neighbor and the second nearest neighbor is used to constrain feature point matching. Two images are stitched together as an example, referred to as the first image and the second image, respectively.

[0066] First, a KD-tree data structure is used to store the feature description vectors in an ordered manner. Using the feature points in the first image as a reference, the Fast Approximate k-Nearest Neighbor (FLANN) algorithm is then used to find the nearest and second nearest neighbors in the second image. If the ratio of the distance between the nearest and second nearest neighbors is less than a threshold, the matching relationship between the feature point and its nearest neighbor is preserved.

[0067] S23. Use the random sampling consistency algorithm to delete erroneous correspondences and calculate the homography matrix between adjacent images;

[0068] Specifically, it includes:

[0069] S231. Randomly select n sets of matching points from the set S to form a subset s, where s is the assumed interior point, and n is at least 4. (Since the homography matrix has 8 unknown parameters, 8 linear equations need to be solved. Corresponding to the point position information, a pair of points can be listed as two equations, that is, they satisfy the same mapping relationship in the x and y coordinates, so there are at least 4 pairs of matching points.)

[0070] S232. Calculate the homography matrix from the feature points in the first image to the corresponding feature points in the second image based on the feature points in the subset s.

[0071] S233. Traverse all feature point pairs in dataset S except for subset s. For each feature point pair, if the error between the mapping point calculated by the feature point of the first image according to the transformation matrix and the feature point of the second image is within the given error e, then mark it as an interior point; otherwise, mark it as an exterior point.

[0072] S234. All interior points form a uniform set. If the number of points in the uniform set satisfies a given threshold TT, then the homography matrix is ​​recalculated using all interior points in the uniform set, and then the algorithm ends.

[0073] S235. If the number of interior points in the consistent set is less than the threshold TT, then select a new subset s and return to step 231.

[0074] If the number of points in the K uniform sets obtained after K iterations does not meet the given threshold, then select a uniform set with the largest number of interior points, recalculate the homography matrix using all interior points in that uniform set, and then terminate the algorithm.

[0075] By employing a random sampling consistency algorithm, outliers are eliminated, and the homography matrix is ​​calculated based solely on inliers, thereby removing erroneous correspondences and making feature point matching more accurate.

[0076] Among them, homography matrix according to Calculate the homography matrix H. Here, (x,y) represents the feature points in the first image of the feature point pair, and (x',y') represents the feature points in the second image of the feature point pair.

[0077] S24. Based on the homography matrix between adjacent images, different images are mapped onto the same image to obtain the stitched image.

[0078] The homography matrix between images is estimated using a sample consistency algorithm, and all images are then warped onto a common image plane. Typically, this common plane is the central image plane. One approach is to create a very large image, such as one filled entirely with zeros, parallel to the central image, and then warp all the images onto it to obtain the stitched image.

[0079] Specifically, the file storage device opening status detection unit includes an edge point detection unit and an opening position calculation unit;

[0080] The edge point detection unit is used to input each image to be detected into the file storage device opening position detection model to obtain the edge points at the opening of the file storage device in each image to be detected.

[0081] The opening position calculation unit is used to calculate the opening position of the archive storage device based on the edge points at the opening of the archive storage device in each image to be detected.

[0082] After obtaining the image to be detected, the edge point detection unit inputs the image to be detected into the trained file storage device opening position detection model to obtain the coordinates of the opening edge points of the file storage device.

[0083] Specifically, the trained file storage device opening location detection model is obtained through the following methods:

[0084] S31. Obtain top views of the archive storage device under different opening states. For each top view of the archive storage device, extract multiple sample images and obtain the edge points of the archive storage device opening marked in each sample image. Combine the sample images and the corresponding edge points to form a training dataset.

[0085] In practice, the process of extracting sample images can refer to the process of obtaining the image to be detected by the aforementioned image processing unit, and will not be repeated here.

[0086] After extracting the sample images, the opening positions of the archive storage devices in the images can be manually labeled. Since the top view is obtained, the top edge points of the adjacent cabinets at the opening of the archive storage devices are labeled, and the coordinate positions are used as the labels corresponding to the sample images. The sample images and the corresponding labels constitute the training dataset.

[0087] S32. Construct a lightweight convolutional neural network model, train the model based on the training dataset, and obtain a trained file storage device opening position detection model.

[0088] To facilitate real-time detection and improve detection efficiency, a lightweight convolutional neural network model was constructed as a model for detecting the opening position of the archive storage device.

[0089] Specifically, the lightweight convolutional neural network model is an improved ShuffleNet model, such as... Figure 4 As shown in (b), the improved ShuffleNet model includes:

[0090] The first convolutional layer is used to extract shallow features from the input data and output a shallow feature map. The first convolutional layer is a one-dimensional convolutional layer.

[0091] Max pooling layers are used to reduce the dimensionality of shallow feature maps;

[0092] Multiple ShuffleNet modules are used to extract deep semantic features from the dimensionality-reduced shallow feature maps by employing pointwise group convolution and channel shuffling.

[0093] The second convolutional layer is used to reduce the channel dimension of deep semantic features and output the detection results.

[0094] Specifically, the first convolutional layer is a one-dimensional convolutional layer with a stride of 2 and 24 kernels. It is used to extract shallow features from the input data, reduce the data dimensionality, and increase the number of feature channels.

[0095] For example, the input sample image is a 4000×1×3 image, where the third dimension 3 represents the number of channels. The feature map obtained after the first convolutional layer is 2000×1×24, and the output feature map after the pooling layer is 1000×1×24.

[0096] The dimensionality-reduced feature maps are processed through multiple ShuffleNet modules to extract deep semantic features. Each module further reduces the feature map dimensionality while increasing the number of feature channels, gradually extracting deep semantic features from edge points. ShuffleNet employs pointwise group convolution and channel shuffling, effectively reducing the computational load of the network and significantly lowering computational costs while ensuring minimal loss of accuracy.

[0097] Specifically, in order to enhance the feature extraction capability of the network, each ShuffleNet module includes a one-dimensional ShuffleNet unit and a channel attention module connected in sequence; the convolutional layer of the one-dimensional ShuffleNet unit is a one-dimensional convolution.

[0098] That is, all convolutional layers in a ShuffleNet unit are replaced with one-dimensional convolutions, and a channel attention module is added after each ShuffleNet unit structure to further extract more accurate features. For example... Figure 4 As shown in (a), the channel attention module consists of global average pooling, two fully connected layers, one ReLU layer, and one sigmoid activation layer. It generates corresponding weights for each channel, enabling the network to autonomously select based on the importance of each feature channel. The features output by the ShuffleNet unit are multiplied by the weights calculated by the channel attention module to obtain the features with added attention weights.

[0099] The improved ShuffleNet model in this application removes the global average pooling and fully connected layers at the end of the original ShuffleNet model and replaces them with a single convolutional layer, namely the second convolutional layer. This convolutional layer has a kernel size of 1×1, two kernels, and a stride of 1. For example, if the input data is (125×1×1536), the output data is (125×1×2).

[0100] It's important to note that the lightweight convolutional neural network here outputs a feature dimension of (125×1×2), producing two feature maps, each 125×1 in size. This means the one-dimensional input vector is divided into 125 equal intervals, each containing two values: one representing the predicted edge location and the other representing the confidence score of the presence of an edge within that interval. In other words, the first element in each of the two feature maps represents the confidence score of the presence of an edge within the first interval of the input data and the coordinates of the predicted edge. Furthermore, since the input data is a one-dimensional vector, only one coordinate value is needed to represent the edge location.

[0101] After building a lightweight convolutional neural network model, the model is trained based on training sample data. During the network training process, the gradient descent method is used to backpropagate and update the parameters to optimize the model parameters, thereby obtaining a trained file storage device opening position detection model.

[0102] Specifically, the loss of the lightweight convolutional neural network model is calculated using the following formula, and the model parameters are updated accordingly:

[0103]

[0104]

[0105] Loss=Point_Loss+Confidence_Loss

[0106] Where S represents the length of the features output by the model; k i This represents the position of the edge point within the i-th interval predicted by the model. c represents the actual edge point position within the i-th interval; i This represents the confidence level that there are edge points in the i-th interval; This indicates whether there is actually a target point in the i-th interval; the value is 1 if there is, and 0 otherwise. λ obj , λ noobj and λ point This represents the weighting coefficient.

[0107] Specifically, It is determined based on the coordinates of the edge points in the label. For example, if the edge point coordinates of the sample are at the 50th pixel position, and the first interval is from the 1st to the 30th pixel position, then there is no target point in the first interval.

[0108] Once the model reaches the required accuracy or the predetermined number of iterations, training stops, and the trained file storage device opening location detection model is obtained.

[0109] After obtaining the trained model, the archive storage device opening state detection unit inputs each image to be detected into the trained archive storage device opening position detection model to obtain the coordinates of the edge points of the archive storage device in each image to be detected.

[0110] After obtaining the coordinates of the edge points of the archive storage device, the opening position calculation unit calculates the opening position of the archive storage device based on the edge points of the opening of the archive storage device in each image to be detected.

[0111] During implementation, since the edge point of the archive storage device obtained from the top view is the top edge point of the adjacent cabinet at the opening of the archive storage device, the opening position calculation unit needs to convert it into the coordinates of the bottom of the cabinet, so as to facilitate the accurate calculation of the specific data of the opening position of the archive storage device in the world coordinate system.

[0112] Specifically, the opening position calculation unit calculates the opening position of the file storage device in the following way:

[0113] S41. Calculate the coordinates of the bottom of the cabinet corresponding to each edge point in each image to be detected based on the coordinate projection transformation;

[0114] Specifically, the coordinates (x, y) of the bottom of the cabinet at the opening of the file storage device are calculated using the following formula. * ,y * ):

[0115]

[0116] Where (x,y) are the coordinates of the edge point at the opening of the archive storage device in the image to be detected, (x * ,y * (x) represents the coordinates of the bottom of the cabinet at the opening of the file storage device. c ,y c ) represents the coordinates of the image center point, H represents the camera mounting height, and h represents the height of the file storage cabinet.

[0117] It should be noted that the coordinates here are the coordinates of the edge point of the opening of the file storage device in the image to be detected in the top view of the file storage device, and the coordinate system is the pixel coordinate system.

[0118] S42. Based on the standard calibration plate, calibrate the camera on the ground plane to obtain the camera's internal parameters, external parameters, and distortion coefficients; convert the coordinates of the bottom of the cabinet at the opening of the archive storage device into actual position data according to the camera's internal parameters, external parameters, and distortion coefficients.

[0119] In practice, the Zhang Zhengyou calibration method can be used to calibrate the camera on the ground plane based on a standard calibration plate to obtain the camera's internal parameters, external parameters, and distortion coefficients.

[0120] The following formula is used to convert the coordinates of the bottom of the cabinet at the opening of the archival storage device into actual position data based on the camera's internal parameters, external parameters, and distortion coefficients:

[0121] According to the formula: Convert the pixel coordinates of the edge points to image coordinates (x, y, z); where (u, v) represents the pixel coordinates of the edge points, dx and dy represent the actual physical size of the pixels; and (u0, v0) represents the pixel coordinates of the image center point.

[0122] According to the formula Calculate the camera coordinates (X) of the edge points c ,Y c Z c ); where f represents the camera focal length;

[0123] According to the formula Calculate the world coordinates (X) of the edge point w ,Y w Z w ), where R represents the rotation matrix and T represents the translation matrix.

[0124] Where dx, dy, (u0,v0) and f are the camera's internal parameters; R and T are the camera's external parameters.

[0125] Because distortions caused by the optical lens, including radial and tangential distortion, exist during the camera's imaging process, distortion correction is necessary. This distortion correction is performed in the image coordinate system.

[0126] Radial distortion correction is performed on the position data:

[0127]

[0128]

[0129] Tangential distortion correction is performed on the position data:

[0130]

[0131]

[0132] Where k1, k2, k3, p1, and p2 are distortion coefficients, and (x, y) represents the uncorrected coordinates in the image coordinate system. The coordinates are the corrected coordinates in the image coordinate system, and r represents the distance of the coordinate point (x, y) from the center point of the image. The farther away from the center point, the greater the distortion.

[0133] After the opening position calculation unit obtains the coordinates of all edge points in the world coordinate system, it connects the edge points on the same side of the opening to obtain the edge lines on both sides. Based on the two edge lines, the actual opening position of the archive storage device can be accurately obtained, that is, the opening state of the archive storage device.

[0134] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0135] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A status detection system for an archive storage device, characterized in that, include: One or more image acquisition devices are installed above the archive storage device for acquiring a top view of the archive storage device; The image processing unit is used to preprocess the top view of the file storage device acquired by the image acquisition device to obtain the image to be detected; The document storage device opening state detection unit is used to obtain the opening state of the document storage device based on the image to be detected obtained by the image processing unit and a trained document storage device opening position detection model. The file storage device opening location detection model is an improved ShuffleNet model, which includes: The first convolutional layer is used to extract shallow features from the input data and output a shallow feature map. The first convolutional layer is a one-dimensional convolutional layer. Max pooling layers are used to reduce the dimensionality of shallow feature maps; Multiple ShuffleNet modules are used to extract deep semantic features from the dimensionality-reduced shallow feature maps by employing pointwise group convolution and channel shuffling. The second convolutional layer is used to reduce the channel dimension of deep semantic features and output the detection results.

2. The status detection system for the archive storage device according to claim 1, characterized in that, The image processing unit preprocesses the top view of the file storage device acquired by the image acquisition device to obtain the image to be detected, including: In the top view of the archive storage device, multiple images with a width of W and a length of L are extracted at equal intervals in a direction perpendicular to the moving direction of the archive storage device as images to be detected, wherein the long side of each image to be detected is parallel to the moving direction of the archive storage device.

3. The status detection system for the archive storage device according to claim 1, characterized in that, The image processing unit further includes an image stitching unit; the image stitching unit is used to stitch together the images acquired by each image acquisition device to obtain a complete top view of the archive storage device when there is more than one image acquisition device.

4. The status detection system for the archive storage device according to claim 3, characterized in that, The image stitching unit uses the following steps to stitch together the images acquired by each image acquisition device to obtain a complete top view of the archive storage device: The SIFT algorithm is used to extract feature points from each image; The fast nearest neighbor algorithm is used to match feature points between two adjacent images to obtain the initial correspondence between feature points in adjacent images. The random sampling consistency algorithm is used to remove erroneous correspondences, and the homography matrix between adjacent images is calculated; The stitched image is obtained by mapping different images onto the same image based on the homography matrix between adjacent images.

5. The status detection system for the archive storage device according to claim 1, characterized in that, The loss of the file storage device opening location detection model is calculated using the following formula: Loss=Point_Loss+Confidence_Loss Where S represents the length of the features output by the model; k i This represents the position of the edge point within the i-th interval predicted by the model. c represents the actual edge point position within the i-th interval; i This represents the confidence level that there are edge points in the i-th interval; This indicates whether there is actually a target point in the i-th interval; the value is 1 if there is, and 0 otherwise. λ obj , λ noobj and λ point This represents the weighting coefficient.

6. The status detection system for the archive storage device according to claim 1, characterized in that, The file storage device opening status detection unit includes an edge point detection unit and an opening position calculation unit; The edge point detection unit is used to input each image to be detected into the file storage device opening position detection model to obtain the edge points at the opening of the file storage device in each image to be detected. The opening position calculation unit is used to calculate the opening position of the archive storage device based on the edge points at the opening of the archive storage device in each image to be detected.

7. The status detection system for the archive storage device according to claim 6, characterized in that, The opening position calculation unit calculates the opening position of the file storage device in the following ways: Based on coordinate projection transformation, calculate the coordinates of the bottom of the cabinet corresponding to the edge point of the opening of the archive storage device in each image to be detected; Based on the standard calibration board, the camera is calibrated on the ground plane to obtain the camera's internal parameters, external parameters, and distortion coefficients; based on the camera's internal parameters, external parameters, and distortion coefficients, the coordinates of the bottom of the cabinet at the opening of the archive storage device are converted into actual position data.

8. The status detection system for the archive storage device according to claim 7, characterized in that, Calculate the coordinates (x) of the bottom of the cabinet at the opening of the file storage device using the following formula. * ,y * ): Where (x,y) are the coordinates of the edge point of the opening of the file storage device to be detected, (x * ,y * (x) represents the coordinates of the bottom of the cabinet at the opening of the file storage device. c ,y c ) represents the coordinates of the image center point, H represents the camera mounting height, and h represents the height of the file storage cabinet.

9. The status detection system for the archive storage device according to claim 7, characterized in that, The following formula is used to convert the coordinates of the bottom of the cabinet at the opening of the archival storage device into actual position data based on the camera's internal parameters, external parameters, and distortion coefficients: According to the formula: Convert the pixel coordinates of the edge points to image coordinates (x, y, z); Where (u,v) represents the pixel coordinates of the edge point, dx and dy represent the actual physical size of the pixel; and (u0,v0) represents the pixel coordinates of the center point of the image. According to the formula Calculate the camera coordinates (X) of the edge points c ,Y c Z c ); where f represents the camera focal length; According to the formula Calculate the world coordinates (X) of the edge point w ,Y w Z w ), where R represents the rotation matrix and T represents the translation matrix.

Images