Digital intelligent archive management method and system based on machine vision

Abstract

The invention relates to a digital intelligent archive management method and system based on machine vision, different storage periods are divided by detecting a cabinet door of an archive cabinet, and when the cabinet door is opened, a camera automatically collects an inclined video in the cabinet and extracts a target frame representing operation completion from the inclined video before the cabinet door is closed. And analyzing the blocks corresponding to all the storage grids in the target frame in combination with the pressure data in the cabinet. Rough detection based on gray feature analysis is adopted for the image blocks without pressure change, and fine detection based on gray features, geometric features and character recognition is adopted for the image blocks with pressure change. Therefore, the detection efficiency and the detection accuracy are ensured. And after the detection is completed, the position distribution information of all the file pockets is updated, and in combination with the position distribution information of the previous storage period, the state updating of the file pockets can be efficiently completed, so that the automatic information management is realized. The method has the advantages of efficient and accurate management and high automation degree.

Description

Technical Field

[0001] The present invention relates to the technical field of archive management, and in particular to a digital intelligent archive management method and system based on machine vision. Background Art

[0002] Archives are an important carrier for recording human activities and social changes. Proper management of archives helps to protect historical memory and ensure that future generations can understand and learn from past experiences and lessons. For enterprises, archives provide valuable historical data and information resources, which can serve as a basis for formulating current and future strategies and decisions, and improve the quality of decision-making.

[0003] In the prior art, although digital technology is gradually being used to manage archives, the original source of archive information, that is, the archive itself, is still recorded in a manual management manner. Therefore, for the archive itself, if there is a change in status, it can only rely on personnel to record its change information, which is prone to information omissions, resulting in information errors or even loss of archives. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a digital intelligent file management method and system based on machine vision to solve the above technical problems.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A digital intelligent archive management method based on machine vision of the present invention comprises the following steps: Obtaining the location distribution information of all the file bags in the filing cabinet in the previous storage period, and obtaining the door status of the filing cabinet, wherein the storage period is determined based on the door status; When the cabinet door is in an open state, a state change timestamp and a tilt video of the filing cabinet in an open state are obtained; shape features and grayscale features of each storage area of ​​multiple frames of images in the tilt video are extracted based on a pre-constructed mask; The tilted video is screened based on the shape feature and the grayscale feature to obtain a target frame and a timestamp of the target frame; Extracting a pressure data sequence of each pressure sensing unit in the filing cabinet in a time window starting from the state change timestamp and ending at the timestamp of the target frame; and determining a target position where the pressure changes based on the pressure data sequence, wherein the pressure data sequence includes pressure values ​​at multiple time points of a position; The target frame is segmented based on a pre-built mask extraction to obtain a target block containing the target position and a non-target block not containing the target position; a rough inspection based on grayscale features is performed on the non-target block, and a fine inspection based on grayscale features, geometric features and text recognition is performed on the target block to obtain a detection result for each block; The position distribution information of all the file bags in the current storage period is constructed based on the detection result of each block, and all the file bags in the filing cabinet are managed based on the position distribution information of the current storage period and the position distribution information of the previous storage period.

[0006] In one embodiment of the present application, the shape features and grayscale features of each storage area of ​​the multiple frames of images in the oblique video are extracted based on a pre-constructed mask, including: Dynamically extracting frames from the tilted video to obtain multiple frames of images; Performing high-pass filtering and grayscale conversion processing on the multiple frames of images to obtain multiple frames of pre-processed images; Based on affine transformation, multiple frames of pre-processed images are converted from oblique viewing angles to normal viewing angles to obtain multiple frames of intermediate images; Performing AND operations on the pre-constructed mask and the multiple frames of intermediate images respectively to obtain multiple AND operation images; and marking the positions of the storage areas in the multiple AND operation images based on the pre-constructed mask; Extracting a target area that conforms to a pre-constructed first grayscale range from the plurality of AND operation images; Determine the storage area corresponding to each target area based on the marks of multiple target areas; extract the contour features of each storage area, extract the perimeter, area and rectangularity of the contour features, and obtain the shape features; The average grayscale and grayscale variance of each storage area are extracted to obtain the grayscale features.

[0007] In an embodiment of the present application, the tilted video is screened based on the shape feature and the grayscale feature to obtain a target frame and a timestamp of the target frame, including: Comparing the shape features and grayscale features of the multiple frames of intermediate images with pre-constructed shape feature screening parameters and grayscale feature screening parameters, respectively, wherein the shape feature screening parameters include a perimeter tolerance range, an area tolerance range, and a rectangular tolerance range, and the grayscale feature screening parameters include a grayscale tolerance range and a grayscale variance tolerance range; The intermediate image whose shape features and grayscale features are matched with the pre-constructed shape feature screening parameters and grayscale feature screening parameters respectively is used as the complete image of the storage area that is not blocked; A closing timestamp when the cabinet door is closed is obtained, and a plurality of continuous complete images closest to the closing timestamp are used as target frames, and the timestamp of the target frame is determined.

[0008] In one embodiment of the present application, determining the target position where the pressure changes based on the pressure data sequence includes: Performing time series differentiation on the pressure values ​​at any adjacent time points in the pressure data sequence to obtain a differential sequence; Sliding along the differential sequence based on a pre-constructed sliding window, and calculating the sum of the pressure difference values ​​within the sliding window during each sliding; When the sum of the pressure differences is greater than a preset difference threshold, it is determined that the position corresponding to the pressure data sequence is the target position.

[0009] In one embodiment of the present application, a rough inspection based on grayscale features is performed on non-target blocks, including: Perform high-pass filtering and grayscale conversion on non-target blocks to obtain a grayscale image ; For the grayscale image Perform brightness adjustment to obtain a brightness adjusted image , wherein the brightness adjustment image The mathematical expression is:

[0010] In the formula, Adjust the pixels in the image for brightness The gray value of is the pixel in the grayscale image The gray value of is the minimum grayscale value in the grayscale image, is the maximum grayscale value in the grayscale image; Extract the pixel points whose grayscale values ​​fall into the preset second grayscale range from the brightness-adjusted image to construct the archive bag area; perform morphological processing on the archive bag area image to obtain the archive bag area image. ; Based on the portfolio area image Area and the grayscale image Area Calculate the effective proportion of archive bags in the current storage period , The mathematical expression is:

[0011] Calculate the effective proportion of archive bags in the current storage period The effective proportion of the archive bags in the previous storage cycle Deviation rate , , at the deviation rate When the deviation rate is less than a preset threshold, it is determined that there is no file bag change in the storage unit corresponding to the current non-target block; When the deviation rate is greater than or equal to a preset threshold, the current non-target block is marked as containing the target block, and the target block is finely inspected based on grayscale features, geometric features, and text recognition.

[0012] In one embodiment of the present application, a target block is subjected to a fine inspection based on grayscale features, geometric features, and text recognition, including: Perform high-pass filtering and grayscale conversion on the target image block to obtain a target grayscale image ; For the target grayscale image Perform brightness adjustment to obtain the target brightness adjusted image ; Adjust the image from the target brightness Extracting pixel points whose grayscale values ​​fall into a preset second grayscale range to construct a target file bag area; and adjusting the image from the target brightness Extract contour features from Adjust the image to the target brightness The contours in the image are screened to obtain a target contour that meets the size of the file bag; the target file bag area and the target contour features are morphologically processed to obtain a file bag area image containing the target contour features. ; Adjusting the brightness of the target image based on the target contour feature Segmentation is performed to obtain a plurality of single images of the archive bags; the grayscale features and geometric features of the single image of each archive bag are extracted, and text recognition is performed on the single image of each archive bag to obtain text recognition results; and matching features are constructed based on the grayscale features, geometric features and text recognition results of the single image of each archive bag; Match the matching features of multiple file bags in the current storage period of the storage unit corresponding to the target block with the matching features of multiple file bags in the previous storage period to obtain a matching result; and determine the file bag change information of the storage unit based on the matching result, wherein the file bag change information includes addition, reduction and position change.

[0013] In one embodiment of the present application, the matching features of multiple file bags of the target image block corresponding to the storage unit in the current storage period are matched with the matching features of multiple file bags in the previous storage period to obtain a matching result, including: Calculate the current storage period of one of the archive bags The same as the archive bag from the previous storage cycle Grayscale similarity , geometric similarity And the cosine similarity of the text recognition results , where grayscale similarity and geometric similarity The mathematical expressions are:

[0014]

[0015] In the formula, For the The first storage cycle The average gray value of the file bag, For the The first storage cycle The average gray value of the file bag, For the The first storage cycle The gray value variance of the file bag, For the The first storage cycle The gray value variance of the file bag, For the The first storage cycle The perimeter of the outline of the file bag, For the The first storage cycle The perimeter of the outline of the file bag, For the The first storage cycle The area of ​​the outline of the file bag, For the The first storage cycle The area of ​​the outline of the file bag, For the The first storage cycle The aspect ratio of the outline of the portfolio, For the The first storage cycle The aspect ratio of the outline of the portfolio, is the first weight, is the second weight, is the third weight, is the fourth weight, is the fifth weight; Based on the grayscale similarity , geometric similarity And cosine similarity Calculate the matching degree , wherein the matching degree The mathematical expression is:

[0016] The file bags whose matching degree is greater than the preset matching degree threshold are matched to obtain the matching result.

[0017] In one embodiment of the present application, all the file bags in the filing cabinet are managed based on the location distribution information of the current storage period and the location distribution information of the previous storage period, including: Determine the difference information between the location distribution information of the current storage period and the location distribution information of the previous storage period; and change the storage status of the file bag and the retrieval information based on the difference information.

[0018] In one embodiment of the present application, a method for determining a storage period includes: Get the time difference between the cabinet door opening timestamp and the cabinet door closing timestamp; When the time difference is greater than the target duration, the period after the cabinet door is closed is taken as the next storage period.

[0019] The present application also provides a digital intelligent archive management system based on machine vision, including: An acquisition module, used to acquire the position distribution information of all the file bags in the filing cabinet in the previous storage period, and acquire the door status of the filing cabinet, wherein the storage period is determined based on the door status; A feature extraction module, used for obtaining a state change timestamp and a tilt video of the filing cabinet in the state where the cabinet door is open when the cabinet door is in the open state; extracting shape features and grayscale features of each storage area of ​​multiple frames of images in the tilt video based on a pre-built mask; A screening module, used for screening the oblique video based on the shape feature and the grayscale feature to obtain a target frame and a timestamp of the target frame; A pressure analysis module, for extracting a pressure data sequence of each pressure sensing unit in the filing cabinet in a time window from the state change timestamp to the target frame timestamp; and determining a target position where the pressure changes based on the pressure data sequence, wherein the pressure data sequence includes pressure values ​​at multiple time points of a position; A detection module is used to segment the target frame based on a pre-built mask extraction to obtain a target block containing the target position and a non-target block not containing the target position; perform a rough inspection based on grayscale features on the non-target block, and perform a fine inspection based on grayscale features, geometric features and text recognition on the target block to obtain a detection result for each block; A management module is used to construct the position distribution information of all the file bags in the current storage period based on the detection result of each block, and manage all the file bags in the filing cabinet based on the position distribution information of the current storage period and the position distribution information of the previous storage period.

[0020] The beneficial effects of the present invention are as follows: the digital intelligent archive management method and system based on machine vision of the present invention divides different storage periods by detecting the cabinet door of the filing cabinet. When the cabinet door is opened, the camera automatically collects the tilt video inside the cabinet, and extracts the target frame indicating the completion of the operation before the cabinet door is closed from the tilt video. The blocks corresponding to all storage compartments in the target frame are analyzed in combination with the pressure data in the cabinet. For blocks that do not produce pressure changes, a rough inspection based on grayscale feature analysis is adopted, and for blocks that produce pressure changes, a fine inspection based on grayscale features, geometric features and text recognition is adopted. Thereby ensuring the detection efficiency and detection accuracy. After the detection is completed, the position distribution information of all file bags is updated. Combined with the position distribution information of the previous storage period, the file bag status update can be efficiently completed, thereby realizing automatic information management. The present application has the advantages of efficient and accurate management and a high degree of automation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 This is an application scenario diagram of a digital intelligent archive management method based on machine vision shown in an embodiment of the present application; Figure 2 It is a flow chart of a digital intelligent archive management method based on machine vision shown in an embodiment of the present application; Figure 3 This is a schematic diagram of a rough inspection process in an embodiment of the present application; Figure 4 This is a schematic diagram of the precision inspection process in one embodiment of the present application; Figure 5 It is a structural diagram of a digital intelligent archive management system based on machine vision shown in an embodiment of the present application; Figure 6 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0024] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show the layers related to the present invention rather than being drawn according to the number, shape and size ratio of the layers in actual implementation. In actual implementation, the type and quantity of each layer may be changed arbitrarily, and the layer layout may also be more complicated.

[0025] In the following description, numerous details are discussed to provide a more thorough explanation of embodiments of the present invention; however, it is apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details.

[0026] Figure 1 is an application scenario diagram of a digital intelligent archive management method based on machine vision shown in an embodiment of the present application, such as Figure 1 As shown, in this application, a camera facing the front of the filing cabinet is set on the top of the filing cabinet, and a door magnetic switch is set on the cabinet door, and the door magnetic switch is used to sense the switch status of the cabinet door. After each cabinet door is opened, the camera shoots the filing cabinet at an inclined angle to avoid being blocked by the cabinet door. The operator will also block the image during the operation, but the inclined video contains a time window after the hands leave the cabinet after the operation is completed. This application extracts the complete image by collecting the time window when the hands leave the cabinet before the door is closed, and detects the image to obtain the change information of each file bag, thereby completing the automatic update of the status of the file bag.

[0027] Figure 2 is a flow chart of a digital intelligent archive management method based on machine vision shown in an embodiment of the present application, such as Figure 2 As shown: A digital intelligent archive management method based on machine vision in this embodiment may include steps S210 to S260: S210, obtaining the position distribution information of all the file bags in the filing cabinet in the last storage period, and obtaining the door status of the filing cabinet, wherein the storage period is determined based on the door status; In this application, the storage period is determined by the switch state of the cabinet door, including: (1) Get the door opening timestamp and door closing timestamp Time difference ; (2) In the time difference When it is greater than the target time (such as 5 seconds), the period after the cabinet door is closed is taken as the next storage period.

[0028] In time difference When the time is longer than the target time (e.g. 5 seconds), it is considered a valid access operation, which may involve the movement or rearrangement of the file bag, and the period after the cabinet door is closed is marked as the next storage period. This helps the system determine the correct storage period.

[0029] S220, when the cabinet door is in an open state, obtaining a state change timestamp and a tilted video of the filing cabinet in an open state; extracting shape features and grayscale features of each storage area of ​​multiple frames of images in the tilted video based on a pre-constructed mask; In order to avoid the cabinet door blocking the picture when opening the cabinet door, the camera in this application is tilted towards the inside of the cabinet, but this setting will also cause the manager's arm to block the picture when operating. Therefore, this application extracts frames from the tilted video by storing the morphology and grayscale features in the grid image. The principle is that the morphology (determined by the outline) and grayscale features of the storage grid image are different when it is not blocked and when it is blocked.

[0030] The specific process of extracting the shape features and grayscale features of each storage area of ​​the multi-frame image in the oblique video based on the pre-built mask includes: S221, dynamically extracting frames from the oblique video to obtain multiple frames of images; Dynamically extract a number of frames from the oblique video as the basis for analysis. This can be done by setting a fixed frame interval (such as extracting a frame every few seconds) or by deciding when to extract a new frame based on changes in the video content.

[0031] In this embodiment, based on dynamic frame extraction, multiple frames of images can be extracted when the degree of picture change is large, so as to retain more picture information.

[0032] S222, performing high-pass filtering and grayscale conversion processing on the multiple frames of images to obtain multiple frames of pre-processed images; High-pass filtering is used to enhance edge information in images and reduce the impact of background noise.

[0033] Grayscale conversion converts color images into grayscale images to simplify subsequent processing. Grayscale conversion is usually achieved by calculating the weighted average of the RGB values ​​of each pixel.

[0034] S223, converting the multiple frames of pre-processed images from oblique viewing angles to frontal viewing angles based on affine transformation to obtain multiple frames of intermediate images; Use affine transformation to adjust the angle of the image and flip the image to the normal view. Affine transformation requires a pre-determined transformation matrix, which can be calculated from known key points (such as the corner points of the filing cabinet).

[0035] S224, performing AND operations on the pre-constructed mask and the multiple frames of intermediate images respectively to obtain multiple AND operation images; and marking the positions of the storage areas in the multiple AND operation images based on the pre-constructed mask; By performing an AND operation, the image of the storage area can be retained. Thus, the border area of ​​the storage grid is removed. Each storage area is pre-marked in the mask, so that when the mask is aligned with multiple intermediate images, each storage area can be directly marked in the image.

[0036] S225, extracting a target area that conforms to a pre-constructed first grayscale range from the plurality of AND operation images; Under normal circumstances, the file bag and the background in the storage compartment will fall into a specific grayscale range. If the arm interferes, the grayscale will change. Therefore, the present application first extracts the target area that falls into the first grayscale range as the unobstructed storage compartment area.

[0037] S226, determining a storage area corresponding to each target area based on the marks of the multiple target areas; extracting contour features of each storage area, extracting the perimeter, area and rectangularity of the contour features, and obtaining shape features; Then, the contour features of the unobstructed storage compartment area are extracted to verify whether the shape of the unobstructed storage compartment area is complete, thereby determining whether it is obstructed.

[0038] S227, extracting the average grayscale and grayscale variance of each storage area to obtain grayscale features.

[0039] In addition, the average grayscale and grayscale variance of the storage area are directly extracted to verify whether the storage area has grayscale value changes due to occlusion.

[0040] S230, screening the oblique video based on the shape feature and the grayscale feature to obtain a target frame and a timestamp of the target frame; After obtaining the shape features and grayscale features, the shape features and grayscale features can be used to verify whether the storage grid image is blocked by hands, so as to extract the frame where the storage grid area that is not blocked is located. Specifically, it includes: S231, comparing the shape features and grayscale features of the multiple frames of intermediate images with pre-constructed shape feature screening parameters and grayscale feature screening parameters, respectively, wherein the shape feature screening parameters include a perimeter tolerance range, an area tolerance range, and a rectangular tolerance range, and the grayscale feature screening parameters include a grayscale tolerance range and a grayscale variance tolerance range; The present application constructs the corresponding tolerance range by extracting the shape features and grayscale features of multiple storage grid areas in advance from the unobstructed standard image. The construction of the tolerance range is based on statistical principles, that is, for each storage box, after extracting the image multiple times, the perimeter, area and rectangularity of multiple images are extracted. Then, the mean and variance are calculated based on the perimeter, area and rectangularity of multiple images, and the perimeter tolerance range, area tolerance range and rectangularity tolerance range are constructed based on 3 times the standard deviation.

[0041] Similarly, for each storage box, after extracting images multiple times, the grayscale means and grayscale variances of multiple images are extracted, and the means and variances of multiple grayscale means and grayscale variances are calculated respectively, and the grayscale tolerance range and grayscale variance tolerance range are constructed based on 3 times the standard deviation.

[0042] S232, taking the intermediate image whose shape features and grayscale features are matched with the pre-constructed shape feature screening parameters and grayscale feature screening parameters respectively as the complete image of the storage area that is not blocked; During the comparison, if both the grayscale features and the shape features fall within the corresponding tolerance range, it means that the grayscale and shape of the storage cell in the current image are consistent with the unobstructed image, which can indicate that the current image is not obstructed by the operator's hand, thereby causing changes in the contour and grayscale features.

[0043] S233, obtaining a closing timestamp when the cabinet door is closed, and taking a plurality of continuous complete images closest to the closing timestamp as target frames, and determining the timestamp of the target frames.

[0044] Considering that the operator may be in a daze in front of the filing cabinet, or perform other actions, in order to accurately filter out the image that best reflects the operation result, the present application also extracts multiple continuous complete images before the cabinet door is closed as the target frame.

[0045] S240, extracting a pressure data sequence of each pressure sensing unit in the filing cabinet in a time window between the state change timestamp and the target frame timestamp; and determining a target position where the pressure changes based on the pressure data sequence, wherein the pressure data sequence includes pressure values ​​at multiple time points of a position; In order to efficiently detect the image of each storage compartment unit, the present application uses a pressure sensor at the bottom of the storage compartment to detect the pressure change of each storage compartment, so as to find the storage compartment with possible state change. For the storage compartment image without pressure change, a rough inspection method is adopted to improve the detection efficiency and speed.

[0046] Wherein, determining the target position where the pressure changes based on the pressure data sequence includes: S241, performing time series difference on the pressure values ​​at any adjacent time points in the pressure data sequence to obtain a difference sequence; For any pressure data sequence at a position , perform time difference , we can get the difference series reflecting the pressure value changes at multiple time points .

[0047] S242, sliding along the differential sequence based on a pre-built sliding window, and calculating the sum of the pressure difference values ​​within the sliding window during each sliding; For the difference sequence, a sliding window is used to extract the time window and the sum of the pressure differences within the time window is accumulated. , the pressure changes in multiple time windows can be obtained.

[0048] S243: When the sum of the pressure differences is greater than a preset difference threshold, determining that the position corresponding to the pressure data sequence is the target position.

[0049] If the pressure changes greatly within a time window, so that the sum of the pressure differences is greater than the preset difference threshold, it means that there is a change in the file bag at the corresponding position, which causes the pressure change.

[0050] In addition, in this embodiment, a storage cell has multiple pressure sensors, that is, multiple positions.

[0051] S250, segmenting the target frame based on a pre-constructed mask extraction to obtain a target block containing the target position and a non-target block not containing the target position; performing a rough inspection based on grayscale features on the non-target block, and performing a fine inspection based on grayscale features, geometric features, and text recognition on the target block to obtain a detection result for each block; Before using the mask to segment the target frame, it is necessary to average the images of multiple target frames. Then segment again to obtain blocks of multiple storage cells. If these blocks have target positions, they are marked as target blocks; if they do not have target positions, they are marked as non-target blocks. Then the non-target images are roughly inspected, and if there are no problems through rapid inspection, they are marked as "no change", and the position distribution information in the storage cell continues the position distribution information of the previous storage period. For the target block, it is necessary to accurately detect whether the file bag in the storage cell corresponding to the target block has changed.

[0052] Figure 3 FIG. 1 is a schematic diagram of a rough inspection process in an embodiment of the present application. Figure 3 As shown in the figure, a rough inspection based on grayscale features is performed on non-target blocks, including: S2501, high-pass filtering and grayscale conversion are performed on the non-target blocks to obtain a grayscale image ; S2502, the grayscale image Perform brightness adjustment to obtain a brightness adjusted image , wherein the brightness adjustment image The mathematical expression is:

[0053] In step S2502, the grayscale image is first Normalize it, that is , and then expand the normalized image to the grayscale space with a maximum grayscale value of 255, so as to avoid the grayscale difference caused by different ambient brightness and provide a unified brightness parameter for subsequent processing.

[0054] In the formula, Adjust the pixels in the image for brightness The gray value of is the pixel in the grayscale image The gray value of is the minimum grayscale value in the grayscale image, is the maximum grayscale value in the grayscale image; S2503, extracting pixel points whose grayscale values ​​fall into a preset second grayscale range from the brightness-adjusted image to construct a file bag region; performing morphological processing on the file bag region image to obtain a file bag region image ; The second grayscale range is the approximate grayscale range of the archive bag under standard brightness conditions. The second grayscale range can also be constructed by statistical methods. Thus, the area where the archive bag is located can be extracted. For some of the extracted discrete abnormal points, morphological processing (including corrosion and expansion) is used to remove them, and an image of the archive bag area that only retains the grayscale information of the archive bag is obtained. .

[0055] S2504, based on the image of the file bag area Area and the grayscale image Area Calculate the effective proportion of archive bags in the current storage period , The mathematical expression is:

[0056] By calculating the effective proportion of the file bag The proportion of the file bag area to the storage compartment area can be roughly determined, and the number of file bags can be inferred.

[0057] S2505, calculate the effective proportion of archive bags in the current storage period The effective proportion of the archive bags in the previous storage cycle Deviation rate , , at the deviation rate When the deviation rate is less than a preset threshold, it is determined that there is no file bag change in the storage unit corresponding to the current non-target block; When the deviation rate is greater than or equal to a preset threshold, the current non-target block is marked as containing the target block, and the target block is finely inspected based on grayscale features, geometric features, and text recognition.

[0058] If the current storage cycle ratio The ratio to the previous storage cycle If the deviation rate is not much different, it means that the file bag has not changed. The storage unit has not undergone a large pressure change and the file bag shape change. It can be determined that the storage unit has not undergone a state change of the file bag.

[0059] If the current storage cycle ratio The ratio to the previous storage cycle If the deviation rate is quite different, further fine-tuning is required.

[0060] Figure 4 FIG. 1 is a schematic diagram of a precision inspection process in an embodiment of the present application. Figure 4 In the embodiment shown, the implementation process of the precision inspection is as follows: S2511, high-pass filtering and grayscale conversion are performed on the target image block to obtain a target grayscale image ; S2512, the target grayscale image Perform brightness adjustment to obtain the target brightness adjusted image ; The process and principle of brightness adjustment in step S2512 are consistent with the process and principle of brightness adjustment in the rough inspection process, and will not be repeated here.

[0061] S2513, adjusting the image from the target brightness Extracting pixel points whose grayscale values ​​fall into a preset second grayscale range to construct a target file bag area; and adjusting the image from the target brightness Extract contour features from The precision inspection process requires not only the grayscale information of the target file bag area, but also the contour features of the file bag to verify the geometric shape of the file bag. Therefore, it is also necessary to extract the contour features.

[0062] S2514, adjusting the image for the target brightness The contours in the image are screened to obtain a target contour that meets the size of the file bag; the target file bag area and the target contour features are morphologically processed to obtain a file bag area image containing the target contour features. ; In order to accurately filter out the outlines belonging to the archive bag, this application adjusts the image to the target brightness The contours in the image are morphologically screened, and the contours that meet the preset area range, perimeter range, and aspect ratio range are used as the contours of the archive bag, and other contours are deleted. In addition, in order to make the target contour completely closed, the target contour is also eroded and expanded to obtain a contour image containing the complete contour.

[0063] S2515, adjusting the target brightness image based on the target contour feature Segmentation is performed to obtain a plurality of single images of the archive bags; the grayscale features and geometric features of the single image of each archive bag are extracted, and text recognition is performed on the single image of each archive bag to obtain text recognition results; and matching features are constructed based on the grayscale features, geometric features and text recognition results of the single image of each archive bag; After obtaining the closed contour, use the closed contour to adjust the image to the target brightness Segmentation is performed. Thus, a single image of a single file bag is extracted. The grayscale features and geometric features of each single image are extracted. In addition, the text on the side label of the file bag is recognized through an externally retrieved text recognition model. Thus, grayscale features (including average grayscale value and grayscale value variance), geometric features (including contour area, perimeter and aspect ratio) and text recognition results are obtained. They serve as the reference features for subsequent matching.

[0064] S2516, matching the matching features of multiple file bags in the current storage period of the storage unit corresponding to the target block with the matching features of multiple file bags in the previous storage period to obtain a matching result; and determining the file bag change information of the storage unit based on the matching result, wherein the file bag change information includes addition, reduction and position change.

[0065] After obtaining the matching features of the current storage cycle, the matching features of the previous storage cycle are compared to obtain the changes in the file bags in the storage unit, such as the addition of new file bags, the reduction of file bags, and the change of the position of file bags. The matching features of the previous storage cycle are extracted in the same way, which will not be repeated here.

[0066] In one embodiment of the present application, the matching features of multiple file bags of the target image block corresponding to the storage unit in the current storage period are matched with the matching features of multiple file bags in the previous storage period to obtain a matching result, including: S25161, calculate one of the archive bags in the current storage period The same as the archive bag from the previous storage cycle Grayscale similarity , geometric similarity And the cosine similarity of the text recognition results , where grayscale similarity and geometric similarity The mathematical expressions are:

[0067]

[0068] In the formula, For the The first storage cycle The average gray value of the file bag, For the The first storage cycle The average gray value of the file bag, For the The first storage cycle The gray value variance of the file bag, For the The first storage cycle The gray value variance of the file bag, For the The first storage cycle The perimeter of the outline of the file bag, For the The first storage cycle The perimeter of the outline of the file bag, For the The first storage cycle The area of ​​the outline of the file bag, For the The first storage cycle The area of ​​the outline of the file bag, For the The first storage cycle The aspect ratio of the outline of the portfolio, For the The first storage cycle The aspect ratio of the outline of the portfolio, is the first weight, is the second weight, is the third weight, is the fourth weight, is the fifth weight; In this embodiment, the grayscale deviation rate and grayscale variance deviation rate are calculated respectively as grayscale similarity. The perimeter deviation rate, area deviation rate and aspect ratio deviation rate are calculated as the geometric similarity. Finally, the grayscale similarity is weighted using exponential weighting. , geometric similarity Calculate. In general, the weight The average value is 1.

[0069] S25162, based on the grayscale similarity , geometric similarity And cosine similarity Calculate the matching degree , wherein the matching degree The mathematical expression is:

[0070] in, is the sixth weight, is the seventh weight, The eighth weight.

[0071] Then use the grayscale similarity , geometric similarity To calculate the comprehensive matching degree, in this application, the text is more important, so Take 0.6, You can also configure different weight values ​​according to actual needs.

[0072] S25163, matching the file bags whose matching degree is greater than a preset matching degree threshold value to obtain a matching result.

[0073] In the specific matching process, the matching features of a file bag in the current storage unit of the current storage cycle are extracted and matched with the matching features of multiple file bags in the current storage unit of the previous storage cycle. It is judged whether the current file bag is a newly added file bag. If the current file bag cannot match the multiple file bags in the current storage unit of the previous storage cycle, it means that the current file bag is a newly added file bag. If there is a matching file bag, its position information is compared to see if it is consistent, so as to judge whether there is a position change. After the matching is completed, if there is a file bag that is not matched among the multiple file bags in the current storage unit of the previous storage cycle, it means that there is a reduction in the number of file bags.

[0074] The above changes can all be matched to the file bag identification code through text recognition results.

[0075] S260, constructing the position distribution information of all the file bags in the current storage period based on the detection result of each block, and managing all the file bags in the filing cabinet based on the position distribution information of the current storage period and the position distribution information of the previous storage period.

[0076] Specifically, it includes: determining the difference information between the location distribution information of the current storage period and the location distribution information of the previous storage period; and changing the storage status of the file bag and the retrieval information based on the difference information.

[0077] For example, if there is a reduction in the number of file bags A1 in the storage unit A, the reduction information of the file bag A1 can be recorded in the location distribution information of the current storage period. In combination with the personnel registration information, it can be quickly determined who took the file bag A1.

[0078] The updated location distribution information can also help personnel to quickly and accurately retrieve files, avoiding unnecessary retrieval errors. This application can effectively help managers to manage information on file bags.

[0079] The present invention discloses a digital intelligent file management method based on machine vision, which divides different storage periods by detecting the door of the filing cabinet. When the door is opened, the camera automatically collects the tilt video inside the cabinet, and extracts the target frame indicating the completion of the operation before the door is closed from the tilt video. The image blocks corresponding to all storage compartments in the target frame are analyzed in combination with the pressure data in the cabinet. For the image blocks that do not produce pressure changes, a rough inspection based on grayscale feature analysis is adopted, and for the image blocks that produce pressure changes, a fine inspection based on grayscale features, geometric features and text recognition is adopted. Thereby ensuring the detection efficiency and detection accuracy. After the detection is completed, the position distribution information of all file bags is updated, and combined with the position distribution information of the previous storage period, the file bag status update can be efficiently completed, thereby realizing automatic information management. The present application has the advantages of efficient and accurate management and a high degree of automation.

[0080] like Figure 5 As shown, the present application also provides a digital intelligent archive management system based on machine vision, including: An acquisition module, used to acquire the position distribution information of all the file bags in the filing cabinet in the previous storage period, and acquire the door status of the filing cabinet, wherein the storage period is determined based on the door status; A feature extraction module, used for obtaining a state change timestamp and a tilt video of the filing cabinet in the state where the cabinet door is open when the cabinet door is in the open state; extracting shape features and grayscale features of each storage area of ​​multiple frames of images in the tilt video based on a pre-built mask; A screening module, used for screening the oblique video based on the shape feature and the grayscale feature to obtain a target frame and a timestamp of the target frame; A pressure analysis module, for extracting a pressure data sequence of each pressure sensing unit in the filing cabinet in a time window from the state change timestamp to the target frame timestamp; and determining a target position where the pressure changes based on the pressure data sequence, wherein the pressure data sequence includes pressure values ​​at multiple time points of a position; A detection module is used to segment the target frame based on a pre-built mask extraction to obtain a target block containing the target position and a non-target block not containing the target position; perform a rough inspection based on grayscale features on the non-target block, and perform a fine inspection based on grayscale features, geometric features and text recognition on the target block to obtain a detection result for each block; A management module is used to construct the position distribution information of all the file bags in the current storage period based on the detection result of each block, and manage all the file bags in the filing cabinet based on the position distribution information of the current storage period and the position distribution information of the previous storage period.

[0081] A digital intelligent archive management system based on machine vision of the present invention divides different storage periods by detecting the doors of the filing cabinets. When the doors are opened, the camera automatically collects the tilt video inside the cabinet, and extracts the target frame indicating the completion of the operation before the doors are closed from the tilt video. The blocks corresponding to all storage compartments in the target frame are analyzed in combination with the pressure data in the cabinet. For blocks that do not produce pressure changes, a rough inspection based on grayscale feature analysis is adopted, and for blocks that produce pressure changes, a fine inspection based on grayscale features, geometric features and text recognition is adopted. Thereby ensuring the detection efficiency and detection accuracy. After the detection is completed, the position distribution information of all file bags is updated. Combined with the position distribution information of the previous storage period, the file bag status update can be efficiently completed, thereby realizing automatic information management. The present application has the advantages of efficient and accurate management and a high degree of automation.

[0082] Figure 6 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 6 The computer system of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0083] like Figure 6 As shown, the computer system includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603, such as executing the method in the above embodiment. In the RAM 603, various programs and data required for system operation are also stored. The CPU 601, the ROM 602 and the RAM 603 are connected to each other through the bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0084] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.

[0085] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 609, and / or installed from a removable medium 611. When the computer program is executed by a central processing unit (CPU) 601, various functions defined in the system of the present application are executed.

[0086] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the above-mentioned module, a program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0088] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0089] Another aspect of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the above method. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0090] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in each of the above embodiments.

[0091] The above embodiments are only preferred embodiments for fully illustrating the present application, and the protection scope of the present application is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present application is within the protection scope of the present application.

Claims

1. A digital intelligent archive management method based on machine vision, characterized in that: Includes steps: Obtaining the location distribution information of all the file bags in the filing cabinet in the last storage period, and obtaining the door status of the filing cabinet, wherein the storage period is determined based on the door status; When the cabinet door is in an open state, a state change timestamp and a tilt video of the filing cabinet in an open state are obtained; shape features and grayscale features of each storage area of ​​multiple frames of images in the tilt video are extracted based on a pre-constructed mask; The tilted video is screened based on the shape feature and the grayscale feature to obtain a target frame and a timestamp of the target frame; Extracting a pressure data sequence of each pressure sensing unit in the filing cabinet in a time window starting from the state change timestamp and ending at the timestamp of the target frame; and determining a target position where the pressure changes based on the pressure data sequence, wherein the pressure data sequence includes pressure values ​​at multiple time points of a position; The target frame is segmented based on a pre-built mask extraction to obtain a target block containing the target position and a non-target block not containing the target position; a rough inspection based on grayscale features is performed on the non-target block, and a fine inspection based on grayscale features, geometric features and text recognition is performed on the target block to obtain a detection result for each block; The position distribution information of all the file bags in the current storage period is constructed based on the detection result of each block, and all the file bags in the filing cabinet are managed based on the position distribution information of the current storage period and the position distribution information of the previous storage period.

2. According to the method of claim 1, the digital intelligent archive management method based on machine vision is characterized in that: Extracting shape features and grayscale features of each storage area of ​​the multiple frame images in the oblique video based on a pre-built mask, including: Dynamically extracting frames from the tilted video to obtain multiple frames of images; Performing high-pass filtering and grayscale conversion processing on the multiple frames of images to obtain multiple frames of pre-processed images; Based on affine transformation, multiple frames of pre-processed images are converted from oblique viewing angles to normal viewing angles to obtain multiple frames of intermediate images; Performing AND operations on the pre-constructed mask and the multiple frames of intermediate images respectively to obtain multiple AND operation images; and marking the positions of the storage areas in the multiple AND operation images based on the pre-constructed mask; Extracting a target area that conforms to a pre-constructed first grayscale range from the plurality of AND operation images; Determine the storage area corresponding to each target area based on the marks of multiple target areas; extract the contour features of each storage area, extract the perimeter, area and rectangularity of the contour features, and obtain the shape features; The average grayscale and grayscale variance of each storage area are extracted to obtain the grayscale features.

3. The method for digital intelligent archive management based on machine vision according to claim 2 is characterized in that: The tilted video is screened based on the shape feature and the grayscale feature to obtain a target frame and a timestamp of the target frame, including: Comparing the shape features and grayscale features of the multiple frames of intermediate images with pre-constructed shape feature screening parameters and grayscale feature screening parameters, respectively, wherein the shape feature screening parameters include a perimeter tolerance range, an area tolerance range, and a rectangular tolerance range, and the grayscale feature screening parameters include a grayscale tolerance range and a grayscale variance tolerance range; The intermediate image whose shape features and grayscale features are matched with the pre-constructed shape feature screening parameters and grayscale feature screening parameters respectively is used as the complete image of the storage area that is not blocked; A closing timestamp when the cabinet door is closed is obtained, and a plurality of continuous complete images closest to the closing timestamp are used as target frames, and the timestamp of the target frame is determined.

4. The method for digital intelligent archive management based on machine vision according to claim 1, characterized in that: Determining a target position where pressure changes based on the pressure data sequence includes: Performing time series differentiation on the pressure values ​​at any adjacent time points in the pressure data sequence to obtain a differential sequence; Sliding along the differential sequence based on a pre-constructed sliding window, and calculating the sum of the pressure difference values ​​within the sliding window during each sliding; When the sum of the pressure differences is greater than a preset difference threshold, it is determined that the position corresponding to the pressure data sequence is the target position.

5. The method for digital intelligent archive management based on machine vision according to claim 1 is characterized in that: Perform a rough inspection of non-target blocks based on grayscale features, including: Perform high-pass filtering and grayscale conversion on non-target blocks to obtain a grayscale image ; For the grayscale image Perform brightness adjustment to obtain a brightness adjusted image , wherein the brightness adjustment image The mathematical expression is: In the formula, Adjust the pixels in the image for brightness The gray value of is the pixel in the grayscale image The gray value of is the minimum grayscale value in the grayscale image, is the maximum grayscale value in the grayscale image; Extract the pixel points whose grayscale values ​​fall into the preset second grayscale range from the brightness-adjusted image to construct the archive bag area; perform morphological processing on the archive bag area image to obtain the archive bag area image. ; Based on the portfolio area image Area and the grayscale image Area Calculate the effective proportion of archive bags in the current storage period , The mathematical expression is: Calculate the effective proportion of archive bags in the current storage period The effective proportion of the archive bags in the previous storage cycle Deviation rate , , at the deviation rate When the deviation rate is less than a preset threshold, it is determined that there is no file bag change in the storage unit corresponding to the current non-target block; When the deviation rate is greater than or equal to a preset threshold, the current non-target block is marked as containing the target block, and the target block is finely inspected based on grayscale features, geometric features, and text recognition.

6. A digital intelligent archive management method based on machine vision according to claim 1 or 5, characterized in that: Perform precision inspection on the target image blocks based on grayscale features, geometric features, and text recognition, including: Perform high-pass filtering and grayscale conversion on the target image block to obtain a target grayscale image ; For the target grayscale image Perform brightness adjustment to obtain the target brightness adjusted image ; Adjust the image from the target brightness Extracting pixel points whose grayscale values ​​fall into a preset second grayscale range to construct a target file bag area; and adjusting the image from the target brightness Extract contour features from Adjust the image to the target brightness The contours in the image are screened to obtain a target contour that meets the size of the file bag; the target file bag area and the target contour features are morphologically processed to obtain a file bag area image containing the target contour features. ; Adjusting the brightness of the target image based on the target contour feature Segmentation is performed to obtain a plurality of single images of the archive bags; the grayscale features and geometric features of the single image of each archive bag are extracted, and text recognition is performed on the single image of each archive bag to obtain text recognition results; and matching features are constructed based on the grayscale features, geometric features and text recognition results of the single image of each archive bag; Match the matching features of multiple file bags in the current storage period of the storage unit corresponding to the target block with the matching features of multiple file bags in the previous storage period to obtain a matching result; and determine the file bag change information of the storage unit based on the matching result, wherein the file bag change information includes addition, reduction and position change.

7. A digital intelligent archive management method based on machine vision according to claim 6, characterized in that: Matching the matching features of multiple archive bags in the current storage period of the storage unit corresponding to the target block with the matching features of multiple archive bags in the previous storage period to obtain a matching result, including: Calculate the current storage period of one of the archive bags The same as the archive bag from the previous storage cycle Grayscale similarity , geometric similarity And the cosine similarity of the text recognition results , where grayscale similarity and geometric similarity The mathematical expressions are: In the formula, For the The first storage cycle The average gray value of the file bag, For the The first storage cycle The average gray value of the file bag, For the The first storage cycle The gray value variance of the file bag, For the The first storage cycle The gray value variance of the file bag, For the The first storage cycle The perimeter of the outline of the file bag, For the The first storage cycle The perimeter of the outline of the file bag, For the The first storage cycle The area of ​​the outline of the file bag, For the The first storage cycle The area of ​​the outline of the file bag, For the The first storage cycle The aspect ratio of the outline of the portfolio, For the The first storage cycle The aspect ratio of the outline of the portfolio, is the first weight, is the second weight, is the third weight, is the fourth weight, is the fifth weight; Based on the grayscale similarity , geometric similarity And cosine similarity Calculate the matching degree , wherein the matching degree The mathematical expression is: In the formula, is the sixth weight, is the seventh weight, is the eighth weight; the file bags whose matching degree is greater than the preset matching degree threshold are matched to obtain the matching result.

8. The method for digital intelligent archive management based on machine vision according to claim 1, characterized in that: Managing all the file bags in the filing cabinet based on the location distribution information of the current storage period and the location distribution information of the previous storage period includes: Determine the difference information between the location distribution information of the current storage period and the location distribution information of the previous storage period; and change the storage status of the file bag and the retrieval information based on the difference information.

9. The method for digital intelligent archive management based on machine vision according to claim 1, characterized in that: Methods for determining the storage period include: Get the time difference between the cabinet door opening timestamp and the cabinet door closing timestamp; When the time difference is greater than the target duration, the period after the cabinet door is closed is taken as the next storage period.

10. A digital intelligent archive management system based on machine vision, characterized in that: include: An acquisition module, used to acquire the position distribution information of all the file bags in the filing cabinet in the previous storage period, and acquire the door status of the filing cabinet, wherein the storage period is determined based on the door status; A feature extraction module, used for obtaining a state change timestamp and a tilt video of the filing cabinet in the state where the cabinet door is open when the cabinet door is in the open state; extracting shape features and grayscale features of each storage area of ​​multiple frames of images in the tilt video based on a pre-built mask; A screening module, used for screening the oblique video based on the shape feature and the grayscale feature to obtain a target frame and a timestamp of the target frame; A pressure analysis module, for extracting a pressure data sequence of each pressure sensing unit in the filing cabinet in a time window from the state change timestamp to the target frame timestamp; and determining a target position where the pressure changes based on the pressure data sequence, wherein the pressure data sequence includes pressure values ​​at multiple time points of a position; A detection module is used to segment the target frame based on a pre-built mask extraction to obtain a target block containing the target position and a non-target block not containing the target position; perform a rough inspection based on grayscale features on the non-target block, and perform a fine inspection based on grayscale features, geometric features and text recognition on the target block to obtain a detection result for each block; A management module is used to construct the position distribution information of all the file bags in the current storage period based on the detection result of each block, and manage all the file bags in the filing cabinet based on the position distribution information of the current storage period and the position distribution information of the previous storage period.

Images