Enterprise asset intelligent checking system and method based on computer vision

Abstract

The invention relates to the technical field of inventory systems, in particular to an enterprise asset intelligent inventory system and method based on computer vision, and the system comprises an asset label generation module, an asset warehousing module, an asset inventory module, a computer-based inventory management module, a computer-based inventory management module, a computer-based inventory management module, a computer-based inventory management module, a computer-based inventory management module and a computer-based inventory management module, and the computer-based inventory management module is in communication connection with the asset label generation module and the asset warehousing module and is in communication connection with the asset warehousing module. Acquiring an asset image; checking and classifying the assets through an image recognition technology based on the asset images; inventory data is generated and transmitted to the abnormity monitoring module; the abnormity monitoring module is in communication connection with the asset checking module; all assets are monitored in real time based on a preset rule; when an abnormal condition is detected, an alarm signal is generated; the abnormity positioning module is in communication connection with the abnormity monitoring module; based on the alarm signal, the asset image and the asset position information are utilized to accurately position the abnormal point, so that the intellectualization and automation of enterprise asset management are realized, and remarkable effects in multiple aspects are brought.

Description

Technical Field

[0001] The present invention relates to the technical field of inventory counting systems, in particular to an enterprise asset intelligent inventory counting system and method based on computer vision. Background Art

[0002] With the continuous expansion of enterprise scale and the increasing number of asset types, the traditional manual inventory method can no longer meet the needs of modern enterprises for asset management. In recent years, with the rapid development of information technology, some innovative solutions have also emerged in the field of enterprise asset management. Among them, the asset management system based on RFID technology has improved the efficiency of inventory to a certain extent, but there are still some limitations. For example, RFID technology has poor reading effect in metal environments and cannot intuitively identify the appearance status of assets.

[0003] On the other hand, with the advancement of computer vision technology, some companies have begun to try to apply image recognition technology to asset management. This method can intuitively capture the appearance information of assets, but it still faces problems such as slow recognition speed and low accuracy in large-scale asset management. In addition, it is difficult to achieve accurate positioning and real-time tracking of assets by relying solely on image recognition technology.

[0004] At present, there are also some asset management systems that combine RFID and image recognition technology on the market. However, these systems often simply use the two technologies in parallel, lacking deep integration and coordinated optimization. Therefore, there are still many problems in practical applications, such as slow system response, untimely anomaly detection, and low positioning accuracy.

[0005] In addition, existing asset management systems generally lack intelligent abnormal monitoring and alarm mechanisms. Most systems can only perform simple quantity verification, and are unable to detect and locate abnormal assets in a timely manner, let alone take appropriate measures based on the severity of the abnormality. In this case, enterprises face potential asset loss risks, and management efficiency is difficult to substantially improve.

[0006] In view of the above problems, there is an urgent need for an enterprise asset management system that can organically integrate RFID technology and computer vision technology and has intelligent abnormal monitoring and precise positioning capabilities. The present invention is proposed to address this demand. Summary of the invention

[0007] The technical problem to be solved by the present invention is how to build an efficient, accurate and intelligent enterprise asset inventory system to achieve rapid identification, precise positioning and real-time monitoring of assets, while being able to promptly discover and handle abnormal situations.

[0008] The present invention proposes an enterprise asset intelligent inventory system based on computer vision, comprising:

[0009] Asset tag generation module for:

[0010] Generate a QR code containing asset information;

[0011] Splicing the QR code with the asset image to generate a label image;

[0012] The asset storage module is connected to the asset tag generation module for:

[0013] Receiving the label image sent by the asset label generation module;

[0014] storing the label image on a server;

[0015] Binding the label image to the RFID label;

[0016] The asset inventory module is connected to the asset storage module for:

[0017] Obtain the RFID tag information of the asset through the RFID reader;

[0018] Collect asset images;

[0019] Based on the asset images, inventory and classify the assets using image recognition technology;

[0020] Generate inventory data and transmit it to the abnormal monitoring module;

[0021] The abnormal monitoring module is connected to the asset inventory module for:

[0022] Receiving inventory data sent by the asset inventory module;

[0023] Real-time monitoring of all assets based on preset rules;

[0024] When an abnormal situation is detected, an alarm signal is generated;

[0025] The abnormality location module is connected to the abnormality monitoring module for:

[0026] Receiving an alarm signal sent by the abnormality monitoring module;

[0027] Based on the alarm signal, the abnormal point is accurately located using the asset image and asset location information.

[0028] Preferably, the asset tag generating module comprises:

[0029] A QR code generator, used to generate a QR code based on asset information;

[0030] A text information extractor, in communication with the QR code generator, for extracting asset text information from the enterprise management system;

[0031] A two-dimensional code conversion module, which is in communication with the two-dimensional code generator and is used to convert the generated two-dimensional code into an image format;

[0032] The first image stitcher is connected to the two-dimensional code conversion module for stitching the converted two-dimensional code image with the asset image to generate a final label image.

[0033] Preferably, the asset storage module includes:

[0034] An image storage device is used to store the label image in a designated location of the server;

[0035] The data binder is connected to the image storage device for associating and binding the storage location information of the label image with the RFID label, and storing the binding information in a database.

[0036] Preferably, the asset inventory module includes:

[0037] RFID reader, used to read the RFID tag information of the asset;

[0038] a second image stitcher, in communication with the RFID reader, for stitching the collected asset image with the corresponding tag image obtained from the database to generate a tagged asset image;

[0039] An image recognizer, which is in communication with the second image stitcher and is used to perform image recognition processing on the label asset image;

[0040] A classifier is connected in communication with the image recognizer and is used to classify the counted assets based on the image recognition results.

[0041] Preferably, the abnormality monitoring module comprises:

[0042] A first image processing device, used for pre-processing the asset image;

[0043] A second image processing device, used for extracting features from the tagged asset image;

[0044] A data transmitter, connected to the first image processing device and the second image processing device for transmitting processed image data;

[0045] The abnormality alarm device is connected to the data transmitter for comparing the processed image data based on preset rules and generating an alarm signal when an abnormality is detected.

[0046] Preferably, the abnormality locating module comprises:

[0047] a second image separator, for separating the label asset image into an asset image and a label image;

[0048] a data processor, connected in communication with the second image separator, and configured to perform data processing on the separated asset image and label image;

[0049] A precise locator is communicatively connected to the data processor and is used to accurately locate the abnormal point based on the processed image data and asset location information.

[0050] Preferably, it further comprises an asset location determination module, which is in communication with the asset inventory module and is used to:

[0051] Obtain the geographic coordinate information of the three RFID readers with the highest signal strength with the RFID tag of the asset to be located;

[0052] Projecting the geographic coordinate information of the three RFID readers on an electronic map respectively;

[0053] Calculate the intersection points of the lines connecting the three RFID readers in pairs;

[0054] The intersection point is determined as the geographic coordinates of the asset to be located.

[0055] Preferably, the image recognizer uses an improved convolutional neural network for image recognition, and the improved convolutional neural network includes:

[0056] Input layer, used to receive label asset image data;

[0057] A plurality of convolutional layers, connected to the input layer, for extracting features from the image data, wherein the size and number of convolutional kernels of each convolutional layer can be dynamically adjusted;

[0058] A plurality of pooling layers, alternately connected with the convolutional layers, are used to reduce the dimension of the feature map and extract the main features;

[0059] The fully connected layer, connected to the last pooling layer, is used to map the extracted features to the classification space;

[0060] An output layer, connected to the fully connected layer, for outputting asset classification results;

[0061] Wherein, the convolutional layer and the fully connected layer both use ReLu as the activation function.

[0062] Preferably, it also includes an asset abnormality alarm module, which is in communication connection with the abnormality monitoring module and the abnormality positioning module, and is used for:

[0063] Receive the abnormal information sent by the abnormal monitoring module and the abnormal location information sent by the abnormal location module;

[0064] Based on the preset abnormality level rules, abnormalities are graded;

[0065] According to the abnormality level, select the corresponding alarm method, including but not limited to SMS reminder, email notification, system pop-up window and sound and light alarm;

[0066] An exception report is generated and sent to a designated manager.

[0067] The enterprise asset intelligent inventory method based on the system includes the following steps:

[0068] S100, textual processing is performed on the enterprise assets to obtain textual information;

[0069] S200, using an asset tag generation module to generate a tag image, storing it in an image memory, and binding it to an RFID tag;

[0070] S300: Inventory the assets using the RFID reader, including:

[0071] S310, reading the RFID tag through the RFID reader;

[0072] S320, identifying and locating the RFID tag by image recognition;

[0073] S330, according to the RFID tag, obtain the corresponding asset name, specification and quantity, and calculate the overall asset name and inventory quantity;

[0074] S340, classifying the assets through a convolutional neural network classification model to obtain different types of assets and their quantities;

[0075] S400, monitor the asset status in real time through the abnormal monitoring module, and alarm if an abnormal situation occurs;

[0076] S500, the abnormal location module locates the abnormal points of the asset location according to the monitoring alarm records, including:

[0077] S510, the abnormality positioning module receives the monitoring alarm information and transmits the abnormal label image to the data processor;

[0078] S520, the data processor analyzes the monitoring alarm information to obtain the location of the abnormal product;

[0079] S530, according to the abnormal position, find out the asset tag image information of the abnormal type;

[0080] S540, indexing abnormal asset image information in a database according to abnormal asset tag image information;

[0081] S550. Use improved image recognition technology to identify the image of the asset to be counted and locate abnormal positions of the assets.

[0082] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0083] The computer vision-based enterprise asset intelligent inventory system and method provided by the present invention realizes the intelligent and automated management of enterprise assets by organically integrating RFID technology and computer vision technology and combining with deep learning algorithms, bringing significant effects in many aspects.

[0084] First, from a macro perspective, the system of the present invention greatly improves the efficiency and accuracy of enterprise asset management. Traditional manual inventory methods usually require a lot of manpower and time, and are prone to errors. The system of the present invention can complete the inventory of large-scale assets in a short period of time, greatly reducing labor costs and time costs. At the same time, due to the use of advanced image recognition technology and RFID technology, the accuracy of inventory has been significantly improved, effectively reducing errors and omissions in asset management.

[0085] Secondly, the system of the present invention realizes the real-time and dynamic nature of asset management. Through the abnormal monitoring module and the asset location determination module, the system can track the status and location of each asset in real time. This not only improves asset utilization, but also can detect and handle abnormal situations in a timely manner, minimizing the risk of asset loss. Especially in large or distributed enterprises, this real-time monitoring and dynamic management capability is particularly important.

[0086] From a microscopic perspective, the system of the present invention has innovations and breakthroughs in multiple technical aspects. For example, the asset tag generation module innovatively splices the QR code and the asset image to generate an information-rich tag image. This design is not only convenient for machine recognition, but also for manual verification, reflecting the concept of human-machine collaboration. For another example, the image recognizer uses an improved convolutional neural network, in which the dynamic adjustment mechanism enables the system to adapt to asset recognition tasks of different types and complexities, greatly improving the system's adaptability and generalization capabilities.

[0087] In addition, the system of the present invention also performs well in resolving technical contradictions. For example, RFID technology and computer vision technology each have their own advantages and disadvantages. The present invention achieves complementary advantages of the two technologies through ingenious system design. RFID technology provides rapid asset identification and positioning capabilities, while computer vision technology provides intuitive asset status monitoring capabilities. The combination of the two technologies not only improves the overall performance of the system, but also enhances the robustness of the system.

[0088] Another outstanding advantage of the present invention is its intelligent exception handling mechanism. The asset abnormality alarm module can not only detect abnormalities in a timely manner, but also take corresponding alarm measures according to the severity of the abnormality and automatically generate a detailed abnormality report. This intelligent exception handling mechanism greatly improves the enterprise's response speed and processing efficiency to abnormal situations, and effectively reduces the risk of asset management.

[0089] Finally, it is worth mentioning that the system of the present invention has good scalability and adaptability. Through modular design, the system can be customized and expanded according to the specific needs of the enterprise. For example, the image recognition algorithm can be adjusted according to the asset type of the enterprise, or the exception handling strategy can be adjusted according to the management process of the enterprise. This flexibility enables the system to adapt to the needs of enterprises of different sizes and industries.

[0090] In summary, the computer vision-based enterprise asset intelligent inventory system and method provided by the present invention effectively solves many problems existing in the existing asset management system through innovative technical solutions and system design. It not only improves the efficiency and accuracy of asset management, but also realizes real-time monitoring and intelligent management of assets, bringing revolutionary changes to enterprise asset management. The application of this system will greatly improve the operational efficiency and management level of enterprises, and provide strong support for enterprises to gain advantages in the fierce market competition. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 It is a logic block diagram of the whole system of the present invention.

[0092] Figure 2 This is a logic block diagram of the asset tag generation module of the present invention.

[0093] Figure 3 It is a logic block diagram of the asset storage module of the present invention.

[0094] Figure 4 This is a logic block diagram of the asset inventory module of the present invention.

[0095] Figure 5 It is a logic block diagram of the abnormal monitoring module of the present invention.

[0096] Figure 6 It is a logic block diagram of the abnormality location module of the present invention. DETAILED DESCRIPTION

[0097] See also Figure 1-6The present invention provides an enterprise asset intelligent inventory system and method based on computer vision. The system includes an asset tag generation module 1, an asset storage module 2, an asset inventory module 3, an abnormality monitoring module 4 and an abnormality positioning module 5. These modules work together to realize the intelligent management and inventory of enterprise assets.

[0098] The asset tag generation module 1 is used to generate an asset tag image. Specifically, the module first generates a QR code containing asset information, then splices the QR code with the asset image, and finally generates a tag image. This tag design that combines the QR code and the physical image of the asset not only contains rich asset information, but also can intuitively display the appearance of the asset, greatly improving the accuracy and efficiency of asset identification.

[0099] The asset storage module 2 is in communication with the asset tag generation module 1, and is used to store the tag image on the server and bind the tag image to the RFID tag. This step lays the foundation for subsequent asset inventory and management. Preferably, the asset storage module 2 can adopt distributed storage technology to improve the reliability and access efficiency of data storage. For example, the Hadoop distributed file system (HDFS) can be used to store a large amount of asset image data, and the HBase database can be used to store the correspondence between asset information and RFID tags.

[0100] The asset inventory module 3 is one of the core modules of the present invention. The module obtains the RFID tag information of the asset through the RFID reader and collects the asset image at the same time. Then, based on the collected asset image, the asset is counted and classified using advanced image recognition technology. In one embodiment of the present invention, the image recognition technology can use a deep learning algorithm, such as a convolutional neural network (CNN). For example, a pre-trained model such as ResNet50 or VGG16 can be used and fine-tuned according to the specific asset type of the enterprise. This method can greatly improve the inventory efficiency while ensuring the recognition accuracy.

[0101] Specifically, the workflow of the asset inventory module 3 can be described as follows:

[0102] 1. The RFID reader scans the asset and obtains the RFID tag information.

[0103] 2. At the same time, high-definition cameras collect images of assets.

[0104] 3. Input RFID information and asset images into the pre-trained deep learning model.

[0105] 4. The deep learning model outputs information such as asset category and quantity.

[0106] 5. Compare the identification results with the asset information in the database.

[0107] 6. Generate inventory report including asset status, location, quantity and other information.

[0108] In a preferred embodiment of the present invention, the training process of the deep learning model can be expressed as:

[0109]

[0110] Among them, θ represents the model parameters, N is the number of training samples, and x i is the input asset image, y i is the corresponding true label, f(x i ; θ) is the predicted output of the model, L is the loss function (such as cross entropy loss), R(θ) is the regularization term, and λ is the regularization coefficient.

[0111] After the asset inventory module 3 completes the inventory, it transmits the inventory data to the abnormal monitoring module 4. The abnormal monitoring module 4 monitors all assets in real time based on preset rules. When an abnormal situation is detected, the module generates an alarm signal. In one embodiment of the present invention, abnormal monitoring can be based on multi-dimensional indicators, such as changes in asset quantity, position displacement, appearance damage, etc. For example, the following rules can be set:

[0112] 1. An asset quantity change of more than 5% triggers an alarm.

[0113] 2. The asset position deviation exceeds the preset area boundary by 10 meters, triggering an alarm.

[0114] 3. An alarm is triggered when the similarity between the asset appearance and the standard image is less than 0.8.

[0115] These thresholds can be adjusted based on the specific needs of the enterprise.

[0116] When the abnormal monitoring module 4 detects an abnormality and generates an alarm signal, the abnormal location module 5 will receive the signal and use the asset image and asset location information to accurately locate the abnormal point. Preferably, abnormal location can be achieved by combining computer vision technology and positioning algorithms. For example, target detection algorithms such as YOLO (You Only Look Once) or Faster R-CNN can be used to locate the position of abnormal assets in the image, and then combined with RFID positioning information to accurately determine the actual physical location of the abnormal assets.

[0117] The system of the present invention realizes intelligent inventory and management of enterprise assets through the coordinated work of these five modules. Compared with the traditional manual inventory method, the present invention has the following advantages:

[0118] 1. Improve inventory efficiency: Through RFID and computer vision technology, the inventory of large amounts of assets can be completed quickly and accurately.

[0119] 2. Reduce human errors: The automated inventory process reduces human intervention and reduces the error rate.

[0120] 3. Real-time monitoring: The system can monitor asset status in real time and detect and locate abnormal situations in a timely manner.

[0121] 4. Data visualization: By generating intuitive inventory reports and abnormal alerts, managers can quickly understand the status of assets.

[0122] Preferably, the entire system can be deployed on a cloud platform to achieve elastic expansion and high availability of resources. For example, cloud service platforms such as Amazon Web Services (AWS) or Alibaba Cloud can be used to build a reliable and efficient asset management system using the computing, storage, network and other resources they provide.

[0123] In a specific embodiment of the present invention, the asset tag generation module 1 includes a QR code generator 11, a text information extractor 12, a QR code conversion module 13 and a first image stitcher 14. The coordinated work of these submodules makes the asset tag generation process more intelligent and efficient.

[0124] The QR code generator 11 is used to generate a QR code according to the asset information. The present invention preferably uses QR code as the QR code format because QR code has a higher information density and stronger fault tolerance. For example, a 40x40 QR code can store up to 4,296 alphanumeric characters, which is enough to contain detailed asset information.

[0125] The text information extractor 12 is connected to the QR code generator 11 for extracting asset text information from the enterprise management system. The enterprise management system here can be an ERP (Enterprise Resource Planning) system or a special asset management system. The text information extractor 12 can obtain asset information through an API interface or a database query. Typical asset information may include:

[0126] 1. Asset number (e.g. A12345);

[0127] 2. Asset name (e.g. Dell Latitude 5420 laptop);

[0128] 3. Asset type (e.g. electronic equipment);

[0129] 4. Purchase date (e.g. 2023-01-15);

[0130] 5. Current using department (e.g. R&D department);

[0131] 6. Estimated service life (e.g. 5 years).

[0132] The two-dimensional code conversion module 13 is connected to the two-dimensional code generator 11 for converting the generated two-dimensional code into an image format. Usually, the two-dimensional code is converted into an image file in PNG or JPEG format for subsequent image processing and storage. During the conversion process, an appropriate resolution and error correction level can be set to ensure the readability of the two-dimensional code. For example, the two-dimensional code image can be set to 300x300 pixels and the error correction level of M level (15%) can be adopted, which can ensure clarity and tolerate damage to the two-dimensional code to a certain extent.

[0133] The first image stitching module 14 is in communication with the two-dimensional code conversion module 13 and is used to stitch the converted two-dimensional code image with the asset image to generate a final label image. The stitching process can be implemented using an image processing library such as OpenCV.

[0134] The asset image occupies the upper half of the label, the lower half is a QR code on the left, and some key text information (such as asset number and name) on the right. This layout is both intuitive and informative, making it easy for both humans and machines to identify.

[0135] Through this modular design, the asset tag generation module 1 can efficiently generate asset tags containing rich information, laying a foundation for subsequent asset management and inventory work.

[0136] The asset storage module 2 is a key link in the system of the present invention that connects asset tag generation and actual asset management. The module includes an image storage 21 and a data binder 22, which work together to ensure the integrity and traceability of asset information.

[0137] The image storage 21 is used to store the label image in a specified location of the server. In practical applications, considering that an enterprise may have a large number of assets, the image storage 21 needs to be able to efficiently process and store a large amount of image data. The present invention preferably uses a distributed file system to implement the function of the image storage 21. For example, the Hadoop Distributed File System (HDFS) can be used as the underlying storage system. HDFS has the characteristics of high fault tolerance, high scalability and high throughput, and is very suitable for storing and processing a large number of asset label images.

[0138] In HDFS, asset images can be organized according to a certain directory structure. This structured storage method facilitates subsequent retrieval and management.

[0139] The data binder 22 is connected to the image storage 21 for associating the storage location information of the tag image with the RFID tag and storing the binding information in the database. This process is a key step in associating the physical asset with the digital information. The data binder 22 can use a relational database (such as MySQL) or a NoSQL database (such as MongoDB) to store the binding information. The specific choice can be determined based on the actual needs of the enterprise and the existing IT infrastructure.

[0140] A typical data binding record might contain the following fields:

[0141] 1. RFID tag ID (primary key);

[0142] 2. Asset number;

[0143] 3. Label image storage path;

[0144] 4. Bind timestamp;

[0145] 5. Operator ID;

[0146] For example, a binding record might look like this:

[0147] {

[0148] "rfid_id":"E2003411B802011023456789",

[0149] "asset_id":"A12345",

[0150] "image_path":" / assets / electronics / computers / A12345.jpg",

[0151] "bind_time":"2023-05-2014:30:22",

[0152] "operator_id":"OP001"

[0153] }

[0154] To improve the reliability and performance of the system, the data binder 22 may adopt the following strategies:

[0155] 1. Use database transactions to ensure the atomicity of the binding operation between the RFID tag and the image path.

[0156] 2. Optimize database indexes to improve query efficiency. For example, you can create indexes for RFID tag ID and asset number fields.

[0157] 3. Use database master-slave replication mechanism to improve data availability and reading performance.

[0158] 4. Implement regular data backup and recovery mechanism to prevent data loss.

[0159] Through these designs of the asset storage module 2, the system of the present invention can effectively manage a large amount of asset tag information and provide a reliable data basis for subsequent asset inventory and abnormal monitoring. At the same time, this design also has good scalability and can adapt to the growth of the enterprise's asset scale and changes in management needs. The asset inventory module 3 of the present invention is the core component of the entire system, which includes an RFID reader 31, a second image stitcher 32, an image recognizer 33 and a classifier 34. The collaborative work of these submodules makes the asset inventory process more efficient and accurate.

[0160] The RFID reader 31 is used to read the RFID tag information of the asset. In practical applications, the RFID reader 31 can be a handheld device or a fixed device, and the specific choice depends on the inventory needs and asset distribution of the enterprise. For example, for large warehouses or factories, fixed RFID readers can be installed in key locations to achieve automated asset tracking; while for office environments, handheld RFID readers may be more flexible and convenient. In a preferred embodiment of the present invention, the RFID reader 31 uses UHF (ultra-high frequency) technology, with an operating frequency range of 860-960MHz and a reading distance of up to 3-10 meters. This configuration can meet the needs of most enterprise asset inventory.

[0161] The second image stitcher 32 is in communication with the RFID reader 31 and is used to stitch the captured asset image with the corresponding tag image obtained from the database to generate a tagged asset image. The purpose of this step is to compare the real-time captured asset image with the previously generated standard tag image for subsequent image recognition and anomaly detection. The stitching process can be implemented using image processing libraries such as OpenCV. Preferably, the stitched images are arranged side by side, with the real-time captured asset image on the left and the standard tag image on the right for intuitive comparison.

[0162] The image recognizer 33 is in communication with the second image stitcher 32 and is used to perform image recognition processing on the tagged asset image. The system of the present invention uses deep learning technology to achieve image recognition. Specifically, the image recognizer 33 can use a pre-trained convolutional neural network (CNN) model, such as ResNet50 or VGG16, and fine-tune it according to the enterprise-specific asset type. The image recognition process can be expressed as the following mathematical formula:

[0163] F(I)=f n (f n-1 (...f2 (f 1 (I)))),

[0164] Where I represents the input label asset image, f i represents the i-th layer transformation of the neural network, and F(I) represents the final feature representation.

[0165] In order to improve the recognition accuracy, the system of the present invention also uses data enhancement technology. During the model training stage, the original image is randomly rotated, scaled, flipped, and other operations are performed to generate more training samples. This method can improve the generalization ability of the model and enable it to adapt to asset images at different angles and lighting conditions.

[0166] The classifier 34 is in communication with the image recognizer 33 and is used to classify the counted assets based on the image recognition results. The classifier 34 uses a softmax function to map the feature vector output by the image recognizer to different asset categories. The classification process can be expressed as:

[0167]

[0168] Among them, x is the image feature vector, y i is the i-th category, z i is the score corresponding to category i, and K is the total number of categories.

[0169] The system of the present invention can quickly and accurately identify and classify various types of corporate assets through this deep learning-based image recognition and classification method, greatly improving inventory efficiency and accuracy.

[0170] The abnormal monitoring module 4 is a key component responsible for real-time monitoring of asset status in the system of the present invention. The module includes a first image processing device 41, a second image processing device 42, a data transmitter 43 and an abnormal alarm device 44. These submodules work together to achieve comprehensive and real-time monitoring of enterprise assets.

[0171] The first image processing device 41 is used to pre-process the asset image. The pre-processing step includes operations such as image denoising, brightness adjustment, and contrast enhancement, with the purpose of improving the accuracy of subsequent image analysis. In a preferred embodiment of the present invention, the first image processing device 41 uses an image enhancement algorithm based on deep learning, such as a generative adversarial network (GAN). This method can effectively improve image quality, especially in poor lighting conditions or low image resolution.

[0172] The second image processing device 42 is used to extract features from the tagged asset image. Feature extraction is a key step in image analysis and directly affects the effect of subsequent anomaly detection. The system of the present invention adopts a multi-scale feature extraction strategy, combining traditional computer vision algorithms (such as SIFT, SURF) and deep learning methods (such as the intermediate layer features of VGGNet and ResNet). This hybrid method can capture the global and local features of asset images and improve the accuracy and robustness of anomaly detection.

[0173] The data transmitter 43 is connected to the first image processing device 41 and the second image processing device 42 for transmitting the processed image data. Considering that the enterprise asset management system may need to process a large amount of image data, the data transmitter 43 adopts an efficient data compression and transmission protocol. For example, the JPEG2000 image compression standard can be used, which provides a higher compression rate and better image quality than the traditional JPEG. At the same time, the data transmitter 43 also implements a data encryption function to ensure the security of sensitive asset information during transmission.

[0174] The abnormality alarm device 44 is connected to the data transmitter 43 for comparing the processed image data based on preset rules and generating an alarm signal when an abnormality is detected. The abnormality detection algorithm adopts an autoencoder model based on deep learning. The autoencoder can effectively detect abnormal situations by learning the feature distribution of normal asset images. The abnormality detection process can be expressed as:

[0175] S(x)=||xD(E(x))||,

[0176] Where x is the input image, E(x) is the encoder function, D(E(x)) is the decoder function, and S(x) is the anomaly score. When S(x) exceeds the preset threshold, the system triggers an alarm.

[0177] The system of the present invention also introduces an adaptive threshold mechanism to dynamically adjust the anomaly detection threshold according to the characteristics of different types of assets. This method can effectively reduce the false alarm rate and improve the accuracy of anomaly detection.

[0178] The abnormality positioning module 5 is an important component of the system of the present invention responsible for accurately positioning abnormal assets. The module includes a second image separator 51, a data processor 52 and a precise locator 53. The coordinated work of these submodules enables the system to quickly and accurately locate abnormal assets.

[0179] The second image separator 51 is used to separate the tagged asset image into an asset image and a tagged image. The purpose of this step is to separate the real-time acquired asset image from the standard tagged image for more detailed comparison and analysis. The separation process uses an image segmentation algorithm, such as a U-Net model based on deep learning. The U-Net model has good edge preservation capabilities and can accurately segment the asset and tag areas.

[0180] The data processor 52 is connected to the second image separator 51 for data processing of the separated asset image and label image. The data processing includes steps such as feature extraction and feature matching. In a preferred embodiment of the present invention, the feature extraction adopts the SIFT (Scale-Invariant Feature Transform) algorithm, which has good invariance to the rotation, scaling and brightness change of the image. The feature matching adopts the FLANN (Fast Library for ApproximateNearest Neighbors) algorithm, which can quickly find the most similar feature point pairs.

[0181] The precise locator 53 is in communication with the data processor 52 and is used to accurately locate the abnormal point based on the processed image data and asset location information. The positioning process combines the image analysis results and the RFID positioning information. Specifically, the system first determines the location of the abnormal asset in the image through image analysis, and then estimates the actual physical location of the asset using the RFID signal strength information. The fusion of these two types of information can be achieved through the Kalman filter, and its mathematical model can be expressed as:

[0182] x k =Fx k-1 +Bu k +w k ,

[0183] z k =Hx k +v k ,

[0184] Among them, x k is the real location status of the asset, z k is the observation value (including image analysis and RFID positioning results), F is the state transfer matrix, H is the observation matrix, and w k and v k are process noise and observation noise respectively.

[0185] Through this multi-source information fusion method, the system of the present invention can accurately locate abnormal assets in a complex enterprise environment, providing strong support for managers to handle abnormal situations in a timely manner.

[0186] The system of the present invention further comprises an asset location determination module 7, which is in communication with the asset inventory module 3. The introduction of this module greatly improves the system's ability to track and manage asset locations, especially in large warehouses or complex factory environments.

[0187] The working principle of the asset location determination module 7 is based on RFID signal strength analysis and triangulation. Specifically, the module first obtains the geographic coordinate information of the three RFID readers with the highest signal strength with the RFID tag of the asset to be located. These RFID readers are usually fixedly installed at key locations of the enterprise, and their precise coordinates are known in advance.

[0188] Next, the system projects the geographic coordinate information of the three RFID readers onto an electronic map. The electronic map can be a floor plan or a three-dimensional model of the enterprise, depending on the actual needs of the enterprise and the distribution of its assets. The projection process uses a coordinate transformation algorithm to convert the global coordinate system (such as GPS coordinates) into a local coordinate system.

[0189] Then, the system calculates the intersection of the lines connecting the three RFID readers. In theory, these three lines should intersect at one point, which is the location of the asset to be located. However, due to factors such as signal interference and multipath effects, a small triangle may actually be formed. The system of the present invention uses a weighted centroid method to determine the final positioning point, namely:

[0190]

[0191] Among them, (x i ,y i ) are the coordinates of the three intersection points, w i is the corresponding weight, which can be determined according to the RFID signal strength.

[0192] Finally, the system determines the calculated intersection point as the geographic coordinates of the asset to be located. The positioning accuracy of this method can usually reach the meter level, which is sufficient for use in most enterprise asset management scenarios.

[0193] In order to further improve the positioning accuracy, the system of the present invention also introduces a Kalman filter algorithm to fuse and smooth multiple positioning results. This method can effectively reduce positioning jitter and provide more stable and accurate position information.

[0194] By introducing the asset location determination module 7, the system of the present invention can not only realize the intelligent inventory of assets, but also provide real-time location information of assets. This is of great significance for improving the asset utilization rate of enterprises, optimizing asset layout, and preventing asset loss. In the system of the present invention, the image recognizer 33 uses an improved convolutional neural network for image recognition, which is the key to the efficient and accurate operation of the entire system. The improved convolutional neural network includes an input layer, multiple convolutional layers, multiple pooling layers, a fully connected layer and an output layer, and its structure is carefully designed to meet the special needs of enterprise asset management.

[0195] The input layer is used to receive the label asset image data. In the preferred embodiment of the present invention, the input image is uniformly adjusted to 224x224 pixels and normalized to ensure data consistency and network stability. The design of the input layer takes into account the diversity of enterprise assets and can adapt to asset images of different types and sizes.

[0196] Multiple convolutional layers are connected to the input layer to extract features from the image data. The size and number of convolution kernels in each convolutional layer can be adjusted dynamically. This flexible design enables the network to adapt to asset recognition tasks of different complexity. For example, for assets with rich texture details, the number of small-sized convolution kernels can be increased; while for assets with obvious shape features, larger-sized convolution kernels can be used. The convolution operation can be expressed as:

[0197]

[0198] Where f is the input image, g is the convolution kernel, and * represents the convolution operation. Multiple pooling layers are alternately connected with the convolution layer to reduce the dimension of the feature map and extract the main features. The system of the present invention adopts the maximum pooling operation, which can effectively retain important feature information. The pooling operation can be expressed as:

[0199]

[0200] Among them, s is the size of the pooling window, and (i, j) is the coordinate of the output feature map.

[0201] The fully connected layer is connected to the last pooling layer to map the extracted features to the classification space. In an embodiment of the present invention, the fully connected layer uses the dropout technology to randomly discard a part of neurons to prevent overfitting. The dropout operation can be expressed as:

[0202] y=r·(W·x+b),

[0203] Among them, r is a binary random variable that follows a Bernoulli distribution, W is the weight matrix, and b is the bias term.

[0204] The output layer is connected to the fully connected layer to output the asset classification results. The output layer uses the softmax function to convert the feature vector into a probability distribution of each category:

[0205]

[0206] Among them, x is the input feature, y i is the i-th category, z i is the score corresponding to category i, and K is the total number of categories.

[0207] It is worth noting that the system of the present invention uses ReLU (Rectified Linear Unit) as the activation function in both the convolutional layer and the fully connected layer. The ReLU function can be expressed as:

[0208] f(x)=max(0,x)

[0209] The use of the ReLU function greatly speeds up network training and alleviates the gradient vanishing problem, enabling deep networks to learn effectively.

[0210] Through this improved convolutional neural network structure, the system of the present invention can quickly and accurately identify various types of enterprise assets, providing strong technical support for asset management.

[0211] The system of the present invention further comprises an asset abnormality alarm module 9, which is in communication connection with the abnormality monitoring module 4 and the abnormality positioning module 5. The introduction of this module enables the system to handle abnormal situations more intelligently, and provides timely and effective decision support for enterprise managers.

[0212] The asset abnormality alarm module 9 first receives the abnormality information sent by the abnormality monitoring module 4 and the abnormality location information sent by the abnormality positioning module 5. This information includes the abnormality type, abnormality degree, specific location of abnormal assets, etc.

[0213] Next, based on the preset abnormality level rules, the abnormalities are graded. In the preferred embodiment of the present invention, the abnormality levels are divided into three levels: slight abnormality, moderate abnormality and severe abnormality. The grading standards can be customized according to the specific needs of the enterprise. For example, for assets with higher value, even slight abnormalities may be classified as moderate or severe abnormalities.

[0214] Anomaly classification can be implemented using fuzzy logic algorithm, and its membership function can be expressed as:

[0215]

[0216] Among them, x is the quantitative value of the abnormality degree, and a and b are thresholds set according to specific circumstances.

[0217] According to the abnormality level, the system selects the corresponding alarm method. Alarm methods include but are not limited to SMS reminders, email notifications, system pop-up windows, and sound and light alarms. Different levels of abnormalities correspond to different alarm strategies, for example:

[0218] Mild abnormality: system pop-up window reminder;

[0219] Moderate abnormality: SMS and email notification to relevant persons in charge;

[0220] Serious abnormality: triggering sound and light alarms, and notifying management through multiple channels;

[0221] The alarm information is sent using an asynchronous processing mechanism to ensure that the system can still respond quickly when a large number of exceptions occur simultaneously. Preferably, the system uses a message queue (such as RabbitMQ or Apache Kafka) to manage alarm information to improve the reliability and scalability of the system.

[0222] Finally, the asset abnormality alarm module 9 is also responsible for generating abnormality reports and sending them to designated managers. The abnormality report includes detailed information on abnormal assets, the time and location of the abnormality, a specific description of the abnormality, measures taken, and recommended treatment methods. The report is generated using a template engine (such as Jinja2), which can quickly generate a report document with a standardized format and detailed content.

[0223] By introducing the asset abnormality alarm module 9, the system of the present invention can not only accurately identify and locate abnormal assets, but also take corresponding alarm measures according to the severity of the abnormality and generate a detailed abnormality report. This greatly improves the enterprise's response speed and processing efficiency to asset abnormalities and minimizes the risk of asset loss.

[0224] The present invention also provides an enterprise asset intelligent inventory method based on the above system. The method comprises the following steps:

[0225] S100: Textually process the enterprise assets to obtain textual information. This step is usually completed by the enterprise's ERP system or special asset management software. The textual information includes but is not limited to the asset number, name, type, purchase date, using department, etc.

[0226] S200: Generate a label image using the asset label generation module 1, store it in the image storage 21, and bind it to the RFID label. This step realizes the visualization and electronicization of asset information, laying a foundation for the subsequent intelligent inventory.

[0227] S300: Inventory the assets through the RFID reader 31. This includes the following sub-steps:

[0228] S310, reading the RFID tag through the RFID reader 31. This step can quickly obtain basic information of the asset.

[0229] S320, identifying and locating the RFID tag by image recognition. This step combines computer vision technology to improve the accuracy of RFID positioning.

[0230] S330: According to the RFID tag, the corresponding asset name, specification and quantity are obtained, and the overall asset name and inventory quantity are counted. This step realizes the rapid aggregation of asset information.

[0231] S340, classify the assets through the convolutional neural network classification model to obtain different types of assets and their quantities. This step uses deep learning technology to achieve automatic classification of assets.

[0232] S400: Monitor the asset status in real time through the abnormal monitoring module 4. If an abnormal situation occurs, the system will immediately trigger an alarm. This step ensures the real-time and accuracy of asset management.

[0233] S500, the abnormal location module 5 locates the abnormal point of the asset position according to the monitoring alarm record. Specifically, it includes the following sub-steps:

[0234] S510 , the abnormality positioning module 5 receives the monitoring alarm information and transmits the abnormal label image to the data processor 52 .

[0235] S520: The data processor 52 analyzes the monitoring alarm information to obtain the location of the abnormal product.

[0236] S530: Find out the asset tag image information of the abnormal type according to the abnormal position.

[0237] S540. Index the abnormal asset image information in a database according to the abnormal asset tag image information.

[0238] S550. Use improved image recognition technology to identify the image of the asset to be counted and locate abnormal positions of the assets.

[0239] Through this systematic method, the present invention realizes intelligent and automated inventory of enterprise assets, greatly improves the efficiency and accuracy of inventory, and also provides strong support for abnormal monitoring and management of enterprise assets. Each step of the method has been carefully designed, making full use of advanced technologies such as RFID technology, computer vision, and deep learning, reflecting the innovation and practicality of the present invention.

[0240] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. The enterprise asset intelligent inventory system based on computer vision is characterized by: include: Asset tag generation module for: Generate a QR code containing asset information; Splicing the QR code with the asset image to generate a label image; The asset storage module is connected to the asset tag generation module for: Receiving the label image sent by the asset label generation module; storing the label image on a server; Binding the label image to the RFID label; The asset inventory module is connected to the asset storage module for: Obtain the RFID tag information of the asset through the RFID reader; Collect asset images; Based on the asset images, inventory and classify the assets using image recognition technology; Generate inventory data and transmit it to the abnormal monitoring module; The abnormal monitoring module is connected to the asset inventory module for: Receiving the inventory data sent by the asset inventory module; Real-time monitoring of all assets based on preset rules; When an abnormal situation is detected, an alarm signal is generated; The abnormality location module is connected to the abnormality monitoring module for: Receiving an alarm signal sent by the abnormality monitoring module; Based on the alarm signal, the abnormal point is accurately located using the asset image and asset location information.

2. The system according to claim 1, characterized in that The asset tag generation module includes: A QR code generator, used to generate a QR code based on asset information; A text information extractor, in communication with the QR code generator, for extracting asset text information from the enterprise management system; A two-dimensional code conversion module, which is in communication with the two-dimensional code generator and is used to convert the generated two-dimensional code into an image format; The first image stitcher is connected to the two-dimensional code conversion module for stitching the converted two-dimensional code image with the asset image to generate a final label image.

3. The system according to claim 1, characterized in that The asset storage module includes: An image storage device is used to store the label image in a designated location of the server; The data binder is connected to the image storage device for associating and binding the storage location information of the label image with the RFID label, and storing the binding information in a database.

4. The system according to claim 1, characterized in that The asset inventory module includes: RFID reader, used to read the RFID tag information of the asset; a second image stitcher, in communication with the RFID reader, for stitching the collected asset image with the corresponding tag image obtained from the database to generate a tagged asset image; An image recognizer, which is in communication with the second image stitcher and is used to perform image recognition processing on the label asset image; A classifier is connected in communication with the image recognizer and is used to classify the counted assets based on the image recognition results.

5. The system according to claim 1, characterized in that The abnormal monitoring module includes: A first image processing device, used for pre-processing the asset image; A second image processing device, used for extracting features from the tagged asset image; A data transmitter, connected to the first image processing device and the second image processing device for transmitting processed image data; The abnormality alarm device is connected to the data transmitter for comparing the processed image data based on preset rules and generating an alarm signal when an abnormality is detected.

6. The system according to claim 1, characterized in that The abnormality positioning module includes: a second image separator, for separating the label asset image into an asset image and a label image; a data processor, connected in communication with the second image separator, and configured to perform data processing on the separated asset image and label image; A precise locator is communicatively connected to the data processor and is used to accurately locate the abnormal point based on the processed image data and asset location information.

7. The system according to claim 1, characterized in that It also includes an asset location determination module, which is in communication with the asset inventory module and is used to: Obtain the geographic coordinate information of the three RFID readers with the highest signal strength with the RFID tag of the asset to be located; Projecting the geographic coordinate information of the three RFID readers on an electronic map respectively; Calculate the intersection points of the lines connecting the three RFID readers in pairs; The intersection point is determined as the geographic coordinates of the asset to be located.

8. The system according to claim 4, characterized in that The image recognizer uses an improved convolutional neural network to perform image recognition, and the improved convolutional neural network includes: Input layer, used to receive label asset image data; A plurality of convolutional layers, connected to the input layer, for extracting features from the image data, wherein the size and number of convolutional kernels of each convolutional layer can be dynamically adjusted; A plurality of pooling layers, alternately connected with the convolutional layers, are used to reduce the dimension of the feature map and extract the main features; The fully connected layer, connected to the last pooling layer, is used to map the extracted features to the classification space; An output layer, connected to the fully connected layer, for outputting asset classification results; Wherein, the convolutional layer and the fully connected layer both use ReLu as the activation function.

9. The system according to claim 1, characterized in that It also includes an asset abnormality alarm module, which is in communication with the abnormality monitoring module and the abnormality positioning module and is used to: Receive the abnormal information sent by the abnormal monitoring module and the abnormal location information sent by the abnormal location module; Based on the preset abnormality level rules, abnormalities are graded; According to the abnormality level, select the corresponding alarm method, including but not limited to SMS reminder, email notification, system pop-up window and sound and light alarm; An exception report is generated and sent to a designated manager.

10. An enterprise asset intelligent inventory method based on the system according to any one of claims 1 to 9, characterized in that: The following steps are involved: S100, textual processing is performed on the enterprise assets to obtain textual information; S200, using an asset tag generation module to generate a tag image, storing it in an image memory, and binding it to an RFID tag; S300: Inventory the assets using the RFID reader, including: S310, reading the RFID tag through the RFID reader; S320, identifying and locating the RFID tag by image recognition; S330, according to the RFID tag, obtain the corresponding asset name, specification and quantity, and calculate the overall asset name and inventory quantity; S340, classifying the assets through a convolutional neural network classification model to obtain different types of assets and their quantities; S400, monitor the asset status in real time through the abnormal monitoring module, and alarm if an abnormal situation occurs; S500, the abnormal location module locates the abnormal points of the asset location according to the monitoring alarm records, including: S510, the abnormality positioning module receives the monitoring alarm information and transmits the abnormal label image to the data processor; S520, the data processor analyzes the monitoring alarm information to obtain the location of the abnormal product; S530, according to the abnormal position, find out the asset tag image information of the abnormal type; S540, indexing abnormal asset image information in a database according to abnormal asset tag image information; S550. Use improved image recognition technology to identify the image of the asset to be counted and locate abnormal positions of the assets.

Images