An asset status identification system and method based on RFID and machine vision

Abstract

The present invention discloses an asset status identification system based on RFID and machine vision and a method thereof. The system comprises an artificial intelligence processor, and a binocular camera, a pan / tilt, a reader / writer, a Wi-Fi module, an SD card and an asset management server connected to the artificial intelligence processor; the method comprises: 1) constructing a 7-bit binary asset status identification code and a disposal priority through the artificial intelligence processor and initializing parameter information according to task information; 2) according to the storage location information of the target asset, the artificial intelligence processor controls the asset status identification system to reach the vicinity of the target asset, and performs image recognition of the target asset based on the binocular view obtained by the binocular camera module; 3) a label image capture process; 4 a label appearance and response performance evaluation process; 5) an asset status identification report generation; the present invention improves the automation level and accuracy of asset management by combining machine vision technology with a comprehensive evaluation method.

Description

Technical Field

[0001] The present invention relates to the technical field of asset management, and in particular to an asset status identification system and method based on RFID and machine vision. Background Art

[0002] Compared with traditional barcode technology that can only identify one type of asset, RFID (Radio Frequency Identification) electronic tag technology can uniquely identify a single asset, so modern asset management systems often use RFID technology. At the same time, asset management systems that use RFID tags have more comprehensive control over asset location, status and other information, and are more conducive to achieving full life cycle management of assets from procurement to scrapping.

[0003] RFID tags are often attached to the outer shell of an asset by adhesion. When performing an asset inventory, the reader identifies the RFID tag through radio frequency signals and obtains relevant data of the identified asset. Depending on whether there is a built-in power supply, RFID tags can be divided into passive tags and active tags. Regardless of the type of tag, there are problems such as obstruction, performance degradation, external surface deformation, cracks or obvious wear, adhesive aging and shedding, and loss, which bring troubles to the asset inventory work. Relying solely on the reader cannot distinguish the above-mentioned abnormal conditions of the tag.

[0004] Existing asset management technologies based on RFID tags focus on the application of tags in asset management, and pay less attention to the above-mentioned problems encountered in the long-term operation of asset management systems. For example, patent CN202310470348.6 combines RFID tags with dynamic management of tower cranes to achieve precise management of assets. Patent CN202010430961.1 combines blockchain technology to solve asset dispute issues. These existing technologies cannot effectively distinguish between abnormal asset locations and abnormal complex tag situations.

[0005] Today, machine vision technology is very mature and can be easily deployed on lightweight artificial intelligence processors. The comprehensive application of these technologies can provide new solutions to the problems faced by asset management systems in the long term, effectively improve the automation level and accuracy of asset management, and reduce manual intervention. Summary of the invention

[0006] In view of the difficulties in the prior art, the present invention provides an asset status identification system and method based on RFID and machine vision to solve the problems of abnormal asset location and abnormal complex tag identification faced by the asset management system in the long term.

[0007] In order to solve the problems of the prior art, the present invention adopts the following technical solutions:

[0008] An identification method of an asset status identification system based on RFID and machine vision, the asset status identification system based on RFID and machine vision comprises an artificial intelligence processor, and a binocular camera, a pan-tilt, a reader / writer, a Wi-Fi module, an SD card and an asset management server connected to the artificial intelligence processor; wherein: the artificial intelligence processor comprises an asset image recognition module, a tag recognition module, a tag response module, a tag appearance inspection module and a tag performance evaluation module; the artificial intelligence processor is used to receive and execute asset status identification tasks; the binocular camera module is used to collect target assets and their tag images and send the collected data to the artificial intelligence processor; the pan-tilt receives an angle rotation command issued by the artificial intelligence processor to adjust the viewing angle of the binocular camera module;

[0009] The reader receives a tag reading command issued by the artificial intelligence processor, is used to read the target asset RFID tag information, and sends the tag reading information to the artificial intelligence processor; the Wi-Fi module is used for wireless communication between the artificial intelligence processor and the asset management server; the SD card is used to store asset status identification processing data; wherein: the identification method includes the following steps:

[0010] 1) Parameter initialization: The AI ​​processor constructs a 7-bit binary asset status identification code, disposal priority, and initializes parameter information based on task information;

[0011] 2) Asset image capture: Based on the target asset storage location information, the AI ​​processor controls the asset status recognition system to arrive near the target asset and performs image recognition of the target asset based on the binocular view obtained by the binocular camera module;

[0012] 3) Tag image capture: determine whether the left and right views of the target asset match the asset image in the asset management server through image similarity. If the conditions are met, set the asset image flag to 1, output the left and right views with overlapping area marks, store them in the SD card, and proceed to step 4); otherwise, reacquire the left and right views of the target asset by turning the pan / tilt, execute step 2.1), and set the asset image flag to 0;

[0013] 4) Evaluation of label appearance and response performance, including the following steps:

[0014] 4.1) Image capture judgment: Use the tag recognition model to determine whether there is a tag image in the left and right views of the target asset with the overlapping area; if both exist, set the tag image flag to 1, and mark the identified tags in the left and right views respectively, store the left and right views with the tag marks to the SD card, and go to step 4.2); otherwise, set the tag image flag to 0 and go to step 4.3);

[0015] 4.2) Distance adjustment: read the left and right eye views with tag marks from the SD card, calculate the distance between the asset status identification system and the tag through the binocular distance measurement algorithm, and go to step 4.3) when this distance is equal to the tag response detection reference distance, otherwise control the asset status identification system to move, repeat steps 2), 3) and distance calculation until the distance between the asset status identification system and the tag is equal to the tag response detection reference distance;

[0016] 4.3) Response detection: After the reader sends the response test signal, read the identification codes of all responding tags and make a judgment: if the RFID tag identification code corresponding to the target asset is received, set the tag response flag to 1, and increase the number of times the tag has responded by 1. If the tag image flag is also 1 at the same time, go to step 4.4); otherwise, go to step 5);

[0017] 4.4) Appearance detection: Generate a labeled binocular image based on the left and right eye views with label marks and perform preprocessing; then call the label appearance anomaly detection model to perform appearance classification, and set the label appearance flag according to the classification result;

[0018] 4.5) Response performance evaluation: Based on the benchmark tag response performance index database and the tag performance comprehensive evaluation model of the superior and inferior solution distance method, the tag response performance level is evaluated to obtain the tag performance comprehensive evaluation value and performance level evaluation result, and the tag performance flag is set;

[0019] 5) Generation of asset status identification report: The artificial intelligence processor determines the status of assets and tags based on the asset status identification code, sets disposal priority and feedback information, sends the disposal priority, feedback information and related images stored in the SD card to the asset management server through the Wi-Fi module, and generates a task report and stores it in the SD card.

[0020] Further, the asset status identification code in step 1) is used to identify the target asset and the corresponding tag status; it means: the initialization asset status identification code is 0000000, and the disposal priority is 0; the asset status identification code is that the first bit is the asset image flag, the second bit is the tag image flag, the third bit is the tag response flag, the fourth and fifth bits are combined as the tag appearance flag, and the sixth and seventh bits are combined as the tag performance flag; wherein:

[0021] Initialize parameters based on task information, including: target asset storage location, asset baseline image, asset image similarity threshold, and RFID tag response detection baseline distance corresponding to the target asset, tag response detection signal power value, tag identification code, number of times the tag has responded, and tag standard response signal strength value.

[0022] Furthermore, in step 2), according to the target asset storage location information, the artificial intelligence processor controls the asset status recognition system to arrive near the target asset, and performs an image recognition process of the target asset based on the binocular view obtained by the binocular camera module, including the following steps:

[0023] 2.1) Image acquisition: The AI ​​processor instructs the binocular camera module to capture images of the target asset and receives the left and right eye views output by the binocular camera module;

[0024] 2.2) Image recognition: Based on the asset reference image, the AI ​​processor calculates the image similarity according to the following formula to determine the overlapping areas where the left and right views match the target asset image features.

[0025]

[0026] In formula (1), SI is the similarity between two images, T mat is the number of successfully matched corner features, T total is the total number of corner features in the asset benchmark image;

[0027] If the similarities are all greater than the asset image similarity threshold, then the asset image flag is set to 1, and the left and right views with overlapping area marks are output, that is, the left and right views of the target asset, and they are stored in the SD card, and step 3) is executed; otherwise, the asset image flag is set to 0, and step 2.3) is entered;

[0028] 2.3) PTZ rotation: The AI ​​processor instructs the PTZ to rotate 30 degrees clockwise, and determines whether the PTZ has rotated 360 degrees clockwise. If so, the asset image flag is set to 0 and the process goes to step 4); otherwise, the process goes to step 2.1).

[0029] Furthermore, the label appearance flag in step 4.4) includes:

[0030] (a) The label appearance is normal, and the label appearance mark position is 11;

[0031] (b) The label appearance is contaminated or partially blocked, and the label appearance mark position is 10;

[0032] (c) The label has deformation, cracks or obvious wear and tear, and the label appearance mark position is 01;

[0033] (d) There are multiple anomalies in the label appearance at the same time, and the label appearance mark position is 00.

[0034] Furthermore, in step 4.5), based on the benchmark tag response performance index database and the tag performance comprehensive evaluation model of the superior and inferior solution distance method, the tag response performance level is evaluated to obtain the tag performance comprehensive evaluation value and the performance level evaluation result, and the tag performance flag bit process is set, including the following steps:

[0035] 4.51) Construct the original matrix X of label response performance index value:

[0036]

[0037] In the formula, n=m+1, m is the number of benchmark labels, 1 is the number of current labels to be evaluated; x ij (i=1,2,…,n,j=1,2,3,4) is the jth index value of the i-th label; the first m rows of X are the response performance index values ​​of the benchmark labels, and the last row of X is the response performance index value of the current label to be evaluated;

[0038] 4.52) Convert each index value in X into a positive index value and obtain the positive matrix Y:

[0039] α j =max(x 1j ,x 2j ,…,x nj )

[0040] β j =min(x 1j ,x 2j ,…,x nj )

[0041]

[0042] 4.53) Normalize each element of the forward matrix to obtain the standardized matrix Z:

[0043]

[0044] In the formula, z ij is the normalized label response performance indicator value.

[0045] 4.54) Calculate the entropy value and information entropy redundancy of each response performance index:

[0046]

[0047]

[0048] Wherein, C j and g j are respectively the entropy value and the information entropy redundancy of the j-th response performance index.

[0049] 4.55) Calculate the entropy weight w of each response performance index j :

[0050]

[0051] 4.56) Define the ideal optimal value and the ideal worst value

[0052]

[0053] Wherein, and are respectively the maximum value and the minimum value among the z ij values of the j-th standardized index of all tags.

[0054] 4.57) Calculate the Euclidean distance between the standardized tag response performance index value of the i-th tag and the ideal optimal value and the Euclidean distance from the ideal worst value

[0055]

[0056] 4.58) Calculate the comprehensive performance evaluation value S of the i-th tag i :

[0057]

[0058] 4.59) Sort the comprehensive performance evaluation values of all tags from large to small to obtain an evaluation value sequence, retrieve the comprehensive performance evaluation value of the currently to-be-evaluated tag and its ranking r in the evaluation value sequence, and calculate the ranking ratio R:

[0059]

[0060] (4.60) Determine the performance level evaluation result of the to-be-evaluated tag according to the ranking ratio R, and output the comprehensive performance evaluation value S and the performance level evaluation result of the currently to-be-evaluated tag.

[0061] Furthermore, setting the tag performance flag bit according to the tag performance level evaluation result in step 4.5) includes:

[0062] (a) When the evaluation grade is Ⅰ according to 0 < R ≤ 25%, set the tag performance flag bit to 11;

[0063] (b) When 25 < R ≤ 50%, the evaluation level is II, and the label performance mark position is 10;

[0064] (c) When 50 < R ≤ 75%, the evaluation level is III, and the label performance mark position is 01;

[0065] (d) When 75 < R ≤ 100%, the evaluation level is IV, and the label performance mark position is 00.

[0066] Furthermore, in step 5), the artificial intelligence processor judges the conditions of the asset and the label based on the asset condition identification code, and sets the disposal priority and feedback information, including:

[0067] 5.1) Based on the asset condition identification codes 0000000, 0010000, 1100000, 1010000, 1000000, 1111100, 1111000, and 1110000, the set disposal priority is 3, indicating that the asset is not in its original place, or other abnormal events that seriously affect the execution of the asset management task, and manual intervention is required immediately for disposal;

[0068] 5.2) Based on the asset condition identification codes 1111101, 1111001, 1110101, and 1110001, the set disposal priority is 2, indicating that the asset is in its original place, but the label performance has declined, and it may reach the service life limit soon, and enhanced attention and timely replacement are required;

[0069] 5.3) Based on the asset condition identification codes 1111010, 1110110, 1111011, 1110111, and 1110011, the set priority is 1, indicating that the asset is in its original place and the label performance is normal, and the asset management task can be completed, but the outer surface of the label is contaminated or partially blocked and needs to be cleaned;

[0070] 5.4) Based on the asset condition identification codes 1111110 and 1111111; the set priority is 0, indicating that the asset is in its original place and the label performance is normal, and no manual intervention is required.

[0071] Beneficial effects

[0072] Compared with the traditional technical solution, the beneficial effects brought by the present invention are:

[0073] The present invention discloses an asset status identification system and method based on RFID and machine vision. The system and method are based on machine vision technology, and embed a machine vision algorithm and a lightweight embedded self-learning classification model in an artificial intelligence processor to process the images of assets and tags attached thereto, and automatically identify different asset location anomalies and tag appearance anomalies. Based on the benchmark tag response performance index database, the present invention combines historical data and the response performance index value obtained by the reader to form a comprehensive evaluation model for tag performance, and automatically evaluates the tag response performance. At the same time, the present invention uses a machine vision model and a binocular ranging algorithm to automatically control the distance between the asset status identification system and the tag, thereby improving the reliability of the evaluation results of the response performance of the tag to be evaluated.

[0074] The present invention comprehensively applies machine vision technology and comprehensive evaluation methods, which can automatically identify abnormal asset conditions faced in long-term asset management and make disposal priority determinations, further improving the automation level and accuracy of asset management. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a block diagram of an asset status identification system based on RFID and machine vision of the present invention;

[0076] Figure 2 It is a flow chart of an asset status identification method based on RFID and machine vision of the present invention; DETAILED DESCRIPTION

[0077] The following is combined with Figure 1 The present invention is described as follows:

[0078] The following is a detailed description of an asset status identification system and method based on RFID and machine vision of the present invention in conjunction with the embodiments and drawings.

[0079] like Figure 1As shown, an asset status identification system based on RFID and machine vision of the present invention includes an artificial intelligence processor, a binocular camera module, a pan-tilt, a reader / writer, a Wi-Fi module and an SD card connected to the artificial intelligence processor. The artificial intelligence processor is responsible for the control of the asset status identification system, as well as the reception and execution of asset status identification tasks; the binocular camera module is fixed on the pan-tilt, used to collect target assets and RFID tag images, send them to the artificial intelligence processor, and store them in the SD card through the artificial intelligence processor; the pan-tilt receives the angle rotation command issued by the artificial intelligence processor, which is used for viewing angle adjustment and stable shooting of the binocular camera module; the reader / writer receives the tag reading command issued by the artificial intelligence processor, which is used to read the RFID tag information and send the tag reading parameters to the artificial intelligence processor; the Wi-Fi module is used for wireless communication between the artificial intelligence processor and the asset management server; the SD card is used to store programs and data related to asset status identification processing.

[0080] The artificial intelligence processor is an embedded processor with a fast computing unit and a computing power of not less than 0.8TOPS (trillion operations per second) or equivalent, which is deployed with an image matching algorithm, a tag recognition model, a binocular ranging algorithm, a tag appearance anomaly detection model, and a tag performance comprehensive evaluation model. The image matching algorithm is used to achieve the matching of the same elements in two images; the tag recognition model is used to identify the target RFID tag in the image; the binocular ranging algorithm is used to measure the distance between the asset status identification system and the target RFID tag; the tag appearance anomaly detection model is a lightweight embedded self-learning classification model used to detect whether the appearance of the target RFID tag is abnormal; the tag performance comprehensive evaluation model is used to evaluate the current performance of the target RFID tag.

[0081] In the embodiments of the present invention:

[0082] Artificial intelligence processor: You can choose K210, Jeston TX2 or Jeston nano processor;

[0083] Binocular camera module: You can choose OV2640 or OV5640 module;

[0084] PTZ: You can choose SM45BL or SG90 servo PTZ;

[0085] Reader / Writer: You can choose the reader / writer model CPHB701, PRD6 or WYUAN-RD903M;

[0086] Wi-Fi module: You can choose ESP8266, ESP32 or ESP-01 module;

[0087] SD card: You can use an SD card with a capacity greater than or equal to 32g from Lexar, Kingston or SanDisk series products;

[0088] Lightweight embedded self-learning classification model: You can choose YOLOv5s, YOLOv2-Tiny, or ShuffleNet models.

[0089] like Figure 2 As shown, an asset status identification method based on RFID and machine vision of the present invention comprises the following steps:

[0090] 1) Parameter initialization: After receiving the asset status identification task issued by the asset management server, the artificial intelligence processor initializes the asset status identification code to 0000000 and the disposal priority to 0, and initializes the following parameters according to the task information: the target asset storage location, the asset reference image, the asset image similarity threshold, and the RFID tag response detection reference distance corresponding to the target asset, the tag response detection signal power value, the tag identification code, the number of times the tag has responded, and the tag standard response signal strength value.

[0091] The asset status identification code is used to identify the target asset and the corresponding RFID tag status, and is composed of a 7-bit binary number, the first bit is the asset image flag, the second bit is the tag image flag, the third bit is the tag response flag, the fourth and fifth bits are combined as the tag appearance flag, and the sixth and seventh bits are combined as the tag performance flag.

[0092] The disposal priority is used to summarize whether the asset condition is normal and whether manual intervention is required.

[0093] The target asset is the asset to be identified in the task information; the asset storage location is the nearest two-dimensional positioning data of the target asset; the asset reference image is an image with the target asset in the main body and a simple background, which is collected by the asset manager and stored in the asset management database when the asset is first stored; the asset image similarity threshold is a threshold used to determine whether the image is similar to the asset reference image, which is set by the asset manager in the asset management database; the RFID tag corresponding to the target asset refers to an RFID tag that uniquely identifies the target asset and is attached to its surface; the tag response detection reference distance is the reference distance value used in the nominal performance detection of the target tag, which is set by the asset manager in the asset management database; the tag response detection signal power value is the detection signal power value used in the nominal performance detection of the target tag, which is set by the asset manager in the asset management database; the tag identification code is the identification code field stored in the chip inside the RFID tag; the number of times the tag has responded is the number of times the RFID tag corresponding to the target asset has been read recorded in the asset management database; the tag standard response signal strength value is the nominal value of the response signal strength of a good performance tag when the reader uses the tag response detection signal power value to read the tag.

[0094] 2) Asset image capture: Based on the location information of the target asset, the AI ​​processor controls the asset status recognition system to reach the vicinity of the target asset and performs image recognition of the target asset based on the binocular view obtained by the binocular camera module. The specific steps are as follows:

[0095] 2.1) Image acquisition: The AI ​​processor instructs the binocular camera module to capture images of the target asset and receives the left-eye view and right-eye view output by the binocular camera module.

[0096] 2.2) Image recognition: Based on the asset reference image, the artificial intelligence processor calls the image matching algorithm to determine the overlapping areas of the left and right eye views that match the target asset image features, and calculates the image similarity on this basis. If the similarities are both greater than the asset image similarity threshold, the asset image flag is set to 1, and the left and right eye views with overlapping area marks are output, that is, the left and right eye views of the target asset, and they are stored in the SD card and step 3 is executed; otherwise, the asset image flag is set to 0 and step 2.3 is entered. In this embodiment, the image matching algorithm uses the existing image feature matching technology based on Harris corner detection, and the similarity calculation method of the two images is as follows:

[0097]

[0098] In formula (1), SI is the similarity between two images, T mat is the number of successfully matched corner features, Ttotal is the total number of corner features in the asset reference image.

[0099] 2.3) PTZ rotation: The AI ​​processor instructs the PTZ to rotate 30 degrees clockwise, and determines whether the PTZ has rotated 360 degrees clockwise. If so, the asset image flag is set to 0 and the process goes to step 4); otherwise, the process goes to step 2.1).

[0100] 3) Tag image capture: In the marked overlapping areas in the left and right eye views of the target asset, the artificial intelligence processor calls the tag recognition model to recognize the RFID tag. If the tag is recognized in both the left and right eye views, the tag image flag is set to 1, and the recognized tag is marked in the left and right eye views respectively, and then the left and right eye views with the tag are stored in the SD card; otherwise, the tag image flag is set to 0. In this embodiment, the tag recognition model is obtained by training the common machine vision model CNN (Convolutional Neural Network) with all RFID tag images in the asset management database as the training data set.

[0101] 4) Tag appearance and response performance evaluation: The artificial intelligence processor evaluates the tag appearance and response performance through the binocular camera module and the reader / writer. The specific steps are as follows:

[0102] 4.1) Image capture judgment: The artificial intelligence processor judges the tag image flag bit, and goes to step 4.2) when it is 1, and goes to step 4.3) when it is 0.

[0103] 4.2) Distance adjustment: The artificial intelligence processor reads the left and right eye views with the label from the SD card, calls the binocular distance measurement algorithm to calculate the distance between the asset status identification system and the label, and enters step 4.3) when this distance is equal to the label response detection reference distance, otherwise the asset status identification system is controlled to move, and steps 2), 3) and distance calculation are repeated until the distance between the asset status identification system and the label is equal to the label response detection reference distance. In this embodiment, the binocular distance measurement algorithm performs depth calculation based on the parallax principle and uses the existing SGBM (Semi-Global Block Matching) algorithm.

[0104] 4.3) Response detection: The artificial intelligence processor instructs the reader to broadcast a response test signal according to the tag response detection signal power value, sends the identification codes of all response tags to the asset management server through the Wi-Fi module, and makes a judgment: If the RFID tag identification code corresponding to the target asset is received, the tag response flag is set to 1, and the number of times the tag has responded is increased by 1. If the tag image flag is also 1 at the same time, the artificial intelligence processor reads the tag response signal strength value, read-write efficiency, and data transmission rate of this tag, and proceeds to step 4.4); otherwise, proceeds to step 5).

[0105] 4.4) Appearance detection: The AI ​​processor first generates a label binocular image based on the left and right eye views with label marks and stores it in the SD card. It then pre-processes the label binocular image and resizes it to the size required for the classification model input. It then calls the label appearance anomaly detection model to perform appearance classification, and sets the label appearance flag bit based on the classification result. In this embodiment, there are four types of label appearance situations:

[0106] (a) The label appearance is normal, and the label appearance mark position is 11;

[0107] (b) The label appearance is contaminated or partially blocked, and the label appearance mark position is 10;

[0108] (c) The label has deformation, cracks or obvious wear and tear, and the label appearance mark position is 01;

[0109] (d) There are multiple anomalies in the label appearance at the same time, and the label appearance mark position is 00.

[0110] In this embodiment, the label appearance anomaly detection model is obtained by training an existing lightweight embedded self-learning classification model based on establishing four types of label image sample sets: label appearance without abnormalities, with pollution or partial occlusion, with deformation cracks or obvious wear, and with multiple abnormalities at the same time.

[0111] 4.5) Response performance evaluation: Based on the output parameters in step 4.3), the artificial intelligence processor processes the response performance index value of the tag, calls the tag performance comprehensive evaluation model, outputs the tag performance comprehensive evaluation value and performance level evaluation result, and sets the tag performance flag bit according to the evaluation result. In this embodiment, there are four types of tag performance level evaluation results:

[0112] (a) The evaluation level is Ⅰ, and the label performance mark position is 11;

[0113] (b) The evaluation level is II, and the label performance mark position is 10;

[0114] (c) The evaluation level is III, and the label performance mark position is 01;

[0115] (d) The evaluation level is IV and the label performance mark position is 00.

[0116] The response performance index values ​​of the tag include read / write efficiency, data transmission rate, number of responses and tag response signal strength quality. The tag response signal strength quality is calculated based on the tag response signal strength value and the tag standard response signal strength value, and the calculation method is as follows:

[0117]

[0118] In formula (2), Q is the tag response signal strength quality, RSSI is the tag response signal strength value, and RSSI0 is the tag standard response signal strength value.

[0119] The label performance comprehensive evaluation model is based on the benchmark label response performance index database, and evaluates the label performance level based on the TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) algorithm. The benchmark label response performance index database contains multiple groups of benchmark label response performance index values ​​of the same number and model as the target label. The response performance index value of the benchmark label can be obtained by actual label testing, or it can be calculated and generated according to the label performance attenuation law. In this embodiment, the total number of benchmark labels is 48, which are divided into four groups: best performance, basically normal performance, reduced performance, and severely reduced performance. The response performance index value of the benchmark label is calculated and generated according to the label performance attenuation law.

[0120] The label performance level evaluation process is as follows:

[0121] (1) Construct the original matrix X of label response performance index values:

[0122]

[0123] In formula (3), n = m + 1, m is the number of benchmark labels, 1 is the number of current labels to be evaluated; x ij (i=1,2,…,n,j=1,2,3,4) is the jth index value of the ith label; the first m rows of X are the response performance index values ​​of the benchmark labels, and the last row of X is the response performance index value of the current label to be evaluated.

[0124] (2) Convert each index value in X into a positive index value to obtain the positive matrix Y:

[0125] α j =max(x 1j ,x 2j ,…,xnj ) (4)

[0126] β j =min(x 1j ,x 2j ,…,x nj ) (5)

[0127]

[0128]

[0129] (3) Normalize each element of the forward matrix to obtain the standardized matrix Z:

[0130]

[0131] In formula (8), z ij is the normalized label response performance indicator value.

[0132] (4) Calculate the entropy value and information entropy redundancy of each response performance indicator:

[0133]

[0134]

[0135] g j =1-C j (11)

[0136] In formula (10) and (11), C j and g j are the entropy value and information entropy redundancy of the j-th response performance indicator respectively.

[0137] (5) Calculate the entropy weight w of each response performance index j :

[0138]

[0139] (6) The ideal optimal value of each indicator is defined by the maximum value of each standardized label response performance indicator value and the ideal worst value

[0140]

[0141]

[0142] In formula (13) and (14), and are the jth standardized index values ​​z of all labels respectively. ij The maximum and minimum values ​​in .

[0143] (7) Calculate the Euclidean distance between the standardized label response performance index value of the i-th label and the ideal optimal value and the Euclidean distance from the ideal worst value

[0144]

[0145]

[0146] (8) Calculate the comprehensive performance evaluation value S of the i-th tag i :

[0147]

[0148] (9) Sort the comprehensive performance evaluation values ​​of all tags from large to small to obtain an evaluation value sequence, retrieve the comprehensive performance evaluation value of the current tag to be evaluated and its ranking r in the evaluation value sequence, and calculate the ranking ratio R:

[0149]

[0150] (10) Determine the performance level evaluation result of the tag to be evaluated according to the ranking ratio R, and output the comprehensive performance evaluation value S of the current tag to be evaluated and the performance level evaluation result. In this embodiment, the performance level evaluation criteria are shown in Table 1.

[0151] Table 1

[0152]

[0153]

[0154] 5) Generation of asset status identification report: The artificial intelligence processor determines the status of assets and tags based on the asset status identification code, sets disposal priority and feedback information, sends the disposal priority, feedback information and related images stored in the SD card to the asset management server through the Wi-Fi module, and generates a task report and stores it in the SD card.

[0155] In this embodiment, four disposal priorities are set in descending order, among which: 3 is the highest priority and 0 is the lowest priority. Priority 3 means that the asset is not in its original location, or other abnormal events seriously affect the execution of asset management tasks, and immediate human intervention is required for disposal; priority 2 means that the asset is in its original location, but the tag performance has declined and may reach its service life limit in the near future, requiring increased attention and timely replacement; priority 1 means that the asset is in its original location and the tag performance is normal, and the asset management task can be completed, but the outer surface of the tag is contaminated or partially blocked and needs to be cleaned. Priority 0 means that the asset is in its original location and the tag performance is normal, and no human intervention is required. In this embodiment, a total of 21 asset status identification code scenarios are considered, and the specific setting criteria, feedback information and related images used are shown in Table 2.

[0156] Table 2

[0157]

[0158]

[0159]

[0160]

[0161]

[0162] Although the present invention has been described above, the present invention is not limited to the above-mentioned specific embodiments. The above-mentioned specific embodiments are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can make many modifications without departing from the purpose of the present invention, which are all within the protection of the present invention.

Claims

1. An identification method for an asset status identification system based on RFID and machine vision, characterized in that: The asset status identification system based on RFID and machine vision includes an artificial intelligence processor, and a binocular camera, a pan / tilt, a reader / writer, a Wi-Fi module, an SD card and an asset management server connected to the artificial intelligence processor; wherein: the artificial intelligence processor includes an asset image recognition module, a tag recognition module, a tag response module, a tag appearance inspection module and a tag performance evaluation module; the artificial intelligence processor is used to receive and execute asset status identification tasks; the binocular camera module is used to collect images of target assets and their tags and send the collected data to the artificial intelligence processor; The pan / tilt receives the angle rotation command issued by the artificial intelligence processor to adjust the viewing angle of the binocular camera module; the reader receives the tag reading command issued by the artificial intelligence processor to read the target asset RFID tag information and send the tag reading information to the artificial intelligence processor; the Wi-Fi module is used for wireless communication between the artificial intelligence processor and the asset management server; the SD card is used to store asset status identification processing data; wherein: the identification method includes the following steps: 1) Parameter initialization: The AI ​​processor constructs a 7-bit binary asset status identification code, disposal priority, and initializes parameter information based on task information; 2) Asset image capture: Based on the target asset storage location information, the AI ​​processor controls the asset status recognition system to arrive near the target asset and performs image recognition of the target asset based on the binocular view obtained by the binocular camera module; 3) Tag image capture: determine whether the left and right views of the target asset match the asset image in the asset management server through image similarity. If the conditions are met, set the asset image flag to 1, output the left and right views with overlapping area marks, store them in the SD card, and proceed to step 4); otherwise, reacquire the left and right views of the target asset by turning the pan / tilt, execute step 2.1), and set the asset image flag to 0; 4) Evaluation of label appearance and response performance, including the following steps: 4.1) Image capture judgment: Use the tag recognition model to determine whether there is a tag image in the left and right views of the target asset with the overlapping area; if both exist, set the tag image flag to 1, and mark the identified tags in the left and right views respectively, store the left and right views with the tag marks to the SD card, and go to step 4.2); otherwise, set the tag image flag to 0 and go to step 4.3); 4.2) Distance adjustment: read the left and right eye views with tag marks from the SD card, calculate the distance between the asset status identification system and the tag through the binocular distance measurement algorithm, and go to step 4.3) when this distance is equal to the tag response detection reference distance, otherwise control the asset status identification system to move, repeat steps 2), 3) and distance calculation until the distance between the asset status identification system and the tag is equal to the tag response detection reference distance; 4.3) Response detection: After the reader sends the response test signal, read the identification codes of all responding tags and make a judgment: if the RFID tag identification code corresponding to the target asset is received, set the tag response flag to 1, and increase the number of times the tag has responded by 1. If the tag image flag is also 1 at the same time, go to step 4.4); otherwise, go to step 5); 4.4) Appearance detection: Generate labeled binocular images based on the labeled left and right eye views and perform preprocessing; Then the label appearance anomaly detection model is called to perform appearance classification, and the label appearance flag is set according to the classification result; 4.5) Response performance evaluation: Based on the benchmark tag response performance index database and the tag performance comprehensive evaluation model of the superior and inferior solution distance method, the tag response performance level is evaluated to obtain the tag performance comprehensive evaluation value and performance level evaluation result, and the tag performance flag is set; 5) Generation of asset status identification report: The artificial intelligence processor determines the status of assets and tags based on the asset status identification code, sets disposal priority and feedback information, sends the disposal priority, feedback information and related images stored in the SD card to the asset management server through the Wi-Fi module, and generates a task report and stores it in the SD card.

2. The identification method of the asset status identification system based on RFID and machine vision according to claim 1, characterized in that: The asset status identification code in step 1) is used to identify the target asset and the corresponding tag status; It means: the initialization asset status identification code is 0000000, and the disposal priority is 0; the asset status identification code is that the first bit is the asset image flag, the second bit is the label image flag, the third bit is the label response flag, the fourth and fifth bits are combined as the label appearance flag, and the sixth and seventh bits are combined as the label performance flag; wherein: Initialize parameters based on task information, including: target asset storage location, asset baseline image, asset image similarity threshold, and RFID tag response detection baseline distance corresponding to the target asset, tag response detection signal power value, tag identification code, number of times the tag has responded, and tag standard response signal strength value.

3. According to the identification method of the asset status identification system based on RFID and machine vision in claim 1, in the step 2), according to the storage location information of the target asset, the artificial intelligence processor controls the asset status identification system to arrive near the target asset, and performs the image recognition process of the target asset based on the binocular view obtained by the binocular camera module, comprising the following steps: 2.1) Image acquisition: The AI ​​processor instructs the binocular camera module to capture images of the target asset and receives the left and right eye views output by the binocular camera module; 2.2) Image recognition: Based on the asset reference image, the AI ​​processor calculates the image similarity according to the following formula to determine the overlapping areas where the left and right views match the target asset image features. In formula (1), SI is the similarity between two images, T mat is the number of successfully matched corner features, T total is the total number of corner features in the asset benchmark image; If all the similarities are greater than the asset image similarity threshold, set the asset image flag bit to 1, output the left-eye view and right-eye view with overlapping area markings, that is, the target asset left-eye view and right-eye view, store them in the SD card, and execute step 3); otherwise, set the asset image flag bit to 0 and enter step 2.3); 2.3) Pan-tilt rotation: The artificial intelligence processor instructs the pan-tilt to rotate clockwise by 30 degrees, and determines whether the pan-tilt has rotated clockwise by 360 degrees in total. If so, set the asset image flag bit to 0 and enter step 4); Otherwise, enter step 2.1).

4. The identification method of the asset status identification system based on RFID and machine vision according to claim 1, characterized in that: In the said step 4.4), the label appearance flag bits include: (a) The label appearance is normal, and the label appearance flag bit is set to 11; (b) There is contamination or partial occlusion on the label appearance, and the label appearance flag bit is set to 10; (c) There are deformations, cracks or obvious wear on the label appearance, and the label appearance flag bit is set to 01; (d) There are multiple abnormalities on the label appearance at the same time, and the label appearance flag bit is set to 00.

5. The identification method of the asset status identification system based on RFID and machine vision according to claim 1, characterized in that: In the said step 4.5), the process of evaluating the label response performance level by using the label performance comprehensive evaluation model based on the reference label response performance index database and the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS), obtaining the label performance comprehensive evaluation value and the performance level evaluation result, and setting the label performance flag bit includes the following steps: 4.51) Construct the original matrix X of label response performance index values: In the formula, n=m+1, m is the number of benchmark labels, 1 is the number of current labels to be evaluated; x ij (i=1,2,…,n,j=1,2,3,4) is the jth index value of the i-th label; the first m rows of X are the response performance index values ​​of the benchmark labels, and the last row of X is the response performance index value of the current label to be evaluated; 4.52) Convert each index value in X into a positive index value to obtain the positive matrix Y: α j =max(x 1j ,x 2j ,…,x nj ) b j =min(x 1j ,x 2j ,…,x nj ) 4.53) Normalize each element of the positive matrix to obtain the standardized matrix Z: In the formula, z ij is the normalized label response performance indicator value. 4.54) Calculate the entropy value and information entropy redundancy of each response performance index: g j =1-C j In the formula, C j and g j are the entropy value and information entropy redundancy of the j-th response performance indicator respectively. 4.55) Calculate the entropy weight w of each response performance index j : 4.56) The ideal optimal value of each indicator is defined by the maximum value of each standardized label response performance indicator value and the ideal worst value In the formula, and are the jth standardized index values ​​z of all labels respectively. ij The maximum and minimum values ​​in . 4.57) Calculate the Euclidean distance between the standardized label response performance index value of the i-th label and the ideal optimal value and the Euclidean distance from the ideal worst value 4.58) Calculate the comprehensive performance evaluation value S of the i-th label i : 4.59) Sort the label performance comprehensive evaluation values of all labels from large to small to obtain the evaluation value sequence, retrieve the label performance comprehensive evaluation value of the currently to-be-evaluated label and its ranking r in the evaluation value sequence, and calculate the ranking ratio R: (4.60) Determine the performance level evaluation result of the to-be-evaluated label according to the ranking ratio R, and output the label performance comprehensive evaluation value S and the performance level evaluation result of the currently to-be-evaluated label.

6. The identification method of the asset status identification system based on RFID and machine vision according to claim 5, characterized in that: In the said step 4.5), setting the label performance flag bit according to the label performance level evaluation result includes: (a) According to 0 < R ≤ 25%, the evaluation level is Ⅰ, and the label performance flag bit is set to 11; (b) According to 25 < R ≤ 50%, the evaluation level is Ⅱ, and the label performance flag bit is set to 10; (c) According to 50 < R ≤ 75%, the evaluation level is Ⅲ, and the label performance flag bit is set to 01; (d) According to 75 < R ≤ 100%, the evaluation level is Ⅳ, and the label performance flag bit is set to 00.

7. The identification method of the asset status identification system based on RFID and machine vision according to claim 6, characterized in that: In the said step 5), the artificial intelligence processor judges the status of the asset and the label based on the asset status identification code, and sets the disposal priority and feedback information, including: 5.1) Based on the asset status identification codes 0000000, 0010000, 1100000, 1010000, 1000000, 1111100, 1111000 and 1110000, set the disposal priority to 3, indicating that the asset is not in its original place, or other abnormal events that seriously affect the execution of the asset management task, and manual intervention is required immediately for disposal; 5.2) Based on the asset status identification codes 1111101, 1111001, 1110101 and 1110001, the disposal priority is set to 2, indicating that the asset is in its original location, but the tag performance has deteriorated and may reach the service life limit in the near future, requiring increased attention and timely replacement; 5.3) Based on the asset status identification codes 1111010, 1110110, 1111011, 1110111 and 1110011, setting the priority to 1 means that the asset is in place and the tag performance is normal, and the asset management task can be completed, but the outer surface of the tag is contaminated or partially blocked and needs to be cleaned; 5.4) Based on the asset status identification codes 1111110 and 1111111; setting the priority to 0 means that the asset is in place and the tag performance is normal, and no manual intervention is required.

Images