A method for intelligent dynamic inventory of unmanned aerial vehicles

Abstract

The present invention discloses an unmanned aerial vehicle intelligent dynamic inventory method, which relates to the field of warehouse management technology. According to the layout of the warehouse, shelf distribution and material storage area information, the unmanned aerial vehicle flight inventory route is planned, and the uninventoryed area is marked; the unmanned aerial vehicle flies to obtain radio frequency data and material image visual data on the warehouse materials; the collected radio frequency data and material image visual data are preprocessed, and feature extraction is performed to obtain radio frequency features and visual features. The unmanned aerial vehicle intelligent dynamic inventory method enhances the inventory accuracy through the coordinated use of radio frequency and visual technology. Radio frequency identification can accurately read material label information and obtain key data such as material number and category code. The visual recognition system can capture material appearance features from multiple angles and distances to assist in confirming the material category and status. It can efficiently inventory a large area in the warehouse in a short time to ensure smooth warehouse operation.

Description

Technical Field

[0001] The present invention relates to the technical field of warehouse management, and in particular to an intelligent dynamic inventory counting method using an unmanned aerial vehicle. Background Art

[0002] With the development of e-commerce, the warehousing industry is facing unprecedented pressure. Traditional inventory counting methods, such as manually climbing high-altitude shelves for counting, are not only time-consuming and labor-intensive, but also prone to errors. In addition, manual inventory counting also has safety hazards, such as possible personal injuries caused by high-altitude operations. Drone technology, with its high efficiency, flexibility and safety, has become a powerful tool for solving inventory management problems in the warehousing industry. Drones can easily fly to high altitudes, use cameras and sensors to perform fast and accurate scanning, and transmit data to the background system to achieve efficient inventory counting.

[0003] When drones fly in warehouses and use radio frequency technology or vision technology to check and count warehouse materials, they are easily disturbed by factors such as the placement of items on warehouse shelves, damaged or blocked labels, which can easily lead to recognition failures or errors, thereby affecting the accuracy of warehouse material checking and counting. Therefore, how to combine radio frequency technology and vision technology, match the data fit of the two and analyze them to improve the accuracy and stability of warehouse goods checking and counting is the problem we need to solve. To this end, a drone intelligent dynamic inventory method is proposed. Summary of the invention

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for intelligent dynamic inventory of unmanned aerial vehicles, comprising the following steps:

[0005] Step 1: Plan the drone flight route based on the warehouse layout, shelf distribution, and material storage area information, and mark the uninventoryed areas;

[0006] Step 2: The drone flies to obtain radio frequency data and visual image data of materials in the warehouse;

[0007] Step 3: Preprocess the collected radio frequency data and material image visual data, and perform feature extraction to obtain radio frequency features and visual features;

[0008] Step 4: Correlate and match the radio frequency feature data with the visual feature data, and compare the deviation values ​​between the radio frequency feature data and the visual feature data;

[0009] Step 5: Based on the analysis results of the deviation value, mark the abnormal inventory data and verify the abnormal inventory data;

[0010] Step 6. Organize the verified warehouse material inventory results and output the warehouse material verification and inventory report, which includes detailed information on material type, quantity, and storage location, providing a reliable basis for warehouse management.

[0011] Preferably, in step 1, the planning process of the UAV flight inventory route includes:

[0012] Obtain warehouse blueprints and conduct on-site surveys to supplement warehouse shelf status, and verify shelf height, number of layers, actual placement spacing, and obstacles in the flight environment;

[0013] Combined with warehouse inventory data, the storage rules of materials in different areas are clarified according to material categories, batches, and shelf life attributes, and key inventory areas and regular inventory areas are determined. Among them, key inventory areas include areas where frequent in and out of the warehouse, valuables, and fragile items are stored;

[0014] Based on the actual state of the warehouse and the distribution of materials, the grid model is used to describe the inventory area, and the inventory area is divided into multiple inventory sub-areas. Grid partitioning is adopted, and each row of shelves and the aisles between them are used as independent grids. The inventory area of ​​large shelves is partitioned, and the inventory area for scattered storage is set as a separate partition;

[0015] Based on the divided inventory areas, a reciprocating zigzag flight route is designed to ensure that the drone flies close to the shelves, fully scans material labels and captures visual images, so that the flight trajectory covers all points. Combined with the effective range of visual recognition and the coverage radius of radio frequency signals, the drone is kept 0.5 meters away from the shelves during the flight process.

[0016] Preferably, the marking process of the uninventoryed area includes:

[0017] The UAV is equipped with a visual sensor, which can be used to identify images collected by the visual sensor during flight to determine whether there are new obstructions on the flight path.

[0018] If an obstacle is detected on the flight path, the flight control system of the drone will dynamically adjust the path according to the preset algorithm and automatically bypass the obstacle to ensure that the drone can continue to fly safely and smoothly, avoiding interruptions to inventory work due to unexpected situations such as collisions, while also ensuring the continuity of data collection;

[0019] By real-time recording of the flight trajectory and the area where data has been collected, the initially planned inventory area is compared, and the GPS information on the drone is used to synchronously mark the uninventoryed areas that have not been covered by the flight trajectory;

[0020] According to the marked uninventoryed areas, warehouse managers are arranged to check the obstacle status and material status of the marked areas and conduct manual re-inventory to further ensure that the entire warehouse space can be fully covered, minimize missed areas, and improve the completeness and accuracy of warehouse material inventory.

[0021] Preferably, in step 2, the process of acquiring radio frequency data and visual data includes:

[0022] Configure the UAV flight control system, input the flight route and parameters, connect the UAV to the ground control station through a wireless communication link, control the UAV to take off in a predetermined manner through the ground control station, and remotely start the onboard radio frequency identification device and visual recognition system;

[0023] Use RFID equipment to scan the RFID tags on warehouse materials in real time, receive and process RFID signals, extract RFID data containing material numbers and basic information about categories, store the RFID data in the local storage device of the drone, and transmit it to the ground control station in real time;

[0024] Use the visual recognition system to continuously capture images of materials in the passing area, record the visual features of the materials from different angles and distances, including shape, color, and label, form visual data, store the visual data in the local storage device of the drone, and transmit it to the ground control station in real time;

[0025] After the drone completes the inventory flight of the entire area according to the planned flight route, it is controlled to fly back to the take-off point, and the RF data and visual data stored locally on the drone and the data received by the ground control station are summarized. A preliminary integrity check is performed on the data to check whether there is any data loss or damage. If there is a problem, the cause is analyzed in a timely manner and an attempt is made to restore the data to prepare for subsequent data preprocessing and further analysis.

[0026] Preferably, in step 3, the process of acquiring the radio frequency features and the visual features includes:

[0027] Perform signal denoising and calibration preprocessing operations on RF data, where filtering technology (low-pass filter, high-pass filter, band-pass filter, etc.) is used to remove noise components in RF signals to improve signal quality, and calibration is used to eliminate differences between devices to make the extracted features more stable and consistent. Calibration includes frequency calibration, gain calibration, and phase calibration;

[0028] Extract the tag information in the RF tag from the RF data, including the material number, category code, and production batch. By parsing the structure and protocol of the RF tag data, extract the tag information as the main RF feature, and extract the amplitude, energy, and waveform features from the time domain of the RF signal. The amplitude feature reflects the amplitude of the signal, the energy feature represents the energy distribution of the signal, and the waveform feature describes the shape of the signal. Convert the RF signal to the frequency domain, extract the spectrum, harmonics, and phase frequency domain features, and analyze the distribution and characteristics of the signal in frequency.

[0029] Performing preprocessing operations of image denoising, image enhancement and image conversion on visual data, wherein the image is denoised using the filtering technology of mean filtering to improve the image quality, the image is contrast enhanced and sharpened to improve the image clarity and visibility, and the image is converted from the RGB color space to the HSV color space through image conversion to facilitate subsequent feature extraction;

[0030] Use the Sobel edge detection algorithm to extract edge information from the image. Based on the edge detection results, further extract the shape features of the object, including contour, area, and perimeter, for object recognition and classification. Use the local binary pattern texture description method to extract texture features in the image and analyze the texture and detail information on the surface of the object. By extracting the color moment features in the image, describe the color distribution and color attributes of the object.

[0031] The RF features extracted from the RF data, including label information, time domain features, and frequency domain features, are integrated with the visual features extracted from the visual data, including edge information, shape features, texture features, and color features, to form a complete feature vector.

[0032] Preferably, in step 4, the process of comparing the deviation values ​​of the radio frequency feature data and the visual feature data includes:

[0033] The material tag number in the RF data is used as the index. The material tag number is the unique identifier of each material in the RFID system. It has been obtained from the tag information in the RF feature extraction stage. The tag position in the visual data is located through the image recognition algorithm, and the visual data is preliminarily associated with the RF tag number. In the case where the association cannot be directly located in the image through the tag, the flight trajectory data and collection timestamp of the drone are used. Since the RF data and visual data collected at the same time and similar location are likely to belong to the same material, the association index is established based on the proximity in time and space.

[0034] The matching model trained by the supervised learning algorithm inputs RF features and visual features as feature vectors, and the output result is the material matching degree, which determines whether the material matches in the RF data and visual data. The matching model is trained by collecting a large amount of sample data that has been clearly matched (determined by manual annotation) and unmatched. The trained matching model can make matching judgments for new RF-visual feature pairs, thereby improving the accuracy and efficiency of matching.

[0035] For successfully matched RF features and visual feature data, compare the deviations of the materials identified in the RF data and visual data in terms of material quantity, size, location, and category, and set weights for each difference indicator based on the characteristics of the materials and inventory requirements;

[0036] The deviation of materials in various difference indicators is comprehensively considered, combined with the weight set for each difference indicator, to calculate the difference matching index, and analyze to determine the degree of difference between the radio frequency feature data and the visual feature data.

[0037] Preferably, the calculation process of the difference matching index is:

[0038] Extract the associated data of the identified materials in terms of quantity, size, location, and category from the radio frequency data and visual data, and then calculate the deviation of each difference indicator;

[0039] Based on the quantity of materials identified by radio frequency data and the quantity of materials identified by visual data, the ratio of the absolute value of the quantity difference to the maximum value of the quantity is calculated, and the deviation value is normalized to between 0 and 1 to obtain the material quantity deviation value, which is expressed as: Among them, DQ is the material quantity deviation value, n r is the number of materials identified by radio frequency data, n v The number of materials identified for the visual data;

[0040] Based on the material size information in the radio frequency data and the material size information in the visual data, the radio frequency material size vector and the visual material size vector are obtained, respectively. and Calculate the sum of squared deviations of the dimensions for each dimension in the The size differences of multiple dimensions are combined and normalized by dividing by the maximum square sum to obtain the material size deviation value, which is expressed as: Among them, DS is the material size deviation value, s ri is the RF material size of the i-th dimension difference, s vi is the visual material size of the i-th size difference dimension, is the RF material size vector, is the visual material size vector, m is the number of size difference dimensions;

[0041] Material location coordinates based on radio frequency data positioning (x r ,y r , z r ) and the position coordinates (x v ,y v , z v );

[0042] Calculate the square of the spatial Euclidean distance Combined with the maximum square distance of the material location in the warehouse space (predetermined according to the size of the warehouse), calculate the material position deviation value, and analyze the relative degree of position deviation in the entire warehouse space. The expression is: Among them, DL is the material position deviation value, is the square of the spatial Euclidean distance, is the maximum distance squared to the material location;

[0043] The similarity measurement method is used to obtain the feature vectors of material categories in the radio frequency data and visual data, which are and And calculate the cosine similarity between the two Obtain the material category deviation value, convert the cosine similarity into a deviation value, and its expression is: DC = 1-sim, where DC is the material category deviation value, is the feature vector of the material category in the radio frequency data, is the feature vector of the material category in the visual data, sim is the cosine similarity of the feature vectors of the material category in the RF data and the visual data, and the DC range is between 0 and 1. The larger the DC value, the greater the category deviation;

[0044] By combining the weights set for each difference indicator with the calculated material quantity deviation value, material size deviation value, material position deviation value, and material category deviation value, the difference matching index DMI is obtained, which comprehensively measures the degree of difference between the radio frequency feature data and the visual feature data.

[0045] Preferably, the expression of the difference matching index is:

[0046]

[0047] Among them, DMI is the difference matching index, DQ is the material quantity deviation value, W1 is the weight of the material quantity deviation value, DS is the material size deviation value, w2 is the weight of the material size deviation value, DL is the material position deviation value, w3 is the weight of the material position deviation value, DC is the material category deviation value, w4 is the weight of the material category deviation value, and w1+w2+w3+w4=1. DMI is used to comprehensively measure the degree of difference between RF feature data and visual feature data. The value range of DMI is between 0 and 1.

[0048] Preferably, the process of marking abnormal inventory data includes:

[0049] According to the statistical analysis of historical inventory data, material characteristics and warehouse management accuracy requirements, corresponding abnormality judgment thresholds are set for material quantity deviation value, material size deviation value, material location deviation value and material category deviation value respectively as the preliminary standard for judging whether the data is abnormal. Among them, the setting of abnormality judgment threshold is to collect the deviation data of each difference indicator under multiple rounds of normal inventory, observe its distribution law, calculate the mean and finally determine the abnormality judgment threshold;

[0050] For each set of material inventory records corresponding to the RF features and visual feature data for which the difference matching index has been calculated, the deviation values ​​of the material quantity, size, location, and category difference indicators are compared with the corresponding pre-set abnormal judgment thresholds. If the deviation value of any difference indicator of the material exceeds the corresponding abnormal judgment threshold, the group of inventory data is marked as abnormal inventory data, and the abnormal inventory data is highlighted in red in the inventory data report, so that the abnormal inventory data can be quickly identified and screened out later;

[0051] Arrange warehouse staff to go to the location of abnormally marked materials to check the placement and appearance of the materials and whether the materials are blocked. If the materials are partially blocked by other items, it is easy for the visual recognition system to fail to fully obtain their image features, resulting in recognition deviations in size and category. Also check whether the material labels are damaged or contaminated. Damaged labels can easily prevent the RFID device from accurately reading information, resulting in data anomalies.

[0052] Compare the basic information of materials (material number, storage time, batch) in the warehouse inventory management system with the actual materials and the collected radio frequency and visual data to confirm whether there is any information inconsistency. If the material category recorded in the warehouse inventory management system does not match the actual material appearance and the category code in the radio frequency tag, further investigation is required to determine whether the data anomaly is caused by input errors or chaotic material storage;

[0053] For inventory data marked as abnormal, ensure that the causes of the data abnormality are fully investigated. After completing the inspection, comprehensively analyze the results of various aspects, make a final judgment and handling opinions on the abnormal inventory data, and take corresponding measures to correct the abnormal situation and improve the accuracy and reliability of the inventory data.

[0054] Preferably, in step 6, the output process of the warehouse material verification and inventory report includes:

[0055] Collect verified and confirmed warehouse inventory data from warehouse management systems, radio frequency identification devices, and visual identification systems, and integrate data from different sources to ensure data integrity and consistency;

[0056] According to the sorted data, list the types, quantities, and storage location information of materials, including material number, name, specification model, quantity, storage shelf / area, and number of layers, to ensure that the information of each material is accurate, and use charts to intuitively display the quantity distribution and storage location information of materials, add necessary explanatory text below the charts, and form a complete warehouse material verification and inventory report;

[0057] Check the warehouse material verification and inventory report, and publish it in electronic form to relevant personnel to ensure that all relevant personnel can obtain the latest inventory report in a timely manner, and then track the implementation of warehouse management based on the suggestions and measures put forward in the warehouse material verification and inventory report.

[0058] The present invention provides a method for intelligent dynamic inventory of drones. It has the following beneficial effects:

[0059] 1. The intelligent dynamic inventory method of the drone enhances the accuracy of inventory through the coordinated use of radio frequency and vision technology. Radio frequency identification can accurately read the material label information and obtain key data such as material number and category code. The visual recognition system can capture the appearance characteristics of materials from multiple angles and distances, assist in confirming the material category and status, and can efficiently inventory large areas in the warehouse in a short time to ensure smooth warehouse operations.

[0060] 2. The application of the drone intelligent dynamic inventory counting method eliminates the need for warehouse managers to invest a lot of manpower in tedious inventory counting work. The drone can fly autonomously and conduct a comprehensive scan and inventory of warehouse materials according to the preset route, which greatly reduces the need for manual operation. It not only reduces labor costs, but also enables warehouse managers to devote more energy to other important management work, thereby improving the overall operational efficiency of the warehouse. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A method flow chart of a method for intelligent dynamic inventory of unmanned aerial vehicles according to the present invention;

[0062] Figure 2 The flowchart of obtaining radio frequency features and visual features of the present invention;

[0063] Figure 3 The present invention is a flow chart of comparing the deviation values ​​of radio frequency feature data and visual feature data. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are provided for the purpose of illustration and description, and are not intended to be exhaustive or to limit the present invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention, and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

[0065] The first embodiment, as Figure 1 , Figure 2As shown, the present invention provides a technical solution: a method for intelligent dynamic inventory of unmanned aerial vehicles, comprising the following steps:

[0066] Step 1: Plan the flight route of the drone according to the layout of the warehouse, the distribution of shelves, and the information of the material storage area, and mark the uninventoryed areas to ensure that the warehouse space that needs to be counted can be fully covered and the missed areas can be minimized. The RFID equipment and visual recognition system carried by the drone should be checked to ensure that they are working properly. At the same time, the recognition parameters should be set, including the frequency band and power of the RFID, and the resolution and shooting frame rate of the visual recognition.

[0067] Furthermore, the planning process of the drone flight inventory route includes:

[0068] Obtain warehouse blueprints and conduct on-site surveys to supplement the warehouse shelf status, verify shelf height, number of layers, actual placement spacing, and obstacles in the flight environment, and combine warehouse inventory data to clarify the storage rules of materials in different areas according to material categories, batches, and shelf life attributes, and determine key inventory areas and regular inventory areas. Among them, key inventory areas include places where materials are frequently in and out of the warehouse, valuables, and fragile items are stored. Based on the actual status of the warehouse and the distribution status of materials, use a grid model to describe the inventory area, divide the inventory area into multiple inventory sub-areas, and use grid partitioning to treat each row of shelves and the aisles between them as independent grids. The inventory area of ​​large shelves is partitioned, and the scattered storage inventory area is set as a separate partition. According to the divided inventory areas, a reciprocating zigzag flight route is designed to ensure that the drone shuttles close to the shelf, fully scans the material labels and captures the visual images, so that the flight trajectory covers all points, and combined with the effective range of visual recognition and the coverage radius of the radio frequency signal, the drone is kept 0.5 meters away from the shelf during the flight process;

[0069] The process of marking uninventoryed areas includes:

[0070] The drone is equipped with a visual sensor. During the flight of the drone, the visual sensor collects images through image recognition to determine whether there are new obstructions on the flight path. If an obstacle is detected on the flight path, the flight control system of the drone dynamically adjusts the path according to the preset algorithm and automatically bypasses the obstacle to ensure that the drone can continue to fly safely and smoothly, avoiding interruptions in inventory work due to unexpected situations such as collisions. At the same time, it also ensures the continuity of data collection. Through real-time recording of the flight trajectory and the area where data has been collected, compared with the initially planned inventory area range, the GPS information on the drone is used to synchronously mark the uninventoryed areas that have not been covered by the flight trajectory. According to the marked uninventoryed areas, the warehouse management personnel are arranged to check the obstacle status and material status of the marked areas, and conduct manual review and inventory, further ensuring that the entire warehouse space can be fully covered, minimizing the missed areas, and improving the integrity and accuracy of the warehouse material inventory;

[0071] Step 2: The drone flies to obtain the RF data and visual data of the warehouse materials. According to the planned flight route, the drone is controlled to take off and start flying. During the flight, the RFID equipment is used to scan the RFID tags on the warehouse materials in real time to obtain the RF data containing the material number and basic information of the category. The visual recognition system continuously captures the images of the materials passing through the area, records the appearance and visual features of the materials from different angles and distances, forms visual data, configures the drone flight control system, inputs the flight route and parameters, connects the drone to the ground control station through a wireless communication link, controls the drone to take off in a predetermined manner through the ground control station, and remotely starts the onboard RFID equipment and visual recognition system. The RFID equipment is used to scan the RFID tags on the warehouse materials in real time, receives and processes the RFID signals, and extracts the material numbers. , RF data of basic category information, store the RF data in the local storage device of the drone, and transmit it to the ground control station in real time, use the visual recognition system to continuously capture images of materials passing through the area, record the appearance and visual features of the materials from different angles and distances, including shape, color, and label, to form visual data, store the visual data in the local storage device of the drone, and transmit it to the ground control station in real time. When the drone completes the inventory flight of all areas according to the planned flight route, control the drone to fly back to the take-off point, and summarize the RF data and visual data stored in the drone and the data received by the ground control station, conduct a preliminary integrity check on the data to check whether there is any data loss or damage. If there is a problem, analyze the cause in time and try to recover the data to prepare for subsequent data preprocessing and further analysis;

[0072] Step 3: Preprocess the collected RF data and material image visual data, and perform feature extraction to obtain RF features and visual features. Perform signal denoising and calibration preprocessing operations on the RF data. Filtering technology (low-pass filter, high-pass filter, band-pass filter, etc.) is used to remove noise components in the RF signal to improve the signal quality. Calibration eliminates differences between devices to make the extracted features more stable and consistent. Calibration includes frequency calibration, gain calibration, and phase calibration. The tag information in the RF tag is extracted from the RF data, including material number, category code, and production batch. The tag information is extracted as the main RF feature by parsing the structure and protocol of the RF tag data, and the amplitude, energy, and waveform features are extracted from the time domain of the RF signal. The amplitude feature reflects the amplitude of the signal, the energy feature represents the energy distribution of the signal, and the waveform feature describes the shape of the signal. The RF signal is converted to the frequency domain, and the spectrum, harmonics, and phase frequency domain features are extracted. The distribution and characteristics of the signal in frequency are analyzed. Perform preprocessing operations of image denoising, image enhancement and image conversion, wherein the image is denoised using a filtering technique of mean filtering to improve the image quality, the image is contrast enhanced and sharpened to improve the image clarity and visibility, the image is converted from the RGB color space to the HSV color space through image conversion to facilitate subsequent feature extraction, the edge information in the image is extracted using the Sobel edge detection algorithm, and based on the result of edge detection, the shape features of the object are further extracted, including the outline, area, and perimeter, for object recognition and classification, and the texture features in the image are extracted using a texture description method of a local binary pattern, the texture features on the surface of the object are analyzed, the color distribution and color attributes of the object are described by extracting color moment features in the image, and the radio frequency features including label information, time domain features, and frequency domain features extracted from the radio frequency data are integrated with the visual features including edge information, shape features, texture features, and color features extracted from the visual data to form a complete feature vector;

[0073] Step 4: Correlate and match the radio frequency feature data with the visual feature data, and compare the deviation values ​​between the radio frequency feature data and the visual feature data;

[0074] Step 5: Based on the analysis results of the deviation value, mark the abnormal inventory data and verify the abnormal inventory data;

[0075] Step 6. Organize the verified warehouse material inventory results and output the warehouse material verification and inventory report, which includes detailed information on material type, quantity, and storage location, providing a reliable basis for warehouse management.

[0076] The second embodiment is based on the first embodiment. Figure 3As shown, in step 4, the process of comparing the deviation value between the radio frequency feature data and the visual feature data includes:

[0077] The material tag number in the RF data is used as the index. The material tag number is the unique identifier of each material in the RFID system. It has been obtained from the tag information in the RF feature extraction stage. The tag position in the visual data is located through the image recognition algorithm, and the visual data is preliminarily associated with the RF tag number. For situations where the association cannot be directly located in the image through the tag, the flight trajectory data and collection timestamp of the drone are used. Since the RF data and visual data collected at the same time and similar position are likely to belong to the same material, an association index is established based on the proximity in time and space. The matching model trained by the supervised learning algorithm is used, and the RF features and visual features are input as feature vectors. The output result is the material matching degree, which determines the material in Whether there is a match between the RF data and the visual data, the training of the matching model is achieved by collecting a large amount of sample data that has been clearly matched (determined by manual annotation) and unmatched. The trained matching model can make matching judgments on new RF-visual feature pairs to improve the accuracy and efficiency of matching. For the successfully matched RF features and visual feature data, the deviations of the materials identified in the RF data and visual data in terms of material quantity, size, location, and category are compared, and a weight is set for each difference indicator according to the characteristics of the materials and inventory requirements. The deviation of the materials in each difference indicator is comprehensively considered, combined with the weight set for each difference indicator, and the difference matching index is calculated. The difference between the RF feature data and the visual feature data is analyzed to determine the degree of difference;

[0078] The calculation process of the difference matching index is:

[0079] The associated data of the difference indicators of the material quantity, size, location, and category of the identified materials are extracted from the radio frequency data and the visual data, and then the deviation of each difference indicator is calculated. Based on the material quantity identified by the radio frequency data and the material quantity identified by the visual data, the ratio of the absolute value of the quantity difference to the maximum value of the quantity is calculated, and the deviation value is normalized to between 0 and 1 to obtain the material quantity deviation value, which is expressed as: Among them, DQ is the deviation value of material quantity, n r is the number of materials identified by radio frequency data, n v is the number of materials identified by the visual data. Based on the material size information in the RF data and the material size information in the visual data, the RF material size vector and the visual material size vector are obtained, which are respectively and Calculate the sum of squared deviations of the dimensions for each dimension in the The size differences of multiple dimensions are combined and normalized by dividing by the maximum square sum to obtain the material size deviation value, which is expressed as: Among them, DS is the material size deviation value, s ri is the RF material size of the i-th dimension difference, s vi is the visual material size of the i-th size difference dimension, is the RF material size vector, is the visual material size vector, m is the number of size difference dimensions, and the material location coordinates (x r ,y r , z r ) and the position coordinates (x v ,y v , z v );

[0080] Calculate the square of the spatial Euclidean distance Combined with the maximum square distance of the material location in the warehouse space (predetermined according to the size of the warehouse), calculate the material position deviation value, and analyze the relative degree of position deviation in the entire warehouse space. The expression is: Among them, DL is the material position deviation value, is the square of the spatial Euclidean distance, is the maximum distance square of the material location, and the similarity measurement method is used to obtain the feature vectors of the material categories in the radio frequency data and visual data, respectively. and And calculate the cosine similarity between the two Obtain the material category deviation value, convert the cosine similarity into a deviation value, and its expression is: DC = 1-sim, where DC is the material category deviation value, is the feature vector of the material category in the radio frequency data, is the feature vector of the material category in the visual data, sim is the cosine similarity of the feature vectors of the material category in the RF data and the visual data, the DC range is between 0 and 1, the larger the DC value is, the greater the category deviation is. Combining the weights set for each difference index with the calculated material quantity deviation value, material size deviation value, material position deviation value, and material category deviation value, the difference matching index DMI is obtained to comprehensively measure the degree of difference between the RF feature data and the visual feature data;

[0081] Furthermore, the expression of the difference matching index is:

[0082]

[0083] Among them, DMI is the difference matching index, DQ is the material quantity deviation value, w1 is the weight of the material quantity deviation value, DS is the material size deviation value, w2 is the weight of the material size deviation value, DL is the material position deviation value, w3 is the weight of the material position deviation value, DC is the material category deviation value, w4 is the weight of the material category deviation value, and w1+w2+w3+w4=1. DMI is used to comprehensively measure the degree of difference between RF feature data and visual feature data. The value range of DMI is between 0 and 1. The closer the DMI value is to 0, the higher the matching degree between RF and visual feature data and the smaller the degree of difference; the closer the DMI value is to 1, the greater the degree of difference. The value ranges of DQ, DS, DL, and DC are all between 0 and 1. When all difference index deviations (DQ, DS, DL, DC) are 1 (that is, RF and visual data are completely matched in all aspects), DMI=1; when the difference index deviation is large enough to make the weighted sum approach infinity, Approaching 0, DMI approaches 1;

[0084] As the deviation of each difference indicator (DQ, DS, DL, DC) increases, the weighted sum w1·DQ+w2·DS+w3·DL+w4·DC will also increase. will decrease, will increase, that is, the difference matching index reflects that the difference between the RF and visual feature data is increasing. On the contrary, when the deviation of each difference index decreases, the DMI will decrease, indicating that the difference is decreasing.

[0085] The process of marking abnormal inventory data includes:

[0086] According to the statistical analysis of historical inventory data, material characteristics and warehouse management accuracy requirements, corresponding abnormal judgment thresholds are set for the material quantity deviation value, material size deviation value, material location deviation value and material category deviation value as the preliminary standard for judging whether the data is abnormal. Among them, the setting of the abnormal judgment threshold is carried out by collecting the deviation data of each difference index under multiple rounds of normal inventory, observing its distribution law, calculating the mean and finally determining the abnormal judgment threshold. For each group of material inventory records corresponding to the radio frequency feature and visual feature data for which the difference matching index has been calculated, the deviation values ​​of the material quantity, size, location and category difference indicators are compared with the corresponding pre-set abnormal judgment thresholds. If the deviation value of any difference indicator of the material exceeds the corresponding abnormal judgment threshold, the group of inventory data is marked as abnormal inventory data, and the abnormal inventory data is highlighted in red in the inventory data report, so as to facilitate the subsequent rapid identification and screening of abnormal inventory data, and arrange warehouse staff to go to the location of the marked abnormal materials to check the materials on the spot. Check the placement and appearance of the materials, check whether the materials are blocked. If the materials are partially blocked by other items, it is easy for the visual recognition system to fail to fully obtain their image features, resulting in recognition deviations in size and category. Check whether the material labels are damaged or stained. Damaged labels can easily make it difficult for the RFID device to accurately read the information, resulting in data anomalies. Compare the basic information of the materials (material number, storage time, batch) in the warehouse inventory management system with the actual materials and the collected RFID and visual data to confirm whether there is any information inconsistency. If the material category recorded in the warehouse inventory management system does not match the actual material appearance and the category code in the RFID tag, it is necessary to further investigate whether the data anomaly is caused by input errors or chaotic material storage. For the inventory data marked as abnormal, ensure that the causes of the data anomaly are fully investigated. After completing the inspection, comprehensively analyze the results of various aspects, give a final judgment and handling opinions on the abnormal inventory data, and take corresponding measures to correct the abnormal situation to improve the accuracy and reliability of the inventory data;

[0087] In step 6, the output process of the warehouse material verification and inventory report includes:

[0088] Collect verified and confirmed warehouse inventory data from the warehouse management system, radio frequency identification equipment, and visual identification system, and integrate data from different sources to ensure data integrity and consistency. Based on the sorted data, list the type, quantity, and storage location information of the materials, including material number, name, specification model, quantity, storage shelf / area, and number of layers to ensure that the information of each material is accurate, and use charts to intuitively display the quantity distribution and storage location information of the materials. Add necessary explanatory text below the chart to form a complete warehouse material verification and inventory report, verify the warehouse material verification and inventory report, and publish the warehouse material verification and inventory report in electronic form to relevant personnel to ensure that all relevant personnel can obtain the latest inventory report in a timely manner, and then track the implementation of warehouse management based on the suggestions and measures proposed in the warehouse material verification and inventory report.

[0089] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field and related fields without creative work should fall within the scope of protection of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention are implemented according to the conventional means in the field unless otherwise specified and limited.

Claims

1. A method for intelligent dynamic inventory of unmanned aerial vehicles, characterized in that: The following steps are involved: Step 1: Plan the drone flight route based on the warehouse layout, shelf distribution, and material storage area information, and mark the uninventoryed areas; Step 2: The drone flies to obtain radio frequency data and visual image data of materials in the warehouse; Step 3: Preprocess the collected radio frequency data and material image visual data, and perform feature extraction to obtain radio frequency features and visual features; Step 4: Correlate and match the radio frequency feature data with the visual feature data, and compare the deviation values ​​of the radio frequency feature data and the visual feature data. In step 4, the process of comparing the deviation values ​​of the radio frequency feature data and the visual feature data includes: Using the material tag number in the RF data as an index, the tag position in the visual data is located through the image recognition algorithm, and the visual data is preliminarily associated with the RF tag number. In the case where the association cannot be directly located in the image through the tag, the flight trajectory data and collection timestamp of the drone are used to establish an association index based on the proximity of time and space; The matching model trained by the supervised learning algorithm inputs the RF features and visual features as feature vectors, and the output result is the material matching degree, which determines whether the material matches in the RF data and visual data; For successfully matched RF features and visual feature data, compare the deviations of the materials identified in the RF data and visual data in terms of material quantity, size, location, and category, and set weights for each difference indicator based on the characteristics of the materials and inventory requirements; The deviation of the materials in each difference index is combined with the weight set for each difference index to calculate the difference matching index, and the difference between the RF feature data and the visual feature data is analyzed to determine the difference degree; The calculation process of the difference matching index is: Extract the associated data of the identified materials in terms of quantity, size, location, and category from the radio frequency data and visual data, and then calculate the deviation of each difference indicator; Based on the quantity of materials identified by radio frequency data and the quantity of materials identified by visual data, the ratio of the absolute value of the quantity difference to the maximum value of the quantity is calculated, and the deviation value is normalized to between 0 and 1 to obtain the material quantity deviation value, which is expressed as: Among them, DQ is the deviation value of material quantity, n r is the number of materials identified by radio frequency data, n v The number of materials identified for the visual data; Based on the material size information in the radio frequency data and the material size information in the visual data, the radio frequency material size vector and the visual material size vector are obtained, respectively. and Calculate the sum of squared deviations of the dimensions for each dimension in the The size differences of multiple dimensions are combined and normalized by dividing by the maximum square sum to obtain the material size deviation value, which is expressed as: Among them, DS is the material size deviation value, S ri is the RF material size of the i-th dimension, S vi is the visual material size of the i-th size difference dimension, is the RF material size vector, is the visual material size vector, m is the number of size difference dimensions; Material location coordinates based on radio frequency data positioning (x r ,y r , z r ) and the position coordinates (x v ,y v , z v ); Calculate the square of the spatial Euclidean distance Combined with the maximum square distance of the material location in the warehouse space Calculate the material position deviation value and analyze the relative degree of position deviation in the entire warehouse space. The expression is: Among them, DL is the material position deviation value, is the square of the spatial Euclidean distance, is the maximum distance squared to the material location; The similarity measurement method is used to obtain the feature vectors of material categories in the radio frequency data and visual data, which are and And calculate the cosine similarity between the two Obtain the material category deviation value, convert the cosine similarity into a deviation value, and its expression is: DC = 1-sim, where DC is the material category deviation value, is the feature vector of the material category in the radio frequency data, is the feature vector of the material category in the visual data, sim is the cosine similarity between the feature vectors of the material category in the RF data and the visual data; Combining the weights set for each difference index with the calculated material quantity deviation value, material size deviation value, material position deviation value, and material category deviation value, the difference matching index DMI is obtained to comprehensively measure the degree of difference between the radio frequency feature data and the visual feature data; The expression of the difference matching index is: Wherein, DMI is the difference matching index, DQ is the material quantity deviation value, w1 is the weight of the material quantity deviation value, DS is the material size deviation value, w2 is the weight of the material size deviation value, DL is the material location deviation value, w3 is the weight of the material location deviation value, DC is the material category deviation value, w4 is the weight of the material category deviation value, and w1+w2+w3+w4=1, and the value range of DMI is between 0 and 1; Step 5: Based on the analysis results of the deviation value, mark the abnormal inventory data and verify the abnormal inventory data; Step 6. Organize the verified warehouse material inventory results and output the warehouse material verification and inventory report.

2. The method for intelligent dynamic inventory of unmanned aerial vehicles according to claim 1, characterized in that: In step 1, the planning process of the UAV flight inventory route includes: Obtain warehouse blueprints and conduct on-site surveys to supplement warehouse shelf status, and verify shelf height, number of layers, actual placement spacing, and obstacles in the flight environment; Combined with warehouse inventory data, the storage rules of materials in different areas are clarified according to material categories, batches, and shelf life attributes, and key inventory areas and regular inventory areas are determined. Among them, key inventory areas include areas where frequent in and out of the warehouse, valuables, and fragile items are stored; Based on the actual state of the warehouse and the distribution of materials, the grid model is used to describe the inventory area, and the inventory area is divided into multiple inventory sub-areas. Grid partitioning is adopted, and each row of shelves and the aisles between them are used as independent grids. The inventory area of ​​large shelves is partitioned, and the inventory area for scattered storage is set as a separate partition; According to the divided inventory areas, a reciprocating zigzag flight route is designed so that the flight trajectory covers all points. Combined with the effective range of visual recognition and the coverage radius of radio frequency signals, the drone is kept 0.5 meters away from the shelf during the flight process.

3. The method for intelligent dynamic inventory of unmanned aerial vehicles according to claim 2, characterized in that: The marking process of the uninventoryed area includes: The UAV is equipped with a visual sensor, which can be used to identify images collected by the visual sensor during flight to determine whether there are new obstructions on the flight path. If an obstacle is detected on the flight path, the flight control system of the drone will dynamically adjust the path according to the preset algorithm and automatically avoid the obstacle. By real-time recording of the flight trajectory and the area where data has been collected, the initially planned inventory area is compared, and the GPS information on the drone is used to synchronously mark the uninventoryed areas that have not been covered by the flight trajectory; According to the marked uninventoryed areas, arrange warehouse managers to check the obstacle status and material status of the marked areas and conduct manual re-inventory.

4. The method for intelligent dynamic inventory of unmanned aerial vehicles according to claim 3, characterized in that: In the step 2, the process of acquiring radio frequency data and visual data includes: Configure the UAV flight control system, input the flight route and parameters, connect the UAV to the ground control station through a wireless communication link, control the UAV to take off in a predetermined manner through the ground control station, and remotely start the onboard radio frequency identification device and visual recognition system; Use RFID equipment to scan the RFID tags on warehouse materials in real time, receive and process RFID signals, extract RFID data containing material numbers and basic information about categories, store the RFID data in the local storage device of the drone, and transmit it to the ground control station in real time; Use the visual recognition system to continuously capture images of materials in the passing area, record the visual features of the materials from different angles and distances, including shape, color, and label, form visual data, store the visual data in the local storage device of the drone, and transmit it to the ground control station in real time; After the drone completes the inventory flight of the entire area according to the planned flight route, it is controlled to fly back to the take-off point, and the RF data and visual data stored locally on the drone and the data received by the ground control station are summarized. A preliminary integrity check is performed on the data to check whether there is any data loss or damage. If there is a problem, the cause is analyzed and an attempt is made to recover the data.

5. The method for intelligent dynamic inventory of unmanned aerial vehicles according to claim 4, characterized in that: In step 3, the process of acquiring radio frequency features and visual features includes: Performing pre-processing operations of signal denoising and calibration on RF data, wherein filtering technology is used to remove noise components in RF signals, and calibration is performed to eliminate differences between devices, including frequency calibration, gain calibration, and phase calibration; Extract the tag information in the RF tag from the RF data, including the material number, category code, and production batch. By parsing the structure and protocol of the RF tag data, extract the tag information as the main RF feature, and extract the amplitude, energy, and waveform features from the time domain of the RF signal; Performing preprocessing operations of image denoising, image enhancement and image conversion on visual data, wherein the image is denoised using a filtering technique of mean filtering, the image is contrast enhanced and sharpened, and the image is converted from RGB color space to HSV color space through image conversion; Use the Sobel edge detection algorithm to extract edge information from the image. Based on the edge detection results, further extract the shape features of the object, including contour, area, and perimeter. Use the local binary pattern texture description method to extract texture features in the image and analyze the texture and detail information on the surface of the object. Describe the color distribution and color attributes of the object by extracting the color moment features in the image. The RF features extracted from the RF data, including label information, time domain features, and frequency domain features, are integrated with the visual features extracted from the visual data, including edge information, shape features, texture features, and color features, to form a complete feature vector.

6. The method for intelligent dynamic inventory of unmanned aerial vehicles according to claim 5, characterized in that: The marking process of abnormal inventory data includes: According to the statistical analysis of historical inventory data, material characteristics and warehouse management accuracy requirements, set corresponding abnormality judgment thresholds for material quantity deviation value, material size deviation value, material location deviation value and material category deviation value as the preliminary standard for judging whether the data is abnormal; For each set of material inventory records corresponding to the RF features and visual feature data for which the difference matching index has been calculated, the deviation values ​​of the material quantity, size, location, and category difference indicators are compared with the corresponding pre-set abnormal judgment thresholds. If the deviation value of any difference indicator of the material exceeds the corresponding abnormal judgment threshold, the group of inventory data is marked as abnormal inventory data, and the abnormal inventory data is highlighted in red in the inventory data report; Arrange warehouse staff to go to the location of abnormally marked materials to check the placement and appearance of the materials, check whether the materials are blocked, and check whether the material labels are damaged or stained; Compare the basic information of materials in the warehouse inventory management system with the actual materials and the collected radio frequency and visual data to confirm whether there is any information inconsistency. If the material category recorded in the warehouse inventory management system does not match the actual material appearance and the category code in the radio frequency tag, further investigation is required to determine whether the data anomaly is caused by input errors or chaotic material storage; For inventory data marked as abnormal, we will thoroughly investigate the causes of the data anomalies. After completing the inspection, we will combine the analysis results from all aspects to make a final judgment and handling opinion on the abnormal inventory data, and take corresponding measures to correct the abnormal situation.

7. The method for intelligent dynamic inventory of unmanned aerial vehicles according to claim 6, characterized in that: In step 6, the output process of the warehouse material verification and inventory report includes: Collect verified and confirmed warehouse inventory data from warehouse management systems, RFID equipment, and visual recognition systems, and integrate data from different sources; According to the sorted data, list the types, quantities, and storage location information of materials, including material number, name, specification model, quantity, storage shelf / area, and number of layers, and use charts to visually display the quantity distribution and storage location information of materials, add explanatory text below the charts, and form a complete warehouse material verification and inventory report; Check the warehouse material verification and inventory report, and publish it in electronic form to relevant personnel, and then track the implementation of warehouse management based on the suggestions and measures put forward in the warehouse material verification and inventory report.

Images