Intelligent goods shelf dynamic monitoring method and system based on Internet of Things and gravity sensing

Abstract

The invention relates to the technical field of intelligent monitoring, and discloses an intelligent goods shelf dynamic monitoring method and system based on the Internet of Things and gravity induction, and the method comprises the steps: carrying out the data windowing of a goods shelf gravity induction data set of a target goods shelf, and obtaining a data sequence; calculating data relevance between the data sequences and calculating abnormal data; according to the abnormal data and the data relevance, collecting a corresponding goods shelf article image, and identifying goods shelf article information corresponding to the goods shelf article image; performing data calibration on the goods shelf gravity sensing data set according to the goods shelf article information to obtain calibrated goods shelf gravity data; and calculating storage location information of the target goods shelf according to the calibrated goods shelf gravity data, and performing storage location early warning according to the storage location information. The dynamic monitoring accuracy of the gravity sensing intelligent goods shelf can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent monitoring, and particularly to an intelligent shelf dynamic monitoring method and system based on the Internet of Things and gravity sensing. Background Art

[0002] With the rapid development of technology and the needs of enterprises, modern enterprises' requirements for warehousing systems have changed from simply pursuing efficiency to achieving rapid response on the basis of ensuring high accuracy. As an advanced warehousing solution, the gravity sensing shelf system perfectly meets the development needs of modern logistics with its excellent performance in high-density storage, space utilization, operation accuracy, etc., and relatively low investment and operating costs. It can not only effectively improve warehousing efficiency, but also reduce the operating costs of enterprises to a certain extent, thus winning advantages for enterprises in the fierce market competition. Therefore, the gravity shelf system has broad application prospects in the field of modern logistics and is gradually becoming the preferred solution for enterprises to improve warehousing and logistics efficiency.

[0003] Especially in the field of medical consumable storage, the gravity shelf system combined with the Internet of Things and gravity sensing technology can effectively improve warehousing efficiency and management level. For example, through gravity sensing technology, the accuracy of inventory counting can be improved, manual operation errors can be reduced, and operating costs can be lowered. At the same time, it can be docked with the hospital management information system and logistics information system to achieve the full life cycle management of medical consumables and ensure medical quality and safety. Therefore, how to improve the accuracy of intelligent shelf dynamic monitoring has become an urgent problem to be solved. Summary of the Invention

[0004] The present invention provides an intelligent shelf dynamic monitoring method and system based on the Internet of Things and gravity sensing, and its main purpose is to solve the problem of poor accuracy in the elevator task control of medical logistics robot scheduling.

[0005] To achieve the above object, the intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing provided by the present invention includes:

[0006] Obtain the shelf gravity sensing data set of the target shelf, perform data windowing on the shelf gravity sensing data set to obtain the data sequence of the shelf gravity sensing data set;

[0007] Calculate the data correlation between the data sequences and calculate the abnormal data in the shelf gravity sensing data set;

[0008] Collect the corresponding shelf item images according to the abnormal data and the data correlation, and identify the shelf item information corresponding to the shelf item images;

[0009] Calibrate the shelf gravity induction data set according to the shelf item information to obtain calibrated shelf gravity data;

[0010] Calculate the bin location information of the target shelf according to the calibrated shelf gravity data, and perform bin location warning on the target shelf according to the bin location information.

[0011] Optionally, performing data windowing on the shelf gravity induction data set to obtain a data sequence of the shelf gravity induction data set, including:

[0012] Perform data standardization processing on the shelf gravity induction data set to obtain a target gravity induction data set;

[0013] Perform source division and time sorting on the target gravity induction data set to obtain a sensor data set;

[0014] Use a preset time window to perform windowing on the sensor data set to obtain a data sequence of the shelf gravity induction data set.

[0015] Optionally, calculating the data correlation between the data sequences and calculating the abnormal data in the shelf gravity induction data set, including:

[0016] Align the time stamps of the data sequences to obtain a time data sequence, and calculate the data correlation between the data sequences according to the time data sequence;

[0017] Calculate the sequence median and median offset of each shelf gravity induction data in the shelf gravity induction data according to the data sequence;

[0018] Identify the abnormal data in the shelf gravity induction data set according to the sequence median and the median offset.

[0019] Optionally, calculating the data correlation between the data sequences according to the time data sequence, including;

[0020] Calculate the data correlation degree between the time data sequences;

[0021] Calculate the data correlation degree between the time data sequences using the following formula:

[0022]

[0023] where r ij represents the data correlation degree between the i-th time data sequence and the j-th time data sequence, represents the i-th time data sequence and the j-th time data sequence The covariance between them, k represents the number of data in the time data sequence, and Var(·) represents variance;

[0024] Construct an association matrix corresponding to each of the time data sequences according to the data association degree, and calculate an association mean matrix of the time data sequences according to the association matrix;

[0025] Determine the data correlation between the data sequences calculated according to the time data sequence according to the association mean matrix.

[0026] Optionally, calculating the sequence median and median offset of each shelf gravity sensing data in the shelf gravity sensing data according to the data sequence includes:

[0027] Determine the time window size according to the number of data in the data sequence;

[0028] Divide the sensor data set corresponding to the gravity sensing data set according to the time window size to obtain a window data sequence corresponding to each shelf gravity sensing data;

[0029] Calculate the sequence median corresponding to each shelf gravity sensing data according to the window data sequence;

[0030] Calculate the median offset corresponding to each shelf gravity sensing data according to the sequence median;

[0031] Calculate the median offset using the following formula:

[0032] S = max(1.4826 × median{|Y g - M|}, d)

[0033] where S represents the median offset, max(·) represents the maximum value, Median(·) represents the median, Y g represents the window data sequence of the g-th shelf gravity sensing data, M represents the sequence median, and d represents a preset parameter.

[0034] Optionally, identifying the shelf item information corresponding to the shelf item image includes:

[0035] Perform a Fourier transform on the shelf item image to obtain an amplitude component and a phase component corresponding to the shelf item image;

[0036] Perform image enhancement on the shelf item image according to the amplitude component and the phase component to obtain an enhanced image;

[0037] Perform background segmentation on the enhanced image to obtain a target binary image, and construct a target item image according to the target binary image and the enhanced image;

[0038] Identify the shelf item information corresponding to the shelf item image based on the target item image.

[0039] Optionally, the image enhancement of the shelf item image according to the amplitude component and the phase component to obtain an enhanced image includes:

[0040] Perform a spatial transformation on the amplitude component and the phase component to obtain transformed components;

[0041] Perform a spatial transformation on the amplitude component and the phase component using the following formula:

[0042] F f = IFFT(Conv(Conv(LeakyReLU(I Amp ))),I Pha )

[0043] where F f represents the transformed component, IFFT represents the inverse Fourier transform, Conv(·) represents a preset convolution module, LeakyReLU represents an activation function, I Amp represents the amplitude component, and I Pha represents the phase component;

[0044] Perform residual merging on the transformed component and the shelf item image to obtain merged features;

[0045] Perform amplitude component constraint on the merged features to obtain a target amplitude component;

[0046] Perform an inverse Fourier transform on the target amplitude component and the phase component of the shelf item image to obtain the enhanced image corresponding to the shelf item image.

[0047] Optionally, the background segmentation of the enhanced image to obtain a target binary image includes:

[0048] Perform multi-layer convolutional downsampling processing on the enhanced image to obtain multiple downsampled features;

[0049] Perform bilinear interpolation and convolutional processing on each of the downsampled features to obtain one-dimensional convolutional features;

[0050] Perform bilinear interpolation reduction on the one-dimensional convolutional features to obtain a target feature map, and perform feature stitching on the target feature map to obtain stitched features;

[0051] Perform feature activation according to the stitched features to obtain the target binary image corresponding to the enhanced image.

[0052] Optionally, calibrating the shelf gravity induction data set according to the shelf item information to obtain calibrated shelf gravity data includes:

[0053] Searching for the data to be calibrated in the shelf gravity induction data set according to the shelf item information;

[0054] Replacing the data to be calibrated according to the standard mass in the shelf item information to obtain a calibrated gravity data set corresponding to the shelf gravity induction data set;

[0055] Identifying calibration outliers in the calibrated gravity data set and calculating the median of the calibration sequence in the calibrated gravity data set;

[0056] Replacing the calibration outliers according to the median of the calibration sequence to obtain calibrated shelf gravity data.

[0057] To solve the above problems, the present invention also provides an intelligent shelf dynamic monitoring system based on the Internet of Things and gravity induction. The system includes:

[0058] A data windowing module, configured to obtain a shelf gravity induction data set of a target shelf, perform data windowing on the shelf gravity induction data set, and obtain a data sequence of the shelf gravity induction data set;

[0059] A data correlation and abnormal data calculation module, configured to calculate the data correlation between the data sequences and calculate the abnormal data in the shelf gravity induction data set;

[0060] A shelf item information recognition module, configured to collect a corresponding shelf item image according to the abnormal data and the data correlation, and recognize the shelf item information corresponding to the shelf item image;

[0061] A data calibration module, configured to calibrate the shelf gravity induction data set according to the shelf item information to obtain calibrated shelf gravity data;

[0062] A bin location warning module, configured to calculate the bin location information of the target shelf according to the calibrated shelf gravity data, and perform bin location warning on the target shelf according to the bin location information.

[0063] In the embodiments of the present invention, by windowing the gravity sensor data set of the shelf, the gravity sensor data in different time periods can be obtained, which is convenient for observing the changes between the gravity sensor data; calculating the data correlation between the data sequences and calculating the abnormal data can effectively identify the abnormal data points in the gravity sensor data set, thereby improving the accuracy of the dynamic monitoring of the target shelf; collecting the corresponding shelf item images and identifying the corresponding shelf item information is beneficial for data calibration and improving the accuracy of the target shelf monitoring; calibrating the gravity sensor data set of the shelf according to the shelf item information can improve the accuracy of subsequent gravity sensing; and then performing bin location early warning based on the calibrated shelf gravity data, so as to accurately perform the dynamic monitoring of the shelf. Therefore, the intelligent shelf dynamic monitoring method and system based on the Internet of Things and gravity sensing proposed by the present invention can solve the problem of poor accuracy in the dynamic monitoring of gravity sensing intelligent shelves. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic flowchart of the intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing provided by an embodiment of the present invention;

[0065] Figure 2 It is a schematic flowchart of calculating the data correlation between the data sequences and calculating the abnormal data in the shelf gravity sensor data set provided by an embodiment of the present invention;

[0066] Figure 3 It is a schematic flowchart of identifying the shelf item information corresponding to the shelf item image provided by an embodiment of the present invention;

[0067] Figure 4 It is a functional module diagram of the intelligent shelf dynamic monitoring system based on the Internet of Things and gravity sensing provided by an embodiment of the present invention.

[0068] The realization, functional characteristics and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0069] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0070] The embodiments of the present application provide an intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing. The execution subject of the intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing includes, but is not limited to, at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiments of the present application. In other words, the intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to: a single server, a server cluster, a cloud server, or a cloud server cluster, etc. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0071] Referring to Figure 1 As shown, it is a schematic flowchart of an intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing provided by an embodiment of the present invention. In this embodiment, the intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing includes:

[0072] S1. Obtain the shelf gravity sensing data set of the target shelf, and perform data windowing on the shelf gravity sensing data set to obtain the data sequence of the shelf gravity sensing data set.

[0073] In the embodiments of the present invention, the target shelf is a shelf equipped with a gravity sensor. For example, it can be a consumable central warehouse, a picking shelf, etc. The number of product specifications on the target shelf may be thousands. It is necessary to regularly check the quantity corresponding to the product specifications according to the gravity sensors set on the shelf to understand the actual inventory.

[0074] Specifically, the shelf gravity sensing data set is the data collected by each gravity sensor on the target shelf for a period of time. For example, it can be one day, seven days, or other preset time periods, which is convenient for regular monitoring or real-time monitoring, and reflects the changes in the data collected by each gravity sensor. Data windowing is to add a time window to the data time series composed of the shelf gravity sensing data set, and divide the shelf gravity sensing data set at different time points, which is beneficial to analyzing the data changes in the shelf gravity sensing data set.

[0075] In the embodiments of the present invention, performing data windowing on the shelf gravity sensing data set to obtain the data sequence of the shelf gravity sensing data set includes:

[0076] Perform data standardization processing on the shelf gravity sensing data set to obtain a target gravity sensing data set;

[0077] Perform source partitioning and time sorting on the target gravity sensing dataset to obtain a sensor dataset;

[0078] Use a preset time window to window the sensor dataset to obtain the data sequence of the shelf gravity sensing dataset.

[0079] In the embodiment of the present invention, data normalization processing maps each data in the shelf gravity sensing dataset to a fixed data interval, for example, the [0, 1] data interval, to eliminate data differences in the shelf gravity sensing dataset and reduce the amount of calculation.

[0080] Further, data source partitioning is performed according to each gravity sensor to obtain the data corresponding to each gravity sensor, and then sorting is performed according to the time of generation to obtain the sensor dataset corresponding to each gravity sensor. Window the sensor dataset through a time window with a preset window size and time step to obtain the data sequence formed by each time window.

[0081] In the embodiment of the present invention, by windowing the shelf gravity sensing dataset, gravity sensing data within different time periods can be obtained, which is convenient for observing the changes between gravity sensing data, thereby monitoring the changes of the target shelf.

[0082] S2. Calculate the data correlation between the data sequences and calculate the abnormal data in the shelf gravity sensing dataset.

[0083] In the embodiment of the present invention, abnormal data is gravity sensing data that does not conform to normal expectations or rules. For example, data caused by gravity sensor failures, environmental interference, incorrect placement of consumables, etc.

[0084] In the embodiment of the present invention, refer to Figure 2 As shown, calculating the data correlation between the data sequences and calculating the abnormal data in the shelf gravity sensing dataset includes:

[0085] S21. Align the time stamps of the data sequences to obtain a time data sequence, and calculate the data correlation between the data sequences according to the time data sequence;

[0086] S22. Calculate the sequence median and median offset of each shelf gravity sensing data in the shelf gravity sensing data according to the data sequences;

[0087] S23. Identify the abnormal data in the shelf gravity sensing dataset according to the sequence median and the median offset.

[0088] In the embodiments of the present invention, time scale alignment is to align data according to the data generation time, that is, each data in the data sequences from different gravity sensors is sorted according to the generation time to obtain a time data sequence with consistent time.

[0089] Specifically, calculating the data correlation between the data sequences according to the time data sequence includes:

[0090] Calculating the data correlation degree between the time data sequences;

[0091] Constructing a correlation degree matrix corresponding to each time data sequence according to the data correlation degree, and calculating the correlation mean matrix of the time data sequences according to the correlation degree matrix;

[0092] Determining the data correlation between the data sequences according to the time data sequence according to the correlation mean matrix.

[0093] Specifically, the following formula is used to calculate the data correlation degree between time data sequences:

[0094]

[0095] where r ij represents the data correlation degree between the i-th time data sequence and the j-th time data sequence, represents the i-th time data sequence and the j-th time data sequence the covariance between them, k represents the number of data in the time data sequence, and Var(·) represents the variance.

[0096] In the embodiments of the present invention, a correlation degree matrix is constructed with the data correlation degree as matrix elements. Among them, the element in the x-th row and y-th column of the correlation degree matrix represents the data correlation degree between the x-th time data sequence and the y-th time data sequence, and thus can represent each.

[0097] Furthermore, the following formula is used to calculate the correlation mean matrix of the time data sequences:

[0098]

[0099] where, represents the correlation mean matrix, N represents the total number of time data sequences, and R i represents the matrix element corresponding to the i-th time data sequence in the correlation degree matrix.

[0100] In the embodiments of the present invention, the mean value of the data correlation degrees calculated between each time data sequence in the correlation degree matrix and other time data sequences is calculated. For example, R iIt represents the data correlation degree between the i-th time data sequence and other time data sequences in the correlation matrix. By doing so, the correlation degree between each time data sequence and other time data sequences in different time periods can be comprehensively considered, which can better reflect the correlation degree between different time data sequences, avoid misjudgment, and improve the accuracy of data correlation analysis.

[0101] Further, calculating the sequence median and median offset of each shelf gravity sensing data in the shelf gravity sensing data according to the data sequence includes:

[0102] Determining the time window size according to the number of data in the data sequence;

[0103] Dividing the sensor data set corresponding to the gravity sensing data set according to the time window size to obtain the window data sequence corresponding to each shelf gravity sensing data;

[0104] Calculating the sequence median corresponding to each shelf gravity sensing data according to the window data sequence;

[0105] Calculating the median offset corresponding to each shelf gravity sensing data according to the sequence median.

[0106] In the embodiment of the present invention, the number of data in the data sequence is used as the window size to divide the sensor data set corresponding to the gravity sensing data set, and the window data sequence corresponding to each shelf gravity sensing data is obtained. For example, when the time window size is g, the time window sequence corresponding to the i-th shelf gravity sensing data ri is expressed as Ri = {ri - g, …, ri - 1, ri, ri + 1, …, ri + g}. Among them, the sensor data set is the gravity sensing data set after the above data standardization processing, source division, and time sorting.

[0107] Specifically, the median in the window data sequence corresponding to each shelf gravity sensing data is used as the sequence median corresponding to the shelf gravity sensing data, and then the median offset is calculated through the sequence median.

[0108] Specifically, the following formula is used to calculate the median offset:

[0109] S = max(1.4826 × median{|Y g - M|}, d)

[0110] where S represents the median offset, max(·) represents the maximum value, median(·) represents the median, Y g represents the window data sequence of the g-th shelf gravity sensing data, M represents the sequence median, and d represents a preset parameter.

[0111] In an embodiment of the present invention, the identification of abnormal data in the shelf gravity sensing data set according to the sequence median and the median offset includes:

[0112] Calculating a data difference value for each shelf gravity sensing data according to the sequence median and the median offset;

[0113] Determining the abnormal data in the shelf gravity sensing data set according to the data difference value.

[0114] Further, the data difference value is calculated using the following formula:

[0115] l g =|y g -M|-δ×S

[0116] where l g represents the data difference value corresponding to the gravity sensing data y g of the g-th shelf, M represents the sequence median, δ represents a preset difference value parameter, and S represents the median offset.

[0117] In an embodiment of the present invention, the shelf gravity sensing data with a data difference value greater than or equal to zero can be regarded as abnormal data in the gravity sensing data set, which ensures the continuity of the data, avoids the interference of continuous identical data on the identification of abnormal data, effectively identifies the abnormal data points in the gravity sensing data set, and further improves the accuracy of the dynamic monitoring of the target shelf.

[0118] S3. Collecting corresponding shelf item images according to the abnormal data and the data correlation, and identifying the shelf item information corresponding to the shelf item images.

[0119] In an embodiment of the present invention, since the abnormal data can be gravity sensing data that does not conform to normal expectations or rules, the data correlation may represent the degree of association between different gravity sensing data. For consumables with a high degree of association, the replenishment time and quantity can be reasonably arranged according to the inventory situation and inventory level of the associated consumables, avoiding inventory backlogs or out-of-stock phenomena.

[0120] Specifically, the shelf item image is a picture collected by a camera device for monitoring the target shelf according to the abnormal data and the data correlation. The corresponding shelf item information is identified through the shelf item image, so as to dynamically monitor the shelf item information.

[0121] In an embodiment of the present invention, the collection of the corresponding shelf item image according to the abnormal data and the data correlation includes:

[0122] Determining associated data according to the data correlation;

[0123] Collect the monitoring data corresponding to the target shelf according to the associated data and the abnormal data;

[0124] Extract the corresponding shelf item image according to the monitoring data.

[0125] In an embodiment of the present invention, a preset threshold can be used to compare with the mean elements in the associated mean matrix, and the time series greater than the preset correlation threshold obtains the associated data, indicating that the consumables on the target shelf by different gravity sensors or at different times have strong correlation and need to be focused on.

[0126] Further, according to the time points corresponding to the associated data and the abnormal data collected by the preset camera device and the monitoring data corresponding to the current moment, intercept the monitoring data to obtain the corresponding shelf item image.

[0127] In an embodiment of the present invention, refer to Figure 3 As shown, the identifying the shelf item information corresponding to the shelf item image includes:

[0128] S31. Perform Fourier transform on the shelf item image to obtain the amplitude component and phase component corresponding to the shelf item image;

[0129] S32. Perform image enhancement on the shelf item image according to the amplitude component and the phase component to obtain an enhanced image;

[0130] S33. Perform background segmentation on the enhanced image to obtain a target binary image, and construct a target item image according to the target binary image and the enhanced image;

[0131] S34. Identify the shelf item information corresponding to the shelf item image according to the target item image.

[0132] In an embodiment of the present invention, the shelf item image is converted to the Fourier space by performing a fast Fourier transform, and the amplitude component and phase component of each shelf item image can be extracted.

[0133] In an embodiment of the present invention, the performing image enhancement on the shelf item image according to the amplitude component and the phase component to obtain an enhanced image includes:

[0134] Perform spatial conversion on the amplitude component and the phase component to obtain a conversion component;

[0135] Perform residual merging on the conversion component and the shelf item image to obtain a merged feature;

[0136] Perform amplitude component constraint on the merged feature to obtain a target amplitude component;

[0137] Performing an inverse Fourier transform based on the target amplitude component and the phase component of the shelf item image to obtain an enhanced image corresponding to the shelf item image.

[0138] Further, the following formula is used to perform a spatial transformation on the amplitude component and the phase component:

[0139] F f =IFFT(Conv(Conv(LeakyReLU(I Amp ))),I Pha )

[0140] where F f represents the transformed component, IFFT represents the inverse Fourier transform, Conv(·) represents a preset convolution module, LeakyReLU represents an activation function, I Amp represents the amplitude component, and I Pha represents the phase component.

[0141] In the embodiments of the present invention, through spatial transformation, the pixel position information of the shelf item image can be maintained, and the amplitude component and the phase component can be transformed into the spatial domain, which can be used for residual merging with the shelf item image. Specifically, the transformed component can be dimension-reduced through a 3x3 convolution and then added element-wise to the shelf item image to obtain a merged feature.

[0142] Further, the merged feature is dimension-reduced through a convolution operation, and then after a fast Fourier transform operation, an amplitude operation is performed to obtain the amplitude component corresponding to the merged feature. Finally, the amplitude component is constrained within (0,1) through a sigmoid activation function to obtain the target amplitude component. Through the inverse Fourier transform of the target amplitude component and the phase component of the shelf item image, the image quality of the shelf item image can be enhanced, and thus the accuracy of subsequent shelf item information recognition can be improved.

[0143] In the embodiments of the present invention, the shelf item information includes the category of consumables on the shelf, the corresponding standard quality, the placement position, etc., and the specific items corresponding to the target shelf are identified through the shelf item information.

[0144] Specifically, the background segmentation of the enhanced image to obtain a target binary image includes:

[0145] Performing multi-layer convolutional downsampling processing on the enhanced image to obtain a plurality of downsampled features;

[0146] Performing bilinear interpolation and convolutional processing on each of the downsampled features to obtain one-dimensional convolutional features;

[0147] Perform bilinear interpolation reduction on the one-dimensional convolutional features to obtain a target feature map, and perform feature splicing on the target feature map to obtain spliced features;

[0148] Perform feature activation based on the spliced features to obtain the target binary image corresponding to the enhanced image.

[0149] Specifically, the multi-layer convolutional downsampling is multiple independently pre-constructed convolutional encoding modules. Among them, a max pooling layer is set after each convolutional encoding module to perform downsampling on the enhanced image. Each convolutional encoding module corresponds to a feature decoding module. Bilinear interpolation is set before the feature decoding module for upsampling, and a one-dimensional convolutional feature is obtained for each layer of convolution through a convolutional layer. The one-dimensional convolutional feature is restored to the image size corresponding to the enhanced image through bilinear interpolation to obtain a target feature map, and the target feature maps of each layer are spliced to obtain spliced features.

[0150] Furthermore, perform binary probability prediction on the spliced features through a 1×1 convolutional layer and a Sigmoid activation function to obtain the target binary image corresponding to the target shelf items and the background.

[0151] In the embodiment of the present invention, the target binary image and the enhanced image are multiplied point by point by channel to obtain a target item image, and then the pre-constructed goods recognition network is used for recognition to obtain the corresponding shelf item information. It can be a network model composed of a convolutional layer, a pooling layer, a fully connected layer, and a residual network, and then the shelf item information included in the shelf item image is recognized, which is beneficial to data calibration and improves the accuracy of target shelf monitoring.

[0152] S4. Perform data calibration on the shelf gravity sensing data set according to the shelf item information to obtain calibrated shelf gravity data.

[0153] In the embodiment of the present invention, data calibration is to calibrate the gravity sensing data that is inconsistent in the shelf gravity sensing data set according to the shelf item information, and at the same time, the gravity sensor can be detected and calibrated to improve the accuracy of subsequent gravity monitoring.

[0154] In the embodiment of the present invention, the performing data calibration on the shelf gravity sensing data set according to the shelf item information to obtain calibrated shelf gravity data includes:

[0155] Search for the data to be calibrated in the shelf gravity sensing data set according to the shelf item information;

[0156] Replace the data to be calibrated according to the standard mass in the shelf item information to obtain the calibrated gravity data set corresponding to the shelf gravity sensing data set;

[0157] Identify calibration outliers in the calibration gravity dataset and calculate the median of the calibration sequence in the calibration gravity dataset;

[0158] Replace the calibration outliers according to the median of the calibration sequence to obtain calibrated shelf gravity data.

[0159] Specifically, the data to be calibrated is the data of the shelf item information in the shelf gravity sensing dataset, that is, the data of the collected shelf item image in the shelf gravity sensing dataset. For example, if the shelf item information is the information of consumable 1 at time a, then search for the data of consumable 1 at time a in the shelf gravity sensing dataset to obtain the data to be calibrated.

[0160] Furthermore, when using the above steps of calculating abnormal data in the shelf gravity sensing dataset to calculate abnormal data in the calibration gravity dataset to obtain calibration outliers, and the above steps of calculating the median of the calibration sequence corresponding to the shelf gravity sensing data in each abnormal data to calculate the median of the calibration sequence in the calibration gravity dataset, the calibration gravity dataset is calibrated again through the median of the calibration sequence. At the same time, it is not necessary to collect the corresponding video images, which can improve the efficiency while ensuring the data accuracy.

[0161] In the embodiment of the present invention, according to the associated data corresponding to the shelf item information and the data corresponding to the abnormal data in the shelf gravity sensing dataset, the data to be calibrated is obtained. The data to be calibrated is compared with the standard quality of the corresponding item in the shelf item information. If they are consistent, it remains unchanged; if they are inconsistent, it is replaced according to the standard quality data of the corresponding item in the shelf item information to obtain the calibrated calibration gravity dataset.

[0162] In the embodiment of the present invention, the information when the item is put into storage can also be compared through the shelf item information. For example, if the input quality data is inconsistent with the data collected by the gravity sensor in the shelf gravity sensing dataset, it may be that the gravity sensor fails and needs to be calibrated to improve the accuracy of subsequent gravity sensing.

[0163] S5. Calculate the bin information of the target shelf according to the calibrated shelf gravity data, and perform bin warning on the target shelf according to the bin information.

[0164] In the embodiment of the present invention, the bin information of the target shelf is whether each bin for placing medical consumables in the target shelf is vacant and whether the items on the bin need to be replenished, so as to take inventory of the inventory of medical consumables on the target shelf and monitor the warehousing information of the target shelf in real time.

[0165] In the embodiment of the present invention, the calculating the bin information of the target shelf according to the calibrated shelf gravity data includes:

[0166] Determine the existing gravity data of each storage location on the target shelf according to the calibrated shelf gravity data;

[0167] Calculate the storage location information of the target shelf according to the existing gravity data.

[0168] In the embodiment of the present invention, the current gravity data of each storage location on the target shelf will be determined according to the calibrated shelf gravity data, and the weight of consumables that each storage location can still bear will be calculated according to the existing gravity data. Furthermore, it is analyzed whether the storage location is vacant and whether the items on the storage location need to be replenished, so as to obtain the storage location information.

[0169] Furthermore, it is possible to determine whether the consumables in each storage location on the target shelf need to be replenished according to the storage location information, and then give an immediate warning. Also, according to the storage location information, the storage locations for the consumables to be warehoused can be allocated, avoiding the underutilization of the target shelf. Additionally, the data of the warehoused and out-of-stock consumables can be calibrated to ensure the accuracy of the gravity sensing data of the target shelf. Among them, when giving a warning for the storage location, voice reminder and sound and light reminder can be used, so as to accurately conduct dynamic monitoring of the target shelf.

[0170] As Figure 4 shown, it is a functional module diagram of an intelligent shelf dynamic monitoring system based on the Internet of Things and gravity sensing provided by an embodiment of the present invention.

[0171] The intelligent shelf dynamic monitoring system 400 based on the Internet of Things and gravity sensing according to the present invention can be installed in an electronic device. According to the functions achieved, the intelligent shelf dynamic monitoring system 400 based on the Internet of Things and gravity sensing can include a data windowing module 401, a data correlation and abnormal data calculation module 402, a shelf item information recognition module 403, a data calibration module 404, and a storage location warning module 405. The modules in the present invention can also be referred to as units, which refer to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

[0172] In this embodiment, the functions of each module / unit are as follows:

[0173] The data windowing module 401 is used to obtain the shelf gravity sensing data set of the target shelf, perform data windowing on the shelf gravity sensing data set, and obtain the data sequence of the shelf gravity sensing data set;

[0174] The data correlation and abnormal data calculation module 402 is used to calculate the data correlation between the data sequences and calculate the abnormal data in the shelf gravity sensing data set;

[0175] The shelf item information recognition module 403 is configured to collect corresponding shelf item images according to the abnormal data and the data correlation, and recognize the shelf item information corresponding to the shelf item images;

[0176] The data calibration module 404 is configured to calibrate the shelf gravity sensing data set according to the shelf item information to obtain calibrated shelf gravity data;

[0177] The bin location warning module 405 is configured to calculate the bin location information of the target shelf according to the calibrated shelf gravity data, and perform bin location warning on the target shelf according to the bin location information.

[0178] Specifically, each module in the intelligent shelf dynamic monitoring system 400 based on the Internet of Things and gravity sensing in the embodiments of the present invention adopts the same technical means as those Figures 1 to 3 described in the above-mentioned intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing, and can produce the same technical effects, which will not be elaborated here.

[0179] The present invention also provides an electronic device, which may include a processor, a memory, a communication bus, and a communication interface, and may further include a computer program stored in the memory and executable on the processor, such as an intelligent shelf dynamic monitoring method program based on the Internet of Things and gravity sensing.

[0180] Among them, the processor may be composed of integrated circuits in some embodiments. For example, it may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips, etc.

[0181] The memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (such as: SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory may be an internal storage unit of the electronic device in some embodiments, such as the mobile hard disk of the electronic device.

[0182] The communication bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable connection and communication between the memory, at least one processor, and the like.

[0183] The communication interface is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between this electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), and optionally, the user interface may also be a standard wired interface or a wireless interface.

[0184] Only the electronic device with components is shown in the figure. Those skilled in the art can understand that the structure shown in the figure does not constitute a limitation on the electronic device, and it may include fewer or more components than shown in the figure, or combine some components, or have a different component layout.

[0185] Specifically, for the specific implementation method of the above instructions by the processor, reference may be made to the description of the relevant steps in the corresponding embodiments of the accompanying drawings, which will not be elaborated here.

[0186] In several embodiments provided by the present invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0187] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, in each embodiment of the present invention, the functional modules can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware, or in the form of a combination of hardware and software functional modules.

[0189] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-described exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention.

[0190] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.

[0191] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.

[0192] In addition, it is obvious that the term "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or systems stated in the system claims can also be implemented by one unit or system through software or hardware. The terms such as "first" and "second" are used to denote names and do not represent any specific order.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not restrictive. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing, characterized in that, The method includes: Obtaining a shelf gravity induction data set of a target shelf, performing data windowing on the shelf gravity induction data set to obtain a data sequence of the shelf gravity induction data set; Calculating the data correlation between the data sequences and calculating abnormal data in the shelf gravity induction data set; Collecting corresponding shelf item images according to the abnormal data and the data correlation, and identifying shelf item information corresponding to the shelf item images; Performing data calibration on the shelf gravity induction data set according to the shelf item information to obtain calibrated shelf gravity data; Calculating the bin location information of the target shelf according to the calibrated shelf gravity data, and performing bin location warning on the target shelf according to the bin location information.

2. The intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing according to claim 1, characterized in that The performing data windowing on the shelf gravity induction data set to obtain a data sequence of the shelf gravity induction data set includes: Performing data standardization processing on the shelf gravity induction data set to obtain a target gravity induction data set; Performing source division and time sorting on the target gravity induction data set to obtain a sensor data set; Using a preset time window to perform windowing on the sensor data set to obtain a data sequence of the shelf gravity induction data set.

3. The intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing according to claim 1, characterized in that, The calculating the data correlation between the data sequences and calculating abnormal data in the shelf gravity induction data set includes: Aligning the time stamps of the data sequences to obtain time data sequences, and calculating the data correlation between the data sequences according to the time data sequences; Calculating the sequence median and median offset of each shelf gravity induction data in the shelf gravity induction data according to the data sequences; Identifying abnormal data in the shelf gravity induction data set according to the sequence median and the median offset.

4. The intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing according to claim 3, characterized in that The calculating the data correlation between the data sequences according to the time data sequences includes; Calculating the data correlation degree between the time data sequences; Calculating the data correlation degree between the time data sequences using the following formula: where r ij represents the data correlation degree between the i-th time data sequence and the j-th time data sequence, represents the i-th time data sequence and the j-th time data sequence the covariance between them, k represents the number of data in the time data sequence, and Var(·) represents the variance; Constructing a correlation degree matrix corresponding to each of the time data sequences according to the data correlation degree, and calculating a correlation mean matrix of the time data sequences according to the correlation degree matrix; Determining the data correlation between the data sequences according to the time data sequences according to the correlation mean matrix.

5. The intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing according to claim 3, characterized in that, The calculating the sequence median and median offset of each shelf gravity induction data in the shelf gravity induction data according to the data sequences includes: Determining the time window size according to the number of data in the data sequences; Dividing the sensor data set corresponding to the gravity induction data set according to the time window size to obtain a window data sequence corresponding to each shelf gravity induction data; Calculating the sequence median corresponding to each shelf gravity induction data according to the window data sequence; Calculating the median offset corresponding to each shelf gravity induction data according to the sequence median; Calculating the median offset using the following formula: S = max(1.4826 × median{|Y g - M|}, d) Where S represents the median offset, max(·) represents the maximum value, median(·) represents the median value, and Y g represents the window data sequence of the gravity sensing data of the g-th shelf, M represents the sequence median, and d represents a preset parameter.

6. The intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing according to claim 1, characterized in that The identifying shelf item information corresponding to the shelf item images includes: Performing Fourier transform on the shelf item images to obtain an amplitude component and a phase component corresponding to the shelf item images; Perform image enhancement on the shelf item image according to the amplitude component and the phase component to obtain an enhanced image; Perform background segmentation on the enhanced image to obtain a target binary image, and construct a target item image according to the target binary image and the enhanced image; Identify the shelf item information corresponding to the shelf item image according to the target item image.

7. The intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing according to claim 6, characterized in that, The performing image enhancement on the shelf item image according to the amplitude component and the phase component to obtain an enhanced image includes: Perform spatial transformation on the amplitude component and the phase component to obtain transformed components; Use the following formula to perform spatial transformation on the amplitude component and the phase component: F f = IFFT(Conv(Conv(LeakyReLU(I Amp ))), I Pha ) Among them, F f represents the conversion component, IFFT represents the inverse Fourier transform, Conv(·) represents a preset convolution module, LeakyReLU represents an activation function, and I Amp represents the amplitude component, and I Pha represents the phase component; Perform residual merging on the transformed components and the shelf item image to obtain merged features; Perform amplitude component constraint on the merged features to obtain a target amplitude component; Perform inverse Fourier transform on the target amplitude component and the phase component of the shelf item image to obtain the enhanced image corresponding to the shelf item image.

8. The intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing according to claim 6, characterized in that, The performing background segmentation on the enhanced image to obtain a target binary image includes: Perform multi-layer convolutional downsampling processing on the enhanced image to obtain multiple downsampled features; Perform bilinear interpolation and convolutional processing on each downsampled feature to obtain one-dimensional convolutional features; Perform bilinear interpolation reduction on the one-dimensional convolutional features to obtain a target feature map, and perform feature splicing on the target feature map to obtain spliced features; Perform feature activation according to the spliced features to obtain the target binary image corresponding to the enhanced image.

9. The intelligent shelf dynamic monitoring method based on the Internet of Things and gravity sensing according to claim 1, characterized in that The performing data calibration on the shelf gravity sensing data set according to the shelf item information to obtain calibrated shelf gravity data includes: Search for data to be calibrated in the shelf gravity sensing data set according to the shelf item information; Replace the data to be calibrated according to the standard mass in the shelf item information to obtain a calibrated gravity data set corresponding to the shelf gravity sensing data set; Identify calibration outliers in the calibrated gravity data set and calculate the median of the calibration sequence in the calibrated gravity data set; Perform outlier replacement on the calibration outliers according to the median of the calibration sequence to obtain calibrated shelf gravity data.

10. An intelligent shelf dynamic monitoring system based on the Internet of Things and gravity sensing, characterized in that, The system includes: A data windowing module, configured to obtain a shelf gravity sensing data set of a target shelf, and perform data windowing on the shelf gravity sensing data set to obtain a data sequence of the shelf gravity sensing data set; A data correlation and abnormal data calculation module, configured to calculate the data correlation between the data sequences and calculate the abnormal data in the shelf gravity sensing data set; A shelf item information identification module, configured to collect a corresponding shelf item image according to the abnormal data and the data correlation, and identify the shelf item information corresponding to the shelf item image; A data calibration module, configured to perform data calibration on the shelf gravity sensing data set according to the shelf item information to obtain calibrated shelf gravity data; A bin location warning module, configured to calculate the bin location information of the target shelf according to the calibrated shelf gravity data, and perform bin location warning on the target shelf according to the bin location information.

Images