RFID warehouse inventory system and method fusing q-learning algorithm and dynamic power regulation

Abstract

The application relates to the technical field of warehouse inventory, and particularly discloses an RFID warehouse inventory system and method fusing a Q learning algorithm and dynamic power regulation, which comprises a reading device that moves in a warehouse environment and comprises an environment sensing unit that acquires environment sensing parameters reflecting the spatial position relationship between the reading device and goods to be inventoried; and a multi-antenna radio frequency identification unit that transmits a radio frequency signal and receives a tag response signal to identify tag identity information and extract signal characteristic parameters representing signal propagation characteristics. The RFID warehouse inventory system and method fusing the Q learning algorithm and the dynamic power regulation mechanism effectively solve the problem that static parameters in traditional RFID warehouse inventory technology are difficult to adapt to complex warehouse environments, and realize real-time sensing and intelligent decision-making for the warehouse scene by fusing the Q learning algorithm and the dynamic power regulation mechanism.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of warehouse inventory, in particular to an RFID warehouse inventory system and method fusing Q-learning algorithm and dynamic power regulation. BACKGROUND

[0002] Although the existing RFID warehouse inventory technology has achieved a certain degree of automation, it still has significant limitations in dynamic adaptability, anti-interference ability, efficiency and accuracy balance, which are specifically manifested in:

[0003] For example, the patent document with publication number CN119809526B divides levels by the signal strength range of fixed position tags, but the power of the RFID reader is fixed and cannot be dynamically adjusted according to the tag density (such as collision in dense areas) and environmental interference (such as metal shelf reflection), resulting in missed reading in dense areas or excessive energy consumption in sparse areas; and the patent document with publication number CN120278642A optimizes positioning through a probability model, but the power regulation relies on preset thresholds and does not consider the impact of real-time environmental changes on signal quality. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art. To this end, the purpose of the present application is to propose an RFID warehouse inventory system and method fusing Q-learning algorithm and dynamic power regulation to solve the problem of insufficient adaptability of static power in complex environments when performing warehouse inventory.

[0005] To achieve the above-mentioned purpose, the first aspect of the present application proposes an RFID warehouse inventory system fusing Q-learning algorithm and dynamic power regulation, which comprises:

[0006] a reading device that moves in a warehouse environment, comprising:

[0007] an environment perception unit that acquires environment perception parameters reflecting the spatial position relationship between the reading device and the goods to be inventoried;

[0008] a multi-antenna radio frequency identification unit that transmits radio frequency signals and receives tag response signals to identify tag identity information and extract signal feature parameters representing signal propagation characteristics;

[0009] a processing control core configured to:

[0010] construct a dynamic state vector by integrating the environment perception parameters and the signal feature parameters to represent the physical conditions affecting the propagation of radio frequency signals in the current environment;

[0011] run a Q-learning decision model, receive the dynamic state vector as input, and output optimal action instructions based on a preset optimization goal;

[0012] The optimal action instruction is executed, the working parameters of the multi-antenna radio frequency identification unit are dynamically adjusted, and the tag inventory is executed, and the working parameters at least include the transmission power, the antenna beam pointing and the working radio frequency point.

[0013] To achieve the above object, the second aspect embodiment of the present application proposes an RFID warehouse inventory method combining Q learning algorithm and dynamic power adjustment, which comprises the following steps:

[0014] S1, environment and signal perception: driving the reading device, continuously acquiring the environment perception parameters reflecting the physical environment relationship between the reading device and the goods to be inventoried in the inventory process, and synchronously extracting the signal feature parameters reflecting the signal propagation characteristics from the received electronic tag signals;

[0015] S2, state space construction: the environment perception parameters and signal feature parameters obtained in step S1 are fused to construct a dynamic state vector for representing the physical conditions affecting the radio frequency signal propagation in the current environment;

[0016] S3, intelligent decision: inputting the dynamic state vector into a pre-trained Q learning decision model, and outputting the optimal action instruction aiming to maximize the inventory efficiency from the model;

[0017] S4, parameter adaptive adjustment and execution: analyzing the optimal action instruction, and dynamically adjusting the working parameters of the reading device in real time according to the analysis result, and the working parameters at least include the transmission power, the antenna beam pointing and the working radio frequency point, and executing the tag inventory operation under the parameters.

[0018] To achieve the above object, the third aspect embodiment of the present application proposes an electronic device, which comprises a memory, a processor and a computer program stored in the memory, and the computer program is executed by the processor to realize the RFID warehouse inventory method combining Q learning algorithm and dynamic power adjustment.

[0019] Compared with the prior art, the present application has the following beneficial effects:

[0020] The fusion Q learning algorithm and the dynamic power regulation RFID warehouse inventory system and method of the embodiment of the present application effectively solve the problem that the static parameters in the traditional RFID warehouse inventory technology are difficult to adapt to the complex warehouse environment, realize real-time perception and intelligent decision-making of the warehouse scene by fusing the Q learning algorithm and the dynamic power regulation mechanism, and drive the adaptive adjustment of the radio frequency identification parameters by means of the deep fusion of environmental perception and signal characteristics to construct a dynamic state, which can not only flexibly cope with dynamic scenes such as distance changes, multipath interference, and differences in goods density, improve the anti-interference ability and inventory accuracy, but also balance the inventory efficiency and resource consumption through the continuous evolution and optimization of the reinforcement learning decision-making, and overall improve the dynamic adaptability, intelligence and practicality of the warehouse inventory, effectively overcoming the limitations of the traditional technology in dynamic adjustment, anti-interference and efficiency and accuracy balance. BRIEF DESCRIPTION OF DRAWINGS

[0021] The disclosure of the present application will be described with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes, and are not intended to limit the scope of protection of the present application. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0022] Figure 1 is a schematic diagram of the RFID warehouse inventory system fusing the Q learning algorithm and the dynamic power regulation in an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of the change of the inter-antenna signal phase difference measured in a relatively low multipath interference environment in the RFID warehouse inventory system fusing the Q learning algorithm and the dynamic power regulation in an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of the change of the inter-antenna signal phase difference measured in a relatively high multipath interference environment in the RFID warehouse inventory system fusing the Q learning algorithm and the dynamic power regulation in an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of the convergence curve of the average cumulative reward of the Q learning decision-making model in the training process in the RFID warehouse inventory system fusing the Q learning algorithm and the dynamic power regulation in an embodiment of the present application;

[0026] Figure 5 is a schematic diagram of the performance comparison of the inventory accuracy of the system and two existing technologies in the comprehensive warehouse scene in the RFID warehouse inventory system fusing the Q learning algorithm and the dynamic power regulation in an embodiment of the present application;

[0027] Figure 6is a performance comparison diagram of the system and two prior arts in a RFID warehouse inventory system fusing Q learning algorithm and dynamic power regulation in an embodiment of the present application under a high multipath interference environment such as densely packed metal shelves, and the tag miss reading rate of the system under the environment;

[0028] Figure 7 is a final effect verification heat map of different interference suppression strategies adopted by the system in a RFID warehouse inventory system fusing Q learning algorithm and dynamic power regulation in an embodiment of the present application for different levels of multipath interference;

[0029] Figure 8 is a flowchart of a RFID warehouse inventory method fusing Q learning algorithm and dynamic power regulation in an embodiment of the present application;

[0030] Figure 9 is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0032] The RFID warehouse inventory system, method and electronic device fusing Q learning algorithm and dynamic power regulation of the embodiments of the present application are described below with reference to the accompanying drawings.

[0033] Figure 1 is a schematic diagram of a RFID warehouse inventory system fusing Q learning algorithm and dynamic power regulation in an embodiment of the present application. The system aims to solve the problems of low inventory efficiency, high miss reading rate and energy consumption redundancy in traditional RFID warehouse inventory caused by the difficulty of static parameters (such as fixed power and single frequency band) to adapt to complex warehouse environment (multipath interference, dynamic distance change, metal shelf shielding, etc.), and realizes dynamic optimization through the closed-loop mechanism of environment perception-intelligent decision-parameter adaptive adjustment combined with reinforcement learning strategy. The core of the system is to deeply fuse real-time environmental features and signal characteristics to drive intelligent adjustment of radio frequency identification parameters, and finally realize efficient, accurate and low-consumption warehouse tag inventory.

[0034] As shown in Figure 1 , the RFID warehouse inventory system fusing Q learning algorithm and dynamic power regulation includes a reading device and a processing control core, wherein:

[0035] 1. A reading device, the reading device being a mobile device, the reading device being mountable on an AGV trolley or a mechanical arm, the reading device being capable of autonomously cruising in a warehouse, the reading device carrying two sensing organs, including an environment sensing unit and a multi-antenna radio frequency identification unit, wherein:

[0036] (1) The environment sensing unit is equivalent to giving the reading device "eyes and a ruler", and is capable of collecting the spatial position relationship between the reading device and the goods to be checked and the warehouse environment structure information in real time, so as to provide a physical environment basis for signal attenuation evaluation and parameter adjustment.

[0037] Optionally, the environment sensing unit includes the following environment sensing parameters:

[0038] Distance parameter: quantifying the spatial interval between the reading device and the target goods through a laser radar, a millimeter wave radar or a visual sensor (such as a binocular camera), the accuracy can reach ±5 cm, and the free space path loss is evaluated.

[0039] Warehouse environment structure information: combined with a pre-set three-dimensional warehouse map, including shelf material, stacking height, channel width and the like, identifying obstructions such as metal shelves and dense goods, determining channel attenuation characteristics, and quantifying non-distance-related attenuation such as an additional attenuation of 30-50 dB caused by metal reflection.

[0040] Auxiliary parameters: environment temperature and humidity (affecting the speed of radio frequency signal propagation and the attenuation coefficient), goods density (dense areas are prone to tag collision), and the like, which can be expanded to further enrich the environment sensing dimension.

[0041] For example, the distance between the measuring device and the goods on the third layer of the shelf is measured by a laser radar, such as 2.5 meters, which is the "distance parameter"; at the same time, it is known from the pre-stored three-dimensional map of the warehouse that the shelf is of metal material (which will strongly reflect the signal), which is the warehouse environment structure information. Since the traditional system does not know the distance and the shelf material, when the power is fixed, the signal is easily weak to read the tag in the far distance or metal environment, and the power is excessive in the near distance.

[0042] (2) The multi-antenna radio frequency identification unit is equivalent to giving the reading device a "signal stethoscope", the unit transmits a radio frequency signal (such as UHF frequency band 860-960 MHz) and receives a tag response signal, completes identity recognition and extracts signal characteristic parameters representing signal propagation characteristics, i.e. analyzes the state of signal propagation, such as signal strength, whether there is interference, for supporting interference evaluation and parameter optimization;

[0043] Optionally, the multi-antenna radio frequency identification unit adopts a 4-8 channel MIMO antenna array (supporting beamforming and spatial diversity), integrates a radio frequency front end (including a power amplifier and a low noise amplifier) and a signal processor.

[0044] For example, the signal feature parameters include:

[0045] RSSI (Received Signal Strength Indicator): Real-time monitoring of received signal power to assess link quality. For example, if RSSI is lower than -85 dBm, there may be a risk of missed reading.

[0046] Phase fluctuation metric By analyzing the phase difference of the same tag signal received by different antennas, the strength of multipath effect is quantified. The larger the phase difference, the more serious the multipath interference. For ease of understanding, it can be assumed that speaking in an empty room, the voice is clear. But speaking in a complex room with many glass walls, many echoes can be heard, causing the voice to be chaotic, and the same is true for RFID signals. When the signal is reflected multiple times by objects such as metal shelves, the paths to different antennas are not the same, and the phase becomes chaotic. By calculating the fluctuation degree of such phase difference, the severity of "echo" (i.e. multipath interference) can be quantified. This is a more profound and more reflective index of real channel quality than RSSI.

[0047] SNR (Signal-to-Noise Ratio): Reflects the ratio of signal to noise, providing a basis for frequency band switching.

[0048] Compared with the traditional scheme, the present scheme introduces a phase fluctuation metric, so that the system can distinguish between two cases: one is that the signal strength is indeed weak, such as too far away, and the other is that the signal "sounds weak" because the interference is too serious, and the coping strategies for these two cases are completely different.

[0049] 2. Processing control core, integrating embedded processor and algorithm module, such as FPGA+ARM architecture, to ensure real-time performance, and to realize the "perception-decision-execution" closed loop. The specific process and detailed description are as follows:

[0050] (1) Construct a dynamic state vector

[0051] Fusion logic: Map environmental perception parameters and signal feature parameters into a high-dimensional dynamic state vector , comprehensively describe the physical constraints and signal conditions of the current inventory scene, that is, the "observation" results of the physical world (distance, shelf material) and the "auscultation" results of the signal world (RSSI, phase fluctuation metric) are creatively "fused" together to form a comprehensive, multi-dimensional real-time state snapshot of warehouse inventory.

[0052] Composition of dynamic state vector :

[0053] Among them: is the distance parameter; is the phase fluctuation metric; Channel attenuation characteristics (calculated based on environmental structure information, such as metal shelf area , open area ); Product density (number of tags per unit volume, reflecting collision risk).

[0054] For example, state A: {distance = 5 meters, environment = open, phase fluctuation = small, RSSI = weak};

[0055] State B: {distance = 2 meters, environment = metal shelf, phase fluctuation = large, RSSI = weak};

[0056] For traditional technology, states A and B can be identified as "weak signal" at the same time, but the present scheme can clearly distinguish them. It is further understood that state A is simply far away, while state B is strong interference at close range. This precise state definition is the cornerstone of subsequent correct decision-making. The dynamic state vector provides accurate "environmental input" for the Q-learning decision-making model, ensuring that the decision adapts to the current scenario.

[0057] (2) Run the Q-learning decision-making model

[0058] The model receives the dynamic state vector as input and outputs the optimal action instruction based on the preset optimization goal. The principle of this model can be understood as follows: based on the reinforcement learning framework, the goal is to maximize the number of unique tag identifications per unit time, and the action selection strategy is optimized through trial and error learning.

[0059] For example, the core elements of this model include:

[0060] State space: the dynamic state vector described above ;

[0061] Action space: includes transmit power adjustment (such as 5-30 dBm continuous adjustment), antenna beam pointing (such as azimuth -60°~+60°, elevation -30°~+30°), RF frequency switching (such as 860-870 MHz / 920-930 MHz anti-jamming frequency bands);

[0062] Reward function: used to define the preset optimization goal, and the action is continuously learned and optimized, such as fine-tuning the power according to the interference intensity, and then adapting to the changing warehouse environment.

[0063] The parameter adjustment of traditional technology relies on fixed, hard-coded "If-Then" rules. Such rules are very rigid and cannot adapt to the changing real-world scenarios. In the Q-learning decision-making model in the present scheme, "trial and error" can be used to learn:

[0064] For example, in the above-mentioned "state B" (close distance strong interference), it may first try an action such as "increase transmission power"; after execution, it is found that although the power is large, the "echo" (phase fluctuation) is more serious, and as a result, the number of successfully read tags decreases, which is understood as a negative reward;

[0065] At this time, the model has learned that in "state B", "increasing power" is a negative decision; the next time it encounters "state B", it will try another action such as "starting beamforming, focusing the signal into a straight beam, and switching to a cleaner frequency band"; after execution, it is found that the inventory efficiency is greatly improved, that is, a positive reward is obtained;

[0066] Over time, the model accumulates a large amount of "experience" and knows that in what "state", the combination of executing what "action" (power, beam, frequency point) can obtain the highest return, that is, the highest inventory efficiency.

[0067] (3) Execute optimal action instruction and parameter adjustment

[0068] The model dynamically adjusts the working parameters of the multi-antenna radio frequency identification unit and executes tag inventory, and the working parameters at least include transmission power, antenna beam pointing, and working radio frequency frequency point.

[0069] During the execution of the optimal action instruction, feedback is obtained: the number of unique tags identified per unit time is statistically calculated in real time as a reward signal feedback to the Q-learning decision model, realizing online learning-decision optimization iteration.

[0070] For example, in dense areas, through learning adjustment to low power + narrow beam, the tag collision is reduced.

[0071] For example, the system can also add a UWB positioning module and an infrared reflective sensor to form a "multimodal perception fusion" mechanism. The above-mentioned system relies on pre-set warehouse environment structure information, such as metal shelves, but cannot identify the material of the goods itself in real time, such as metal packaging goods, and the distance measurement error of the laser radar is large under metal shielding. Therefore, through UWB and infrared sensors, dual real-time perception of environment + goods is realized, which makes up for the perception blind area of the original scheme, as follows:

[0072] The UWB technology measures distance through ultra-short pulse signals (nanosecond level) and has strong anti-multipath interference capability (especially in metal shielding scenes). When the distance measurement error of the laser radar or vision sensor is greater than 8cm due to metal shelf shielding (non-line-of-sight scene), the UWB module is automatically started, and outputs the distance parameter with an accuracy of ±3cm as the supplementary data of the environment perception unit.

[0073] UWB data can be combined with lidar data through a "weighted fusion algorithm" to correct the distance parameter in the environmental perception parameter, ensuring the accuracy of the "distance parameter d" in the dynamic state vector, for example, UWB weight 0.7, lidar 0.3 in a blocked scene.

[0074] And the infrared sensor is installed at the front end of the reading device, which distinguishes the material by detecting the infrared reflectivity of the surface of the goods, such as metal reflectivity > 80%, non-metal < 30%, and real-time output of material identification signal (metal / non-metal). This signal is used to dynamically correct the channel attenuation feature a - if it is identified as a metal product, the value of a is further increased from 0.8 (the default value of the metal shelf) to 0.9 (additional compensation of 10 dB attenuation), so that the channel attenuation feature in the dynamic state vector is more consistent with the real-time scene.

[0075] For example, the Q-learning algorithm in the above system relies on the Q-learning decision model to adjust the frame length, and does not design a special strategy for dense areas, such as the promotion product stacking area in an e-commerce warehouse, resulting in a collision rate > 20% in high-density scenarios. Therefore, if DFSA is combined with the Q-learning algorithm, intelligent anti-collision can be achieved through "density perception → time slot adaptation". For example, in dense areas, the dynamic frame time slot ALOHA (DFSA) algorithm is started, and the Q-learning decision model is combined to optimize frame time slot allocation (reduce tag response conflicts) and improve identification efficiency, which is as follows:

[0076] DFSA reduces collisions by dynamically adjusting the number of frame time slots (each time slot corresponds to a tag response window) - the higher the tag density, the more frame time slots, such as 10-50 time slots adjustable. For example, when the product density p = 5 / m³, 10 time slots are used; p = 50 / m³, automatically increased to 50 time slots.

[0077] And the Q-learning decision model includes the product density p in the dynamic state vector, and outputs the action instruction "start DFSA" when p > 10 / m³; the Q-learning algorithm optimizes the number of time slots through the reward function: if the collision rate > 15% (the number of failed identifications / the total number of tags) under the current number of time slots, the reward value is reduced to drive the model to learn the optimal strategy "high density → multiple time slots", such as p = 30 / m³, time slot number = 35.

[0078] The system solves the problems of low efficiency and weak anti-interference caused by static parameters in traditional RFID inventory through the deep integration of environmental perception, intelligent decision-making and dynamic adjustment. Compared with the prior art, the core advantages are: based on the Q learning decision model, the parameters are optimized in real time to adapt to dynamic scenes such as distance changes, multi-path interference and differences in product density; through power compensation, beam forming and anti-interference frequency switching, the tag recognition rate is improved and the redundant energy consumption is reduced; it should be noted that the system supports multi-modal perception, edge collaboration and other technology expansion, has continuous evolution ability, and can adapt to different scale warehouse scenes such as e-commerce warehouses and cold chain warehouses.

[0079] In some embodiments of the application, the optimization target preset above can be defined by a reward function designed to take the number of unique electronic tags successfully identified per unit time as positive encouragement to drive the Q learning decision model and output action instructions that can maximize inventory efficiency. The reward function of the Q learning decision model is realized through multi-objective weighted fusion, which ensures inventory efficiency and balances energy consumption and accuracy.

[0080] For example, the calculation formula of the reward function can be:

[0081] wherein, is the total reward value, which can be theoretically positive or negative, and is actually adjusted by parameters to make the effective decision positive; represents the efficiency weight (highest priority); represents the energy consumption weight (secondary priority); represents the accuracy weight (secondary priority); is the number of unique tags successfully identified per unit time (pieces); is the inventory time (seconds), i.e. represents the "number of unique tags identified per second" (core efficiency indicator); : current transmit power (dBm); dBm: maximum allowed power (to avoid high energy consumption); : miss read rate (range 0~1), i.e. "number of unidentified tags / total number of tags" (accuracy indicator).

[0082] Suppose, for example, that the "metal shelf 3rd floor, 2.5 meters away from the reading device, tag density 15 / m³", demonstrates how the reward function drives the model to learn the optimal strategy:

[0083] 1. Initial decision (without optimization)

[0084] Action: transmit power 25dBm (too high), no anti-interference frequency switching, time slot number 10 (too few);

[0085] Result: Efficiency:

[0086] dBm Energy consumption:

[0087] Missed reading rate Precision:

[0088] Total reward: This indicates that the reward is low, and the model needs to be optimized.

[0089] 2. Optimized decision-making (after model learning)

[0090] Action: Transmit power 20 dBm (reduce energy consumption), switch to 920 MHz anti-jamming frequency band, time slots 30 (reduce collision);

[0091] Results: seconds Efficiency: 180 / 10 = 18 per second;

[0092] dBm Energy consumption:

[0093] Missed reading rate Anti-jamming and anti-collision optimization Precision:

[0094] Total reward: This indicates that the reward has been significantly improved, and the model has learned the optimal strategy.

[0095] As Figure 4 shows the convergence curve of the average cumulative reward of the Q-learning decision-making model during the training process with the number of training iterations. In the Q-learning decision-making model, the number of unique electronic tags successfully identified per unit time is used as the core of the reward function. Therefore, Figure 4 the Y-axis average cumulative reward in corresponds directly to the execution efficiency of the inventory task. The higher the reward value, the better the performance of the system in the unit time. The X-axis training iteration number represents the learning process of the model.

[0096] Figure 4 In, the blue convergence curve clearly reveals the three typical learning stages of the Q-learning decision-making model from "ignorance" to "expertise":

[0097] ​​​​​Initial stage (about 0-200 iterations): At the beginning of training, the model's "experience" is blank, and it must explore the results of different actions (adjusting transmit power, antenna beam pointing, operating frequency, etc.) in different states (composed of environmental perception parameters and signal feature parameters) through a large number of random "trial and error". Therefore, as shown in Figure 4 , the average cumulative reward value in this stage is low and volatile, with a value of approximately 240. This indicates that the system's inventory efficiency is low and extremely unstable at this time, which fully meets the exploration characteristics of reinforcement learning algorithms in the initial stage.

[0098] Learning and utilization stage (about 200-800 iterations): As training progresses, the model gradually accumulates effective "experience" through continuous feedback of reward signals (positive rewards for successfully reading more labels), i.e., it builds a mapping relationship between states and optimal actions (Q-Table or Q-Network). It begins to understand that in what distance and interference environment should it take what parameter combination. As shown in Figure 4 , the average cumulative reward in this stage shows a rapid and significant upward trend, rising from a level of about 40 to above 90. This eloquently proves that the Q-learning model of the present scheme has strong learning ability and can convert inventory experience into a continuously optimized inventory strategy.

[0099] Stable stage (after about 800 iterations): When training progresses to the later stage, the model has learned a relatively optimal inventory strategy. As shown in Figure 4 , the reward curve at this time tends to be flat and stable at a high level of about 95, with no significant growth. This indicates that the model has "converged", meaning it has found a general strategy that achieves the highest inventory efficiency in most warehouse scenarios. Subsequent training is only a fine-tuning and consolidation of this strategy.

[0100] Through autonomous learning of the Q-learning decision model, the system can autonomously and intelligently learn a complex control strategy that maximizes inventory efficiency through interaction with the environment and reward feedback, thereby adapting to the dynamically changing warehouse environment.

[0101] It should be noted that if the inventory process has a high collision rate (>15%) in the dense area, it may be due to signal conflict caused by simultaneous tag response, especially when ρ>10 / m³. At this time, the following strategies can be used:

[0102] 1. Introduce "collision penalty" in the reward function: If the collision rate is >15%, reduce the weight to 0.4 to force the model to increase the time slot (e.g., from 10 to 50);

[0103] 2. The time slots are positively correlated with the density p by combining the DFSA algorithm.

[0104] In some embodiments of the present application, the signal feature parameters include a phase fluctuation metric The phase fluctuation metric refers to calculating the standard deviation of the phase difference of the same tag signal received by different antennas in the multi-antenna radio frequency identification unit after analyzing the phase difference, quantifying the interference degree of the multipath effect caused by environmental reflection and diffraction.

[0105] The effectiveness of the technical idea of quantifying the strength of the multipath effect by analyzing the phase relationship of the signals received by the multi-antenna unit is verified through simulation experiments, and the results are shown in Figure 2 and Figure 3 .

[0106] Figure 2 The above-mentioned reading device of the present scheme is simulated in an open area in a warehouse environment, and the visual distance propagation scene is not blocked by a metal shelf. In this scenario, the response signal emitted by the electronic tag attached to the goods can directly and stably reach the multi-antenna radio frequency identification unit of the reading device. The X-axis in the figure represents the continuous sampling points generated by time or the movement of the reading device, and the Y-axis represents the phase difference between the multi-antenna unit receiving the same tag signal, with the unit being radians (rad).

[0107] As shown by the blue curve in Figure 2 , the phase difference between the antennas changes smoothly overall, and the value fluctuates slightly within a very narrow interval (for example, about -0.5 to 0.5 radians). This smooth curve characteristic accurately reflects the physical fact that the signal propagation path is single and the channel state is stable, indicating that the multipath interference is extremely weak at this time.

[0108] This provides a solid foundation for the present scheme: when the "phase fluctuation metric" calculated internally by the system presents a low value as shown in Figure 2 , the processing control core can accurately determine that the current reading environment is good, thereby providing a clear state input for the Q-learning decision model, i.e., a "low interference state".

[0109] And Figure 3 a completely different and more challenging working condition in warehouse inventory is simulated, i.e., the reading device moves between metal shelves filled with goods, or in a complex distance propagation scene such as a corner. In this scenario, the tag signal will be reflected and diffracted multiple times by objects such as shelves and walls, and finally superimposed on the receiving antenna as multiple component signals with different paths and phases.

[0110] As shown in Figure 3The middle red curve shows that the phase difference presents a sharp, irregular broadband jitter. Its numerical fluctuation range is extremely large, and in a very short sampling time, a huge jump from -2.5 radians to +2.5 radians occurs. This sharp fluctuation is a typical characteristic caused by the random, time-varying constructive or destructive interference of multiple path signal components at the antenna. It intuitively and quantitatively reveals the existence of strong multipath effect in the current channel.

[0111] This provides key technical support for the present scheme: when the "phase fluctuation metric" calculated by the system presents a high value as shown in Figure 3 , the processing control core can accurately and quantitatively identify the current poor reading environment. This cannot be achieved by the traditional method of relying only on the received signal strength indication (RSSI), because the decrease of RSSI can be caused by both distance and multipath interference, and there is ambiguity.

[0112] Based on this accurate and non-ambiguous state judgment, the Q decision model of the present scheme can avoid making wrong decisions, such as blindly increasing the transmission power under strong interference, which will only exacerbate the confusion of the signal, and instead will preferentially output action instructions that can activate advanced interference suppression strategies (such as switching to a cleaner anti-interference frequency band or starting multi-antenna beamforming), thereby achieving real and evidence-based intelligent inventory.

[0113] For example, with 4 antennas simultaneously receiving the tag signal, the phase difference between adjacent antennas is collected 10 times in succession, such as the phase difference between antenna 1 and antenna 2 , the phase difference between antenna 2 and antenna 3 , etc. The standard deviation of these 10 groups of phase differences is calculated, which is the phase fluctuation metric . This value reflects the "degree of confusion" of the signal after reflection / diffraction by the environment. Analogous to "echo": in an open warehouse (few reflections), usually <5° (signal clear); in a dense area of metal shelves (multiple reflections), may be >15° (signal confusion, similar to multiple echo superposition).

[0114] In traditional technology, only relying on signal strength RSSI and fixed position tags cannot distinguish between "distance far leading to weak signal" (need to increase power) and "strong interference leading to weak signal" (need to resist interference). The present scheme, through the phase fluctuation metric and the channel attenuation characteristic α, enables the model to accurately judge the scene type, for example:

[0115] If the phase fluctuation metric is small, the channel attenuation characteristic α is small, and the distance parameter d is large → "pure distance far" → the Q learning decision model outputs "increase power";

[0116] Phase fluctuation metric Large channel attenuation feature a, large distance parameter d → "strong metal interference" → Q-learning decision model outputs "switch to anti-interference frequency band + beam focusing".

[0117] It should also be noted that if the phase fluctuation metric The calculation is disturbed by transient noise, such as sudden electromagnetic interference, at which time "sliding window filtering" can be used: the phase fluctuation metric The phase difference average in 100 ms (instead of single sampling) is taken to smooth the transient noise; the signal-to-noise ratio (SNR) can also be used to assist in the judgment: if SNR > 15 dB (low noise), even if the phase fluctuation metric Slightly exceeds the threshold value, the anti-interference instruction is also delayed to avoid misoperation.

[0118] In some embodiments of the present application, the dynamic state vector includes a channel attenuation feature a, which is based on environmental perception parameters (such as distance) and pre-stored warehouse environment structure information (such as shelf material, obstacles), and quantifies non-distance-related signal attenuation (such as additional attenuation caused by metal reflection and product shielding).

[0119] For example, the a of a metal shelf is 0.8 (corresponding to an additional attenuation of 30-50 dB), the a of a wooden shelf is 0.2 (additional attenuation < 10 dB), and the a of an open channel is 0.1 (almost no additional attenuation).

[0120] The channel attenuation feature a can complement the shortcomings of the traditional "free space loss model". The free space loss is only related to the distance (increases with the increase of the distance), while a specifically depicts the attenuation caused by the "environmental structure" (which is independent of the distance and only related to the material / obstruction). This numerical correction adjusts the transmission power, ensuring more accurate power adjustment in metal / obstruction environments. For example: at the same distance, the compensation power of the metal shelf area is 10-20 dB higher than that of the open area, avoiding missed reading.

[0121] In some embodiments of the present application, the environmental perception parameters include a distance parameter, which is used to perform dual calculation tasks:

[0122] (a) First, calculate the "distance-related basic loss" through the free space model, which reflects the natural attenuation of the signal with distance;

[0123] (b) The signal propagation scenario attribute is judged in combination with the warehouse environment structure information, including distinguishing whether the current signal propagation belongs to a line-of-sight propagation scenario or a non-line-of-sight propagation scenario, and in the non-line-of-sight propagation scenario, a dynamic attenuation factor is used to correct the basic path loss to compensate for the additional attenuation caused by the shelter (such as metal shelves). This mechanism solves the limitations of traditional technologies that only rely on fixed power or simple distance compensation, and makes the power adjustment adapt to the distance and environment structure at the same time, providing a more accurate physical loss basis for the Q-learning decision model.

[0124] For example, the path loss in free space describes the natural attenuation of electromagnetic waves with distance and frequency in unobstructed space, and its formula can be: (unit: dB)

[0125] wherein, is the frequency of the radio frequency signal (unit: MHz); is the wavelength of the radio frequency signal (unit: m); , m / s is the speed of light; and the constant 32.44 is derived from .

[0126] For example, the formula for introducing a dynamic attenuation factor to correct the basic path loss can be:

[0127] wherein, is the total path loss after correction (unit: dB); : dynamic attenuation factor (value according to shelter type, metal shelf , wooden shelf , no shelter ); : basic additional attenuation caused by shelter (metal shelf dB, wooden shelf dB, obtained through offline testing).

[0128] Suppose the distance between the reading device and the third layer of goods of the metal shelf is d=3m, the working frequency is f=920MHz, and the shelter type is metal shelf (β=0.8, =40dB).

[0129] Then the basic path loss is calculated as: =20lg(3)+20lg(920)+32.44≈9.54+59.28+32.44=101.26dB;

[0130] Correct the non-line-of-sight additional attenuation: =101.26+0.8×40=101.26+32=133.26dB;

[0131] Comparison with line-of-sight scenario: if it is an open channel (no occlusion, β = 0), then = = 101.26 dB. Thus, the metal occlusion increases the total loss by 32 dB, which needs to be compensated by power adjustment.

[0132] It should be noted that if the laser radar deviates by more than 8 cm in the metal reflection, the distance measurement error at this time leads to Calculation error, at this time, on the one hand, the UWB positioning data can be fused, and the UWB weight is increased to 0.7 in the occlusion scene; on the other hand, the distance d can also be filtered by a sliding window (100 ms window), and the instantaneous error is smoothed.

[0133] In some embodiments of the application, when the processing control core executes the optimal action instruction, the transmission power is corrected through a hierarchical compensation architecture, which is the core mechanism for the processing control core to accurately adjust the transmission power, aiming to balance environmental randomness and physical determinism through hierarchical processing, and solve the problem that a single parameter in traditional power adjustment is difficult to adapt to complex scenes. The specific definition and role of the three-layer architecture are as follows:

[0134] 1. First layer: reference power instruction (to cope with environmental randomness)

[0135] The reference power output by the Q-learning decision model is used to cope with signal fluctuations caused by random factors such as multipath interference and instantaneous noise. The Q-learning model is based on a dynamic state vector, including phase fluctuation metrics , signal-to-noise ratio SNR, etc., and outputs a power reference that adapts to the current random interference through trial and error learning. For example, when the multipath interference is strong > 15°), the model will appropriately increase the reference power to offset the attenuation caused by "chaotic" signals; when the noise is low (SNR > 20 dB), the reference power will be reduced to reduce energy consumption.

[0136] 2. Second layer: deterministic compensation amount (to cope with physical law attenuation)

[0137] Based on the corrected basic path loss in non-line-of-sight scenarios , the compensation amount for the "distance + occlusion" deterministic attenuation is calculated , and the compensation amount is composed of two parts:

[0138] Distance-related compensation: based on the free space loss formula, for every 1-fold increase in distance, the compensation amount increases by 6 dB, e.g. 3m distance needs to be compensated by 6dB more than 1.5m;

[0139] Occlusion compensation: based on dynamic attenuation factor correction formula, the compensation amount of metal shelves (β = 0.8, = 40 dB) is 20 dB more than that of wooden shelves (β = 0.3). Finally is the minimum received power at which the tag can be identified, usually set to -85 dBm.

[0140] 3. Superimpose the compensation amount of the second layer on the reference power of the first layer to output the final transmission power .Both retain the adaptive ability of the Q-learning decision model to random environment and ensure that the power meets the loss of distance + occlusion through physical compensation, achieving precise power output that is both anti-interference and energy-saving.

[0141] In the above hierarchical architecture, the first layer interfaces the Q-learning decision model (intelligent decision layer), ensuring rapid response to environmental randomness; the second layer relies on environmental perception parameters (distance, warehouse structure information) and signal feature parameters (path loss) (perception layer) to achieve precise modeling of physical laws; the combination of the two makes power regulation both intelligent and flexible and in line with physical laws, solving the problem of traditional static power that is either under-compensated (missed reading) or over-compensated (high energy consumption).

[0142] It should also be noted that if the Q-learning decision model is solely relied upon to directly output the final power, the model needs to learn both environmental randomness and physical certainty, which will increase the state space dimension by more than 30%, resulting in slow convergence speed and low decision accuracy. The hierarchical architecture separates the physical certainty task through the second layer, allowing the Q-learning decision model to focus only on randomness adaptation, greatly improving model training efficiency and reducing decision delay, which meets the real-time requirements of warehouse inventory (for example, when the AGV moves at a speed of 0.5 m / s, parameter adjustment needs to be completed within 100 ms).

[0143] In some embodiments of the present application, when the phase fluctuation metric exceeds the preset interference threshold, the Q-learning decision model preferentially outputs the action instruction of the interference suppression strategy, which includes:

[0144] Switching to an anti-interference frequency band or starting multi-antenna beamforming, and verifying whether the phase fluctuation metric is reduced to within the threshold after frequency band switching.

[0145] Optionally, the preset interference threshold refers to a phase fluctuation metric threshold value (usually set to 15°), which indicates that multipath interference has significantly affected inventory accuracy when it is exceeded. Threshold determination basis: through offline testing, when the AGV moves at a speed of 0.5 m / s in a typical warehouse environment (metal shelves, product stacking area), the phase fluctuation metric is usually less than 15°, and the inventory accuracy is usually above 95%.​​​​​ When the phase fluctuation exceeds 15°, the tag recognition rate will drop from 95% to below 70%, so it is used as the critical condition to trigger the interference suppression strategy.

[0146] Multi-antenna beamforming refers to the use of the phase control capability of a multi-antenna array (such as a 4-8 channel MIMO) to focus the radio frequency signal into a directional narrow beam (beam width can be as low as ±5°), reducing signal radiation to the reflection path, thereby suppressing multipath interference. The specific operation is to transmit strong signals only to the target product area, avoiding signal diffusion to the reflector (such as metal shelves) to cause interference.

[0147] The closed-loop execution process of the above interference suppression strategy is as follows:

[0148] 1. Interference detection: Phase fluctuation exceeds threshold value Identify the phase difference of adjacent antennas by the multi-antenna radio frequency identification unit in real time (updated every 10 ms), calculate the standard deviation in the 100 ms sliding window When When the phase fluctuation exceeds 15° for 3 consecutive periods (300 ms), the processing control core determines that it is in a "strong interference state" and triggers the priority decision logic of the Q-learning decision model.

[0149] 2. Intelligent decision making: The Q-learning model outputs the optimal strategy, and the phase fluctuation metric is included in the dynamic state vector , and the strategy is output based on the preset optimization goal (maximize the number of unique tag recognitions per unit time):

[0150] If the current frequency band SNR < 10 dB (noise dominant interference), preferentially output the "switch to 920-930 MHz anti-interference frequency band" instruction;

[0151] If SNR ≥ 10 dB (multipath dominant interference), preferentially output the "start beamforming (pointing to the target product azimuth within ±3°)" instruction; (the decision priority is based on offline training data: the interference suppression effect of beamforming in a multipath scenario is 15-20% higher than that of frequency band switching).

[0152] 3. Strategy execution: Dynamic adjustment of hardware parameters Frequency band switching: The multi-antenna radio frequency identification unit completes the switching from the current frequency band (such as 860 MHz) to the anti-interference frequency band (such as 920 MHz) within 50 ms, synchronously updates the local oscillator frequency and filter parameters of the radio frequency front end, and ensures the stability of signal reception; Beamforming: Adjust the signal phase of each antenna in real time (response time < 30 ms) through FPGA control of the phase shifter, focus the beam to the target product location recognized by the environmental perception unit, and suppress the radiation of sidelobe signals to reflectors.

[0153] 4. Effect verification: After the closed-loop feedback mechanism strategy is executed, continuously monitor Change: If within 300ms If the temperature drops below 15°, the interference suppression is deemed effective, and the current strategy is maintained. If the target is not met, the Q-learning model triggers a "joint strategy" (such as anti-interference frequency band + beamforming) and uses the results of this execution (recognition rate, time consumption) as a negative reward to optimize subsequent decisions.

[0154] To verify the effectiveness and intelligence of the interference suppression strategy's closed-loop mechanism—that is, after the system executes, for example, "adaptive frequency hopping," it verifies its effect (whether the phase fluctuation drops to a preset threshold, such as 15°) and decides whether to activate a more advanced "joint strategy" based on the verification results—simulation experiments were conducted, and the results are attached. Figure 7 As shown.

[0155] Figure 7 The heatmap clearly illustrates the complete closed loop of the system decision-making logic in this solution, where:

[0156] The Y-axis represents the interference suppression strategy executed by the system, indicating the different actions that the Q-learning decision model of this scheme can output after identifying a high-interference state; the X-axis represents the initial phase fluctuation metric, indicating the intensity of environmental interference perceived by the system through phase analysis before taking any action. Figure 7 The color and value of each cell represent the final residual phase fluctuation measure after implementing a specific strategy (Y-axis) at a certain initial interference intensity (X-axis). The cooler the color (blue) and the smaller the value, the better the suppression; the warmer the color (yellow) and the larger the value, the worse the suppression effect.

[0157] refer to Figure 7 This strategy can be clearly understood:

[0158] 1. The baseline for "no intervention" (first row): This row shows that if no measures are taken, the final interference level is exactly the same as the initial level (values ​​from 20° to 100°), and the color is a warm hue representing "suppression failure". This proves the absolute necessity of taking interference suppression measures.

[0159] 2. Limitations of a single strategy (lines 2 and 3):

[0160] When the initial interference is weak (e.g., initial fluctuation of 20° or 40°), both "adaptive frequency hopping" and "multi-antenna beamforming" strategies can achieve good results. Figure 7 As shown, the suppressed phase fluctuations decreased to 12°, 14°, 13°, and 16°, respectively. Among them, 12°, 13°, and 14° are all lower than the preset threshold of 15°, and the cells are displayed in a cool color (blue). This indicates that the strategy was successfully executed and the closed-loop verification was passed.

[0161] However, when the initial interference increases to 60° or higher, the effectiveness of a single strategy decreases significantly. For example, with an initial interference of 60°, the fluctuations after frequency hopping and beamforming are still as high as 25° and 22°, respectively, far exceeding the 15° threshold, and the cells exhibit a warm hue (yellow-green). This intuitively demonstrates that there is an upper limit to the capabilities of a single strategy.

[0162] 3. The sexual advantages of the "joint strategy" and its manifestation in closed-loop verification (fourth line):

[0163] When the system executes a single strategy (such as frequency hopping), and the internal closed-loop verification mechanism finds that the phase fluctuation has not been reduced to within 15°, the Q-learning model will output a more advanced "frequency hopping and beamforming joint strategy" instruction based on this "failure" feedback.

[0164] like Figure 7 As shown in the fourth line, regardless of whether the initial disturbance is a moderate 60° or an extremely severe 100°, the joint strategy can always force the final phase fluctuation to below the 15° threshold (the results are 10°, 12°, and 14° respectively), and the entire line is presented in dark blue to represent "success".

[0165] Traditional RFID technology relies on fixed frequency bands and omnidirectional antennas, resulting in a missed read rate of over 30% under multipath interference. This solution, however, utilizes... As an interference-sensitive indicator, it detects interference earlier than traditional RSSI; it dynamically switches anti-interference frequency bands or focuses the beam to specifically counteract multipath effects, thereby improving the recognition rate in metallic environments and directly supporting the core objective of this solution: "improving anti-interference capability and inventory accuracy." Furthermore, the interference suppression strategy and the hierarchical power compensation architecture complement each other: when... When the threshold is exceeded, simply increasing the power will exacerbate multipath interference (stronger reflected signals). This solution reduces interference at the source by using anti-interference frequency bands and beamforming, and then combines power compensation (second layer of the hierarchical architecture) to offset deterministic attenuation, achieving dual optimization of anti-interference and signal coverage.

[0166] It is important to note that when switching frequency bands, the RF front-end needs to be recalibrated, which may cause a 50-100ms identification interruption. In this case, a pre-scan of anti-interference frequency bands can be performed: before inventory, pre-scan each frequency band through edge nodes. The system establishes a "frequency band-interference level" mapping table, and directly calls the optimal frequency band during switching, compressing the interruption time to within 30ms.

[0167] To objectively and quantitatively demonstrate that this solution represents a significant improvement and has beneficial effects compared to existing technologies in solving practical inventory problems, a comparative experiment was conducted. The performance of this solution was compared with that of two representative existing technologies, and the results are as follows: Figure 5 and Figure 6 As shown.

[0168] Among them, the gray column chart represents the "fixed power method": a most basic prior art, i.e., the reader always works at a fixed transmission power without any environmental adaptability adjustment; the blue column chart represents the "RSSI-based adaptive method": a more improved prior art, which can make a simple linear adjustment of the transmission power according to the feedback of the received signal strength indication (RSSI); and the red column chart represents the present scheme.

[0169] As shown in Figure 5 , the figure shows the final inventory accuracy of the three methods in a comprehensive warehouse scene containing open area, ordinary shelf area, high-density tag area, and other situations.

[0170] As shown in Figure 5 , the accuracy of the fixed power method is the lowest, only 85.3, because it cannot cope with environmental changes. The RSSI-based adaptive method performs slightly better, reaching 92.1%, but its performance improvement is limited.

[0171] And the red column chart representing the present scheme has an inventory accuracy of 99.5%, significantly better than the other two methods. The fundamental reason is that the present scheme not only relies on the single and ambiguous indicator of RSSI, but also innovatively integrates environmental perception parameters (such as distance) and signal feature parameters (such as phase fluctuation measure) to construct a comprehensive dynamic state vector, as defined above. Combined with the Q-learning decision model shown in Figure 4 , which has strong self-learning ability, the system can accurately and dynamically optimize the multi-dimensional parameter combination including transmission power, antenna beam pointing, and working frequency point, so that it can find the optimal reading strategy in various complex scenarios, and finally achieve a qualitative leap in inventory accuracy.

[0172] As shown in Figure 6 , the figure focuses on the high multipath interference scene that is the biggest challenge to RFID technology, such as the metal shelf dense warehouse area, and compares the tag miss-read rates of the three methods. The lower the miss-read rate, the better the performance.

[0173] In this harsh environment, the performance of the fixed power method and the RSSI-based adaptive method deteriorates sharply, with miss-read rates of 25.7% and 5.2% respectively. This is because the two methods cannot accurately identify and cope with strong signal interference caused by metal reflection.

[0174] In sharp contrast, the miss-read rate of the present scheme is successfully suppressed to only 1.5%, showing strong environmental robustness. This is due to the following two points:

[0175] 1. The system can accurately determine that it is currently in a high-interference environment through the key indicator of phase fluctuation measure.

[0176] 2、In the case of high interference, the Q-learning model can intelligently call advanced interference suppression strategies, such as adaptive frequency hopping or multi-antenna beamforming technology, to fundamentally avoid or counter multi-path interference.

[0177] As Figure 8 shown, a RFID warehouse inventory method combining Q-learning algorithm and dynamic power adjustment is also proposed, which realizes dynamic adaptive inventory in warehouse scenarios through a closed-loop design of perception-modeling-decision-execution-feedback. The method specifically includes the following steps:

[0178] S1, environment and signal perception - multi-dimensional dynamic scene capture, by breaking through the limitations of traditional single signal perception, through the multi-dimensional fusion of "physical environment + signal characteristics", accurate input is provided for subsequent decision-making.

[0179] Technical implementation: 1. Environment perception parameter acquisition

[0180] The reading device (which can be mounted on an AGV or a mechanical arm) acquires distance information of the goods to be inventoried through laser radar (measurement accuracy of about ±5 cm) and UWB positioning technology (activated in complex scenes such as metal shielding, accuracy of about ±3 cm); At the same time, combined with the pre-stored warehouse three-dimensional map (including shelf material, stacking height, channel width, etc.), the type of shielding object (such as metal shelves will be marked as high attenuation characteristics) is identified. In the auxiliary perception aspect, infrared sensors are used to identify the material of the goods (metal or non-metal), further correcting the attenuation effect of the environment on the signal; temperature and humidity sensors (error ±2℃ / ±5%RH) are used to compensate for the slight changes in signal propagation speed; visual algorithms are used to count the density of goods (pieces per cubic meter) to judge the risk level of tag collision.

[0181] 2. Signal feature parameter extraction

[0182] Multi-antenna RFID unit (using a 4-channel multi-antenna array) real-time acquisition of three key signal features:

[0183] Received signal strength, filtered through a 100 ms sliding window to reduce the impact of instantaneous fluctuations, directly reflecting the signal link quality;

[0184] Phase fluctuation degree, by calculating the phase difference change of the same tag signal received by different antennas, the larger the value, the more serious the multi-path interference, such as significant fluctuation in a metal environment;

[0185] Signal-to-noise ratio, calculated by the ratio of signal power to noise power, used to judge the interference level of the current frequency band, when the ratio is too low, the anti-interference frequency band needs to be switched.

[0186] S2, State space construction - High-dimensional dynamic vector modeling, by fusing different types of perception data into a unified state vector, fully describing all physical conditions that affect radio signal propagation.

[0187] Technical implementation: 1, Vector composition and standardization

[0188] The dynamic state vector contains six key dimensions: distance (standardized value), phase fluctuation metric (standardized value), received signal strength (standardized value), signal-to-noise ratio (standardized value), channel attenuation characteristics (such as high value corresponding to metal shelves), and product density (standardized value). Each dimension is standardized to the range of 0-1 or -1-1 to ensure consistent input scale for subsequent intelligent decision-making models.

[0189] For example, the vector of a metal shelf area with a distance of 3 meters and a high interference scene can be represented as [0.6, 0.8, -0.7, 0.3, 0.8, 0.5], accurately distinguishing different scenarios such as "long distance" and "strong interference".

[0190] 2, Dynamic update mechanism

[0191] The vector is updated every 100 milliseconds (matching the real-time requirements of AGV moving speed 0.5 meters / second), capturing sudden changes in the scene (such as temporary stacking of goods leading to a sudden increase in density) through a sliding time window.

[0192] S3, Intelligent decision-making - Optimization output of Q-learning decision-making model, based on dynamic state vector, outputting optimal action instructions that maximize inventory efficiency.

[0193] Technical implementation: 1, Q-learning model training

[0194] Offline pre-training: Simulate millions of different scenarios (distance 0.5-5 meters, obstruction level 1-3 levels, etc.) through digital twin systems to generate "state-action-effect" samples and pre-train the decision model's experience base;

[0195] Online fine-tuning: Real-time collection of actual inventory data (experience base updated every 1 second), using a "exploration-exploitation" balance strategy, specifically: try new actions more in the early stage, use optimal actions more in the later stage, and thus quickly adapt to edge scenarios such as sudden obstructions.

[0196] 2, Action space and decision logic

[0197] Adjustable actions include: transmit power, antenna beam pointing, working radio frequency band (860MHz / 920MHz / 840MHz), and number of tag response time slots (10-50, to cope with dense scene collisions). Decision priority follows:

[0198] Strong interference priority: When the phase fluctuation exceeds 15°, prefer to switch to the anti-interference frequency band (such as 920MHz) or start beam focusing;

[0199] Distance compensation priority: When the distance exceeds 2 meters and the signal strength is weak, prefer to offset the distance attenuation through power compensation;

[0200] Collision suppression priority: When the product density exceeds 10 per cubic meter, increase the number of time slots to reduce label signal conflicts.

[0201] S4, parameter adaptive adjustment and execution - closed loop feedback execution, convert decision instructions into hardware parameter adjustment, execute inventory and feedback results to optimize the model.

[0202] Technical implementation: 1. Multi-parameter collaborative adjustment:

[0203] Transmit power adjustment: adopt hierarchical compensation mechanism:

[0204] First layer: execute the reference power output by the Q-learning model to deal with random interference;

[0205] Second layer: calculate the compensation amount of fixed attenuation according to distance and shielding type, such as additional compensation of 30-50 decibels for metal shelves;

[0206] Third layer: final power = reference power + compensation amount, such as total power in metal environment is 32 decibels higher than that in open area.

[0207] Beam forming: control the antenna phase through the chip to focus the signal into a narrow beam, reduce the signal radiation to the reflector, and suppress multipath interference.

[0208] Frequency band switching: switch to anti-interference frequency band when the signal-to-noise ratio is less than 10 decibels, and verify whether the phase fluctuation is reduced to below 15° after switching.

[0209] Time slot adjustment: increase the number of time slots when the goods are dense, such as from 10 to 30, to reduce label signal conflicts.

[0210] 2. Closed loop feedback and iteration

[0211] Every 1 second, three key indicators are counted: the number of unique labels identified per unit time (efficiency), the average transmit power (energy consumption), and the miss reading rate (precision). These indicators are used as feedback signals to update the experience base of the Q-learning model, so that the decision-making ability continues to evolve with task execution, such as gradually optimizing the strategy in dense areas from high power to low power + multiple time slots.

[0212] The method breaks through the limitation of traditional single signal perception through multi-modal perception deep fusion, provides more accurate scene input for decision-making through multi-dimensional description of physical environment + signal characteristics, and combines reinforcement learning and DFSA and other physical / protocol models through Q learning algorithm and physical model cooperation, guarantees the intelligence of decision-making, and improves the physical rationality of parameter adjustment, and supports UWB, edge computing, fault redundancy and other technologies expansion, and has adaptation ability for complex warehouse scenes (cold chain, e-commerce sorting center).

[0213] On the basis of the above method, the application also proposes an optimization process of model iteration in an RFID warehouse inventory method combining Q learning algorithm and dynamic power regulation, which breaks through the limitation of traditional fixed model, and dynamically evolves the Q learning decision-making model through real-time feedback according to the inventory task, and continuously adapts to complex warehouse scenes, such as metal-intensive areas and e-commerce warehouses with frequent changes of goods, and specifically comprises:

[0214] 1. Reward signal generation: feedback quantification with efficiency as the core

[0215] After each inventory operation, such as completing the scanning of a shelf area, the system automatically counts the number of unique tags successfully identified per unit time (such as 180 tags identified in 10 seconds, i.e. 18 per second), which is taken as the core reward signal. This signal directly reflects the effectiveness of the current decision-making strategy - the higher the value, the more adaptive the power, beam, frequency band and other parameters output by the model to the current scene.

[0216] At the same time, "energy consumption" (average transmission power) and "accuracy" (missed reading rate) are also included as auxiliary indicators, and a comprehensive reward is calculated by weighting (such as efficiency weight 60%, energy consumption 20%, accuracy 20%), to avoid the model from excessively consuming power in pursuit of efficiency (such as excessive power consumption in metal areas leading to a sharp increase in energy consumption).

[0217] 2. Online learning: real-time correction of decision bias

[0218] The reward signal is fed back to the Q learning decision-making model every 1 minute:

[0219] If the reward is higher than the historical average (such as 18 per second > historical 15 per second), the model will strengthen the Q value of the "state-action" combination, i.e. "remember" the optimal parameters in this scene, such as using 920MHz + beam focusing in metal areas.

[0220] If the reward is lower than the average (such as 8 per second), the model will reduce the corresponding Q value, and try to fine-tune the action (such as switching to 840MHz low-power frequency + increasing the number of time slots), to explore a better strategy through trial and error.

[0221] For example, in the scenario of adding cold-chain storage (low temperature and high humidity lead to signal attenuation), the initial model may have decision bias due to not being exposed to this environment, with a 20% miss-read rate. However, through online learning, the power compensation can be adjusted within 1 hour (5 dB more than in normal temperature environment), reducing the miss-read rate to 5%.

[0222] 3. Offline retraining: precipitating cross-scenario experience

[0223] After daily inventory is completed, the system aggregates the "state-action-reward" data of the whole day to the edge server for offline retraining:

[0224] Using new scenario data, such as cold-chain area and densely stacked promotional products area, to expand the training set to avoid model "forgetting" rare scenarios;

[0225] Optimizing the learning rate of Q-learning (from 0.1 to 0.05 dynamically), so that the model reduces fluctuations in stable scenarios and maintains the desire to explore in new scenarios.

[0226] The above S3 step relies on the pre-trained model output initial strategy, and the model iteration corrects the strategy through real-time feedback, solving the problem that pre-training cannot cover all scenarios, such as high interference caused by temporary stacking of goods. For example, after parameter adjustment in step S4, the miss-read rate rises sharply, and the model iteration quickly learns to increase the beam focusing intensity when new metal shielding is added, making subsequent decision-making more accurate.

[0227] On the basis of the above method, the present application also proposes a RFID warehouse inventory method of fusing Q-learning algorithm and dynamic power adjustment, which aims to link the inventory data and warehouse management system, replace traditional manual verification, and realize unmanned update of inventory information, specifically including:

[0228] 1. Structured aggregation of inventory data

[0229] After completing the inventory of the predetermined area, the system automatically aggregates all recognized electronic tag information to generate a structured list containing "tag ID, shelf layer, recognition time, signal strength".

[0230] At the same time, through a unique tag deduplication algorithm, repeated recognition is eliminated, such as a tag being repeatedly scanned by multiple antennas, ensuring that each tag in the list appears only once, avoiding inventory overestimation.

[0231] 2. Comparison and verification with the back-end system

[0232] The system compares the structured list with the preset inventory data in the warehouse management information system (WMS) (such as "shelf A3-2 layer should store 100 pieces, tag ID range E12300-E12400"):

[0233] If the tag ID is completely matched and the quantity is consistent (100 pieces), the inventory is automatically marked as correct;

[0234] If there is a difference (such as 95 pieces actually identified, missing E12395-E12400), the system will locate the difference position (A3-2 layer right side) and trigger an early warning, such as pushing 5 suspected out-of-stock pieces to the administrator.

[0235] 3. Unmanned data synchronization

[0236] After comparison, the system automatically updates the WMS inventory data, such as changing the "to be counted" state to "confirmed"; if there is a difference, the state is updated after the administrator confirms the processing (such as re-issuing goods), forming a "counting-verification-updating" closed loop.

[0237] The inventory data closed loop management is the final landing of step S4 "parameter adaptive adjustment and execution", step S4 ensures accurate counting data, and the inventory closed loop converts the accurate data into management value, solving the limitation of traditional RFID "only counting without linkage".

[0238] Corresponding to the above embodiment, the application also provides an electronic device.

[0239] As Figure 9 shown is a structural schematic diagram of an electronic device in the application, the electronic device 200 comprises a processor 201 and a memory 203. The processor 201 and the memory 203 are connected, for example, connected through a bus 202. Optionally, the electronic device 200 can also comprise a transceiver 204. It should be noted that in actual application, the transceiver 204 is not limited to one, and the structure of the electronic device 200 does not constitute a limitation on the embodiments of the application.

[0240] The processor 201 can be a CPU, a general-purpose processor, a DSP, an FPGA or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can realize or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure. The processor 201 can also be a combination of computing functions, such as one or more microprocessor combinations, DSP and microprocessor combinations, etc.

[0241] The bus 202 can comprise a channel for transmitting information between the above-mentioned components. The bus 202 can be a PCI bus or an EISA bus, etc. The bus 202 can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 9 only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0242] The memory 203 is configured to store a computer program corresponding to the fusion Q-learning algorithm and the dynamic power regulation RFID warehouse inventory method of the above-mentioned embodiments of the present application, and the computer program is controlled and executed by the processor 201. The processor 201 is configured to execute the computer program stored in the memory 203 to realize the content shown in the above-mentioned method embodiments.

[0243] The electronic device 200 includes but is not limited to mobile terminals such as notebook computers, PADs (tablets), and the like, and fixed terminals such as desktop computers and the like. Figure 9 The electronic device 200 shown is only an example and should not limit the functions and use range of the embodiments of the present application.

[0244] The electronic device 200 of the embodiments of the present application, by executing the fusion Q-learning algorithm and the dynamic power regulation RFID warehouse inventory method by the processor, has many beneficial effects: it can support real-time collection of environmental and signal parameters, construct a dynamic state vector to comprehensively depict the physical conditions and signal characteristics of the warehouse scene; run the Q-learning decision model to realize intelligent decision-making, dynamically adjust the transmission power, antenna beam pointing, working radio frequency points and other parameters, flexibly cope with complex dynamic scenes such as distance changes, multipath interference, and differences in product density, and improve the anti-interference ability and inventory accuracy; support model iteration optimization, continuously evolve the decision-making ability through real-time feedback, balance the inventory efficiency and resource consumption; it can also realize closed-loop management of inventory data, automatically aggregate, compare and synchronize inventory information, replace manual verification, and realize automatic updating of inventory information. Overall, the electronic device effectively overcomes the limitations of poor adaptability of static parameters, weak anti-interference ability, insufficient balance between efficiency and accuracy, and the like in traditional RFID technology, and significantly improves the dynamic adaptability, intelligence and practicality of warehouse inventory.

[0245] It is to be appreciated that the above description and the examples that follow are intended to be illustrative only and that changes can be made to the description, as represented by the above listed elements, by the steps recited in the flow charts, and by the examples that follow, without departing from the spirit of the application. Accordingly, the scope of the present application is intended to be defined only by the appended claims.

[0246] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, can be used: a hybrid of the above technologies, a combination of any of the above technologies, etc.

[0247] In the description of the present application, the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative appearances of the above-mentioned terms in various places in the specification are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0248] Furthermore, the terms "first", "second", etc. are used only for descriptive purposes and do not connote or imply relative importance or a quantity of the indicated technical features. Thus, a feature defined with "first", "second", etc. can include at least one of the features implicitly or explicitly. In the description of the present application, the meaning of "a plurality" is at least two, for example, two, three, etc., unless otherwise specifically defined.

[0249] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that changes, modifications, substitutions and variations can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. An RFID warehouse inventory system integrating Q-learning algorithm and dynamic power adjustment, characterized in that, The system includes: A reading device that moves within a warehouse environment, comprising: An environmental sensing unit acquires environmental sensing parameters that reflect the spatial relationship between the reading device and the goods to be inventoried; the environmental sensing parameters include distance parameters. The multi-antenna radio frequency identification unit transmits radio frequency signals and receives tag response signals to identify tag identity information and extract signal feature parameters that characterize signal propagation characteristics. Among the signal feature parameters, there is a phase fluctuation metric, which is derived by analyzing the standard deviation of the phase difference between the signals received by different antennas of the multi-antenna radio frequency identification unit from the same tag. Its value reflects the degree of interference caused by multipath effects due to environmental reflection and diffraction. The processing control core is configured to: construct a dynamic state vector, and characterize the physical conditions in the current environment that affect the propagation of radio frequency signals by integrating environmental perception parameters and signal characteristic parameters; Run the Q-learning decision model, receive dynamic state vectors as input, and output the optimal action command based on the preset optimization objective; Execute optimal action instructions, dynamically adjust the operating parameters of the multi-antenna RFID unit, and perform tag inventory. The operating parameters include at least the transmit power, antenna beam pointing, and operating RF frequency. The distance parameter is used to perform dual calculation tasks: (a) calculate the basic path loss based on the free space model; (b) determine the signal propagation scenario attributes by combining the warehouse environment structure information, including distinguishing whether the current signal propagation belongs to the line-of-sight propagation scenario or the non-line-of-sight propagation scenario, and in the non-line-of-sight propagation scenario, use a dynamic attenuation factor to correct the basic path loss. When executing the optimal action command, the processing control core implements transmit power correction through a hierarchical compensation architecture: First layer: Executes the baseline power command output by the Q-learning decision model based on the dynamic state vector to cope with signal fluctuations caused by environmental randomness; Second layer: Generates a compensation amount for deterministic distance attenuation based on the corrected basic path loss obtained in the non-line-of-sight propagation scenario; Third layer: Superimposes the compensation amount output by the second layer onto the baseline power of the first layer to output the final transmit power.

2. The system according to claim 1, characterized in that, The preset optimization objective is defined by a reward function, which is designed to use the number of unique electronic tags successfully identified per unit time as a positive incentive, thereby driving the Q-learning decision model to output action instructions that maximize inventory efficiency.

3. The system according to claim 1, characterized in that, The dynamic state vector includes channel attenuation features, which are determined based on environmental perception parameters and preset warehouse environment structure information. These features are used to quantify specific signal attenuation caused by the warehouse environment structure that is not distance-dependent.

4. The system according to claim 1, characterized in that, When the phase fluctuation metric exceeds a preset interference threshold, the Q-learning decision model prioritizes outputting action instructions for the interference suppression strategy, which includes: Switch to an anti-interference frequency band or activate multi-antenna beamforming. After switching the frequency band, it is necessary to verify whether the phase fluctuation measurement has dropped to within the threshold.

5. An RFID warehouse inventory counting method integrating Q-learning algorithm and dynamic power adjustment, characterized in that, The method of performing an inventory using the system as described in claim 1 includes the following steps: S1. Environment and Signal Perception: The driving reading device continuously acquires environmental perception parameters reflecting the physical relationship between itself and the goods to be inventoried during the inventory process, and simultaneously extracts signal feature parameters reflecting the signal propagation characteristics from the received electronic tag signals; the environmental perception parameters include distance parameters; the signal feature parameters include phase fluctuation measurement, which is derived by analyzing the standard deviation of the phase difference of the same tag signal received by different antennas; S2. State Space Construction: The environmental perception parameters and signal feature parameters obtained in step S1 are fused to construct a dynamic state vector, which is used to characterize the physical conditions in the current environment that affect the propagation of radio frequency signals. S3, Intelligent Decision-Making: The dynamic state vector is input into a pre-trained Q-learning decision model, which outputs the optimal action instructions aimed at maximizing inventory efficiency; S4. Parameter Adaptive Adjustment and Execution: Analyze the optimal action command and dynamically adjust the working parameters of the reading device in real time based on the analysis results. The working parameters include at least the transmit power, antenna beam pointing and working radio frequency, and the tag inventory operation is performed under these parameters. The distance parameter is used to perform dual calculation tasks: (a) calculate the basic path loss based on the free space model; (b) determine the signal propagation scenario attributes by combining the warehouse environment structure information, including distinguishing whether the current signal propagation belongs to the line-of-sight propagation scenario or the non-line-of-sight propagation scenario, and in the non-line-of-sight propagation scenario, use a dynamic attenuation factor to correct the basic path loss. In step S4, when executing the optimal action command, the transmit power is corrected through a hierarchical compensation architecture: First layer: Execute the baseline power command output by the Q-learning decision model based on the dynamic state vector to cope with signal fluctuations caused by environmental randomness; Second layer: Generate a compensation amount for deterministic distance attenuation based on the corrected basic path loss obtained in the non-line-of-sight propagation scenario; Third layer: Superimpose the compensation amount output by the second layer onto the baseline power of the first layer to output the final transmit power.

6. The method according to claim 5, characterized in that, The method also includes a model iterative optimization step: The number of unique tags successfully identified per unit time during the current inventory operation is used as a reward signal to feed back to the Q-learning decision model. This model is then used for online learning or offline retraining, so that its decision-making ability can continuously evolve as the inventory task is performed.

7. The method according to claim 5 or 6, characterized in that, The method also includes a closed-loop management step for inventory data: After completing the inventory task in the designated area, the system automatically summarizes all successfully identified electronic tag information, generates a structured inventory data list, and compares and synchronizes this list with the data in the backend warehouse management information system, thereby realizing automated verification and unmanned updating of inventory information.

Images