Commodity identification method and device based on Wi-Fi MIMO signal, electronic equipment and medium

Abstract

The invention discloses a commodity identification method and device based on a Wi-Fi MIMO signal, electronic equipment and a medium, and belongs to the technical field of data processing. The method comprises the following steps: acquiring various CSI channel information acquired from a target commodity based on a Wi-Fi MIMO signal; respectively extracting CSI feature information for the plurality of pieces of CSI channel information; fusing the plurality of pieces of CSI feature information to obtain fused feature data; and identifying the target commodity based on the fused feature data to obtain an identification result. According to the embodiment of the invention, cloud intervention is not needed in the commodity identification process, so that processing lag caused by network transmission delay, cloud computing power fluctuation and large video data volume is avoided, and scenes without network stability can be handled. Compared with commodity identification based on video data, the commodity identification based on the CSI channel information can be independent of illumination conditions and has strong robustness to shelters.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to a product identification method based on Wi-Fi MIMO signals, a product identification device based on Wi-Fi MIMO signals, an electronic device, and a readable storage medium. Background Technology

[0002] Currently, one method for product recognition involves uploading videos to the cloud, where servers then perform product recognition on the cloud-based videos. However, relying on cloud servers to centrally process video streams for product recognition suffers from network transmission latency, fluctuations in cloud computing power, and processing delays caused by large video data volumes. This results in poor real-time performance and an inability to adapt to unstable network scenarios. Consequently, it fails to meet the low-latency and high-reliability requirements of unattended vending machines. Summary of the Invention

[0003] The purpose of this application is to provide a product recognition method based on Wi-Fi MIMO signals, which can solve the problems of network transmission delay, cloud computing power fluctuation, and processing lag caused by large amounts of video data in current product recognition.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a product identification method based on Wi-Fi MIMO signals, the method comprising: Acquire multiple CSI channel information of target products based on Wi-Fi MIMO signals; CSI feature information is extracted from each of the multiple CSI channel information; Multiple CSI feature information are fused to obtain fused feature data; The target product is identified based on the fused feature data to obtain the identification result.

[0005] Optionally, the step of identifying the target product based on the fused feature data to obtain the identification result includes: Obtain fingerprint feature data of one or more products from a preset fingerprint database; The target product is identified based on the fused feature data and the fingerprint feature data to obtain the identification result.

[0006] Optionally, the step of identifying the target product based on the fused feature data and the fingerprint feature data to obtain the identification result includes: Determine the first feature vector corresponding to the fused feature data; Determine the second feature vector corresponding to the fingerprint feature data; Determine the vector distance between the first vector data and the second vector data; Based on the vector distance data, the target product is identified, and the identification result is obtained.

[0007] Optionally, determining the vector distance data between the first vector data and the second vector data includes: Determine the Euclidean distance between the first vector data and the second vector data; Alternatively, determine the cosine similarity between the first vector data and the second vector data.

[0008] Optionally, the target product is identified based on the fused feature data to obtain an identification result, including: Based on the fused feature data, it is determined whether the target product can be identified; When it is determined that the target product can be identified, the product ID corresponding to the target product is output.

[0009] Optionally, it also includes: When it is determined that the target product cannot be identified, the CSI feature information will be input into a preset deep learning classification model, and the reason for the identification anomaly will be output.

[0010] Optionally, the CSI channel information includes amplitude information, phase information, and multipath propagation information.

[0011] Secondly, embodiments of this application provide a device for product identification based on Wi-Fi MIMO signals, the device comprising: The CSI channel information acquisition module is used to acquire various CSI channel information of the target product based on Wi-Fi MIMO signals; The CSI feature information extraction module is used to extract CSI feature information for the multiple CSI channel information respectively; The fused feature data determination module is used to fuse multiple CSI feature information to obtain fused feature data; The product recognition module is used to recognize the target product based on the fused feature data and obtain the recognition result.

[0012] In one embodiment of this application, the product identification module 404 may include: The fingerprint feature data acquisition submodule is used to acquire fingerprint feature data of one or more products from a preset fingerprint database; The identification result determination submodule is used to identify the target product based on the fused feature data and the fingerprint feature data, and obtain the identification result.

[0013] Optionally, the identification result determination submodule may include: The first feature vector determination unit is used to determine the first feature vector corresponding to the fused feature data; The second feature vector determination unit is used to determine the second feature vector corresponding to the fingerprint feature data; A vector distance data determination unit is used to determine the vector distance data between the first vector data and the second vector data; The identification result determination unit is used to determine the identification of the target product based on the vector distance data and obtain the identification result.

[0014] Optionally, when determining the vector distance data, the vector distance data determining unit specifically performs the following: Determine the Euclidean distance between the first vector data and the second vector data.

[0015] Optionally, when determining the vector distance data, the vector distance data determining unit specifically performs the following: Determine the cosine similarity between the first vector data and the second vector data.

[0016] In one embodiment of this application, the product identification module 404 may include: The identification and judgment submodule is used to determine whether the target product can be identified based on the fused feature data; The Product ID submodule is used to output the Product ID corresponding to the target product when it is determined that the target product can be identified.

[0017] Optionally, the product recognition module may include: The anomaly analysis submodule is used to input the CSI feature information into a preset deep learning classification model and output the reason for the anomaly when it is determined that the target product cannot be identified.

[0018] Optionally, the device may further include: The preprocessing module is used to preprocess the multiple CSI channel information; The preprocessing includes one or more of the following: noise reduction processing and synchronization processing.

[0019] Optionally, the CSI channel information includes any one or more of amplitude information, phase information, and multipath propagation information.

[0020] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0021] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0022] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0023] In this embodiment, multiple CSI channel information collected from the target product based on Wi-Fi MIMO signals can be acquired. Then, CSI feature information is extracted from each of the multiple CSI channel information, and these multiple CSI feature information can be fused to obtain fused feature data. Finally, the target product is identified based on the fused feature data to obtain the identification result. This product identification process does not require cloud intervention, thus avoiding network transmission delays, fluctuations in cloud computing power, and processing lag caused by large amounts of video data. It can also cope with scenarios with unstable networks. Furthermore, in this embodiment, product identification based on CSI channel information, compared to product identification using video data, is independent of lighting conditions and has strong robustness to occlusions. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the steps of a product identification method based on Wi-Fi MIMO signals in an embodiment of this application. Figure 2 This is a flowchart illustrating the steps of another product identification method based on Wi-Fi MIMO signals in an embodiment of this application. Figure 3a This is a flowchart illustrating the steps of another product identification method based on Wi-Fi MIMO signals in an embodiment of this application. Figure 3b This is a flowchart illustrating a product identification method according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a commodity identification device based on Wi-Fi MIMO signals in an embodiment of this application; Figure 5 This is a schematic diagram of an electronic device structure in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0027] Currently, one method for product recognition involves uploading videos to the cloud, where servers then perform product recognition on the cloud-based videos. However, relying on cloud servers to centrally process video streams for product recognition suffers from network transmission latency, fluctuations in cloud computing power, and processing delays caused by large video data volumes. This results in poor real-time performance and an inability to adapt to unstable network scenarios. Consequently, it fails to meet the low-latency and high-reliability requirements of unattended vending machines.

[0028] In practical applications, another method for product identification is as follows: obtaining first identification information from the identification tag on the product to be taken; obtaining second identification information from the identification tag on the product to be taken; and determining the product information of the product to be taken based on the first identification information and the second identification information. According to the product identification method of the vending machine of the present invention, the product information of the product to be taken is determined based on the first and second identification information, providing dual identification protection and improving the accuracy and reliability of product identification. Firstly, it reduces the influence of external light on image recognition, thereby improving the recognition accuracy; secondly, it increases the range of products that image recognition can identify, allowing products that have not undergone extensive image training to be placed in the vending machine for sale; and simultaneously, compared to the high cost of RFID tags, the identification tags used in this invention have a lower cost.

[0029] In this embodiment, product recognition based on Wi-Fi MIMO signals can be achieved. Specifically, product recognition can be achieved by collecting CSI data and performing feature extraction and fusion. This process does not involve cloud processing as in the prior art, nor does it involve video data that depends on lighting conditions. This avoids network transmission delays, fluctuations in cloud computing power, and processing lag caused by large amounts of video data. It can also cope with scenarios where the network is unstable. In addition, it has strong robustness to product recognition in scenarios with obstructions.

[0030] The following description, in conjunction with the accompanying drawings, details the product identification based on Wi-Fi MIMO signals provided in this application through specific embodiments and application scenarios.

[0031] Reference Figure 1 The diagram illustrates a step-by-step flowchart of a product identification method based on Wi-Fi MIMO signals according to an embodiment of this application, specifically including the following steps: Step S101: Obtain multiple CSI channel information collected from the target product based on Wi-Fi MIMO signals; In this embodiment of the application, it is necessary to identify the goods in the vending machine. The target goods in the vending machine can be placed in a Wi-Fi environment, and then a signal is transmitted through a Wi-Fi MIMO device (such as a router). The receiving end (such as another router or a dedicated receiving device) collects CSI data to realize the acquisition of CSI channel data.

[0032] In one example, the CSI channel information includes one or more of amplitude information, phase information, and multipath propagation information. Amplitude information can reflect changes in signal strength and is affected by the material and shape of the product; phase information can reflect changes in the signal propagation path length and can be used to detect product movement; multipath propagation characteristics: different products have different reflection and scattering patterns of the signal, forming unique CSI features.

[0033] To acquire CSI channel information, you can pre-configure Wi-Fi devices that support CSI acquisition, such as Intel 5300 network cards (with Linux and CSI tools) or routers. Then, you can place the target product to be detected in the environment between the transmitter and receiver, which may include static placement, dynamic movement (such as when someone passes by), and other scenarios.

[0034] This allows for the transmission of Wi-Fi signals via MIMO antennas, with the receiver collecting CSI data. The channel state information between each MIMO antenna pair includes the amplitude and phase of multiple subcarriers.

[0035] In practical applications, different types of goods have different CSI channel data, which is the core basis for using Wi-Fi MIMO signals for goods identification in this application embodiment.

[0036] For example, consider 500ml of cola and 500ml of Wahaha purified water. Amplitude attenuation: The mean attenuation of 500ml cola is -25dB, with a standard deviation of 1.2dB (due to the sugar and carbon dioxide content, signal absorption is stronger); the mean attenuation of 500ml Wahaha purified water is -20dB, with a standard deviation of 0.8dB (pure water has weaker absorption and more stable multipath propagation). Phase shift: The main phase shift of 500ml cola is -0.15 rad / subcarrier, with a phase noise of 0.02 rad² (high sugar content causes drastic phase changes); the main phase shift of 500ml Wahaha purified water is -0.10 rad / subcarrier, with a phase noise of 0.01 rad² (pure water has a more stable phase). Multipath propagation characteristics: 500ml Coca-Cola (cylindrical bottle) has a time delay spread of 15ns and 4-6 multipaths (curved surface reflection causes multipath dispersion); 500ml Wahaha purified water (approximately cylindrical) has a time delay spread of 10ns and 3-5 multipaths (smoother surface, more concentrated multipaths).

[0037] Step S102: Extract CSI feature information for each of the multiple CSI channel information; After acquiring CSI channel information, different feature extraction methods can be performed according to the type of CSI channel information acquired to obtain the corresponding CSI feature information.

[0038] For example, amplitude information reflects the attenuation of a signal during propagation and can be used to identify the material, size, and distance of an object. Amplitude characteristics can be addressed in the following ways: (1) Statistical characteristics: Calculate the statistical measures of each subcarrier amplitude within the time window, such as mean, variance, skewness, kurtosis, etc.

[0039] (2) Frequency domain features: Perform Fourier transform on the amplitude sequence to extract frequency domain features, such as main frequency, spectral entropy, and frequency band energy.

[0040] (3) Time-frequency features: Use wavelet transform or short-time Fourier transform to extract time-frequency domain features, such as wavelet coefficient energy and time-frequency image features.

[0041] (4) Correlation characteristics: Calculate the amplitude correlation between different antenna pairs and different subcarriers to form a correlation matrix, and extract the eigenvalues ​​or statistics of the matrix.

[0042] The raw phase information is affected by carrier frequency offset (CFO) and sampling frequency offset (SFO), and usually needs to be calibrated first. The calibrated phase can be used for more refined identification, such as position and motion.

[0043] Phase features can be extracted using the following methods: (1) Phase difference: The phase difference between different antennas is used to estimate the angle of arrival (AoA) or eliminate common errors.

[0044] (2) Phase calibration: Linear errors in the phase are removed by linear transformation to obtain the calibrated phase sequence, and then statistical features are extracted.

[0045] (3) Phase change rate: Calculate the rate of change of phase over time, used to detect minute motions.

[0046] In this embodiment, before extracting CSI feature information from the plurality of CSI channel information, preprocessing can be performed on the plurality of CSI channel information; wherein, the preprocessing may include any one or more of denoising processing and synchronization processing. By preprocessing the CSI channel information, interference data can be removed, facilitating subsequent data fusion processing.

[0047] Step S103: Fuse multiple CSI feature information to obtain fused feature data; In practical applications, after acquiring multiple CSI feature information, multiple feature information can be fused to obtain fused feature data. Specifically, the fusion process can include weighted fusion or serial fusion.

[0048] Among them, concatenation fusion can merge features of different dimensions into a single vector, which includes features of all dimensions.

[0049] For example: amplitude feature vector: 2D (mean attenuation, standard deviation); phase feature vector: 2D (main phase shift, phase noise); multipath feature vector: 2D (time delay spread, number of multipaths).

[0050] After concatenation and fusion, a 6-dimensional vector is formed: F = [mean amplitude, standard deviation of amplitude, main phase offset, phase noise, time delay spread, number of multipaths].

[0051] Based on the examples above, we can see that after fusion, F-Cola = [ 1.5, 0.8, 2.0, 1.2, 1.0, 0.5];F_pure water = [ 1.0, 0.5, 1.5, 0.8, 0.5, 0.2].

[0052] In this process, weighted fusion can assign weights based on the importance of different types of features (such as amplitude weight 0.5, phase weight 0.3, multipath weight 0.2), and then calculate the weighted sum of each feature vector to obtain the final fusion vector.

[0053] Step S104: Identify the target product based on the fused feature data to obtain the identification result.

[0054] After obtaining the fused feature data, the fused feature data can reflect the characteristics of the target product, and then the fused feature data can be used to identify the target product and obtain the identification result.

[0055] In this embodiment, multiple CSI channel information collected from the target product based on Wi-Fi MIMO signals can be acquired. Then, CSI feature information is extracted from each of the multiple CSI channel information, and these multiple CSI feature information can be fused to obtain fused feature data. Finally, the target product is identified based on the fused feature data to obtain the identification result. This product identification process does not require cloud intervention, thus avoiding network transmission delays, fluctuations in cloud computing power, and processing lag caused by large amounts of video data. It can also cope with scenarios with unstable networks. Furthermore, in this embodiment, product identification based on CSI channel information, compared to product identification using video data, is independent of lighting conditions and has strong robustness to obstructions.

[0056] Reference Figure 2 This diagram illustrates a step-by-step flowchart of another product identification method based on Wi-Fi MIMO signals according to an embodiment of this application, specifically including the following steps: Step S201: Obtain multiple CSI channel information collected from the target product based on Wi-Fi MIMO signals; In this embodiment of the application, it is necessary to identify the goods in the vending machine. The target goods in the vending machine can be placed in a Wi-Fi environment, and then a signal is transmitted through a Wi-Fi MIMO device (such as a router). The receiving end (such as another router or a dedicated receiving device) collects CSI data to realize the acquisition of CSI channel data.

[0057] In one example, the CSI channel information includes one or more of amplitude information, phase information, and multipath propagation information. Amplitude information can reflect changes in signal strength and is affected by the material and shape of the product; phase information can reflect changes in the signal propagation path length and can be used to detect product movement; multipath propagation characteristics are unique CSI features formed because different products have different reflection and scattering patterns.

[0058] To acquire CSI channel information, you can pre-configure Wi-Fi devices that support CSI acquisition, such as Intel 5300 network cards (with Linux and CSI tools) or routers. Then, you can place the target product to be detected in the environment between the transmitter and receiver, which may include static placement, dynamic movement (such as when someone passes by), and other scenarios.

[0059] This allows for the transmission of Wi-Fi signals via MIMO antennas, with the receiver collecting CSI data. The channel state information between each MIMO antenna pair includes the amplitude and phase of multiple subcarriers.

[0060] In practical applications, different types of goods have different CSI channel data, which is the core basis for using Wi-Fi MIMO signals for goods identification in this application embodiment.

[0061] For example, consider 500ml of cola and 500ml of Wahaha purified water. Amplitude attenuation: The mean attenuation of 500ml cola is -25dB, with a standard deviation of 1.2dB (due to the sugar and carbon dioxide content, signal absorption is stronger); the mean attenuation of 500ml Wahaha purified water is -20dB, with a standard deviation of 0.8dB (pure water has weaker absorption and more stable multipath propagation). Phase shift: The main phase shift of 500ml cola is -0.15 rad / subcarrier, with a phase noise of 0.02 rad² (high sugar content causes drastic phase changes); the main phase shift of 500ml Wahaha purified water is -0.10 rad / subcarrier, with a phase noise of 0.01 rad² (pure water has a more stable phase). Multipath propagation characteristics: 500ml Coca-Cola (cylindrical bottle) has a time delay spread of 15ns and 4-6 multipaths (curved surface reflection causes multipath dispersion); 500ml Wahaha purified water (approximately cylindrical) has a time delay spread of 10ns and 3-5 multipaths (smoother surface, more concentrated multipaths).

[0062] Step S202: Extract CSI feature information for each of the multiple CSI channel information; After acquiring CSI channel information, different feature extraction methods can be performed according to the type of CSI channel information acquired to obtain the corresponding CSI feature information.

[0063] For example, amplitude information reflects the attenuation of a signal during propagation and can be used to identify the material, size, and distance of an object. Amplitude characteristics can be addressed in the following ways: (1) Statistical characteristics: Calculate the statistical measures of each subcarrier amplitude within the time window, such as mean, variance, skewness, kurtosis, etc.

[0064] (2) Frequency domain features: Perform Fourier transform on the amplitude sequence to extract frequency domain features, such as main frequency, spectral entropy, and frequency band energy.

[0065] (3) Time-frequency features: Use wavelet transform or short-time Fourier transform to extract time-frequency domain features, such as wavelet coefficient energy and time-frequency image features.

[0066] (4) Correlation characteristics: Calculate the amplitude correlation between different antenna pairs and different subcarriers to form a correlation matrix, and extract the eigenvalues ​​or statistics of the matrix.

[0067] The raw phase information is affected by carrier frequency offset (CFO) and sampling frequency offset (SFO), and usually needs to be calibrated first. The calibrated phase can be used for more refined identification, such as position and motion.

[0068] Phase features can be extracted using the following methods: (1) Phase difference: The phase difference between different antennas is used to estimate the angle of arrival (AoA) or eliminate common errors.

[0069] (2) Phase calibration: Linear errors in the phase are removed by linear transformation to obtain the calibrated phase sequence, and then statistical features are extracted.

[0070] (3) Phase change rate: Calculate the rate of change of phase over time, used to detect minute motions.

[0071] In this embodiment, before extracting CSI feature information from the plurality of CSI channel information, preprocessing can be performed on the plurality of CSI channel information; wherein, the preprocessing may include any one or more of denoising processing and synchronization processing. By preprocessing the CSI channel information, interference data can be removed, facilitating subsequent data fusion processing.

[0072] Step S203: Fuse multiple CSI feature information to obtain fused feature data; In practical applications, after acquiring multiple CSI feature information, multiple feature information can be fused to obtain fused feature data. Specifically, the fusion process can include weighted fusion or serial fusion.

[0073] Among them, concatenation fusion is to merge features of different dimensions into a single vector, which includes features of all dimensions.

[0074] For example: amplitude feature vector: 2D (mean attenuation, standard deviation); phase feature vector: 2D (main phase shift, phase noise); multipath feature vector: 2D (time delay spread, number of multipaths).

[0075] After concatenation and fusion, a 6-dimensional vector is formed: F = [mean amplitude, standard deviation of amplitude, main phase offset, phase noise, time delay spread, number of multipaths].

[0076] Based on the examples above, we can see that after fusion, F-Cola = [ 1.5, 0.8, 2.0, 1.2, 1.0, 0.5];F_pure water = [ 1.0, 0.5, 1.5, 0.8, 0.5, 0.2].

[0077] In this process, weighted fusion can assign weights based on the importance of different types of features (such as amplitude weight 0.5, phase weight 0.3, multipath weight 0.2), and then calculate the weighted sum of each feature vector to obtain the final fusion vector.

[0078] Step S204: Obtain fingerprint feature data of one or more products from the preset fingerprint database; In this embodiment, different products can be collected in advance, and CSI channel information can be collected based on different products. Then, CSI feature information can be extracted, and multiple CSI feature information can be fused to generate fingerprint feature data corresponding to each product. This data is then stored in a fingerprint database and used as a reference for other products to be detected.

[0079] Step S205: Identify the target product based on the fused feature data and the fingerprint feature data to obtain the identification result.

[0080] In one embodiment of this application, the step of identifying the target product based on the fused feature data and the fingerprint feature data to obtain an identification result includes: determining a first feature vector corresponding to the fused feature data; determining a second feature vector corresponding to the fingerprint feature data; determining the vector distance data between the first vector data and the second vector data; and identifying the target product based on the vector distance data to obtain an identification result.

[0081] In practical applications, each collected fused feature data and fingerprint feature data can be vectorized, and then the vector distance between the two vectors can be calculated. The vector distance reflects the degree of matching between the two; the smaller the vector distance, the better the match. Therefore, the product ID corresponding to the fingerprint feature data with the smallest vector distance can be used as the final recognition result.

[0082] In one embodiment of this application, determining the vector distance data between the first vector data and the second vector data includes: determining the Euclidean distance data between the first vector data and the second vector data.

[0083] In one embodiment of this application, determining the vector distance data between the first vector data and the second vector data includes: determining the cosine similarity between the first vector data and the second vector data.

[0084] In this embodiment, multiple CSI channel information collected from the target product based on Wi-Fi MIMO signals can be acquired. Then, CSI feature information is extracted from each of the multiple CSI channel information, and these multiple CSI feature information can be fused to obtain fused feature data. Finally, the target product is identified based on the fused feature data to obtain the identification result. This product identification process does not require cloud intervention, thus avoiding network transmission delays, fluctuations in cloud computing power, and processing lag caused by large amounts of video data. It can also cope with scenarios with unstable networks. Furthermore, in this embodiment, product identification based on CSI channel information, compared to product identification using video data, is independent of lighting conditions and has strong robustness to occlusions.

[0085] Reference Figure 3a This diagram illustrates a step-by-step flowchart of another product identification method based on Wi-Fi MIMO signals according to an embodiment of this application, specifically including the following steps: Step S301: Obtain multiple CSI channel information collected from the target product based on Wi-Fi MIMO signals; In this embodiment of the application, it is necessary to identify the goods in the vending machine. The target goods in the vending machine can be placed in a Wi-Fi environment, and then a signal is transmitted through a Wi-Fi MIMO device (such as a router). The receiving end (such as another router or a dedicated receiving device) collects CSI data to realize the acquisition of CSI channel data.

[0086] In one example, the CSI channel information includes one or more of amplitude information, phase information, and multipath propagation information. Amplitude information can reflect changes in signal strength and is affected by the material and shape of the product; phase information can reflect changes in the signal propagation path length and can be used to detect product movement; multipath propagation characteristics: different products have different reflection and scattering patterns of the signal, forming unique CSI features.

[0087] To acquire CSI channel information, you can pre-configure Wi-Fi devices that support CSI acquisition, such as Intel 5300 network cards (with Linux and CSI tools) or routers. Then, you can place the target product to be detected in the environment between the transmitter and receiver, which may include static placement, dynamic movement (such as when someone passes by), and other scenarios.

[0088] This allows for the transmission of Wi-Fi signals via MIMO antennas, with the receiver collecting CSI data. The channel state information between each MIMO antenna pair includes the amplitude and phase of multiple subcarriers.

[0089] In practical applications, different types of goods have different CSI channel data, which is the core basis for using Wi-Fi MIMO signals for goods identification in this application embodiment.

[0090] For example, consider 500ml of cola and 500ml of Wahaha purified water. Amplitude attenuation: The mean attenuation of 500ml cola is -25dB, with a standard deviation of 1.2dB (due to the sugar and carbon dioxide content, signal absorption is stronger); the mean attenuation of 500ml Wahaha purified water is -20dB, with a standard deviation of 0.8dB (pure water has weaker absorption and more stable multipath propagation). Phase shift: The main phase shift of 500ml cola is -0.15 rad / subcarrier, with a phase noise of 0.02 rad² (high sugar content causes drastic phase changes); the main phase shift of 500ml Wahaha purified water is -0.10 rad / subcarrier, with a phase noise of 0.01 rad² (pure water has a more stable phase). Multipath propagation characteristics: 500ml Coca-Cola (cylindrical bottle) has a time delay spread of 15ns and 4-6 multipaths (curved surface reflection causes multipath dispersion); 500ml Wahaha purified water (approximately cylindrical) has a time delay spread of 10ns and 3-5 multipaths (smoother surface, more concentrated multipaths).

[0091] Step S302: Extract CSI feature information for each of the multiple CSI channel information; After acquiring CSI channel information, different feature extraction methods can be performed according to the type of CSI channel information acquired to obtain the corresponding CSI feature information.

[0092] For example, amplitude information reflects the attenuation of a signal during propagation and can be used to identify the material, size, and distance of an object. Amplitude characteristics can be addressed in the following ways: (1) Statistical characteristics: Calculate the statistical measures of each subcarrier amplitude within the time window, such as mean, variance, skewness, kurtosis, etc.

[0093] (2) Frequency domain features: Perform Fourier transform on the amplitude sequence to extract frequency domain features, such as main frequency, spectral entropy, and frequency band energy.

[0094] (3) Time-frequency features: Use wavelet transform or short-time Fourier transform to extract time-frequency domain features, such as wavelet coefficient energy and time-frequency image features.

[0095] (4) Correlation characteristics: Calculate the amplitude correlation between different antenna pairs and different subcarriers to form a correlation matrix, and extract the eigenvalues ​​or statistics of the matrix.

[0096] The raw phase information is affected by carrier frequency offset (CFO) and sampling frequency offset (SFO), and usually needs to be calibrated first. The calibrated phase can be used for more refined identification, such as position and motion.

[0097] Phase features can be extracted using the following methods: (1) Phase difference: The phase difference between different antennas is used to estimate the angle of arrival (AoA) or eliminate common errors.

[0098] (2) Phase calibration: Linear errors in the phase are removed by linear transformation to obtain the calibrated phase sequence, and then statistical features are extracted.

[0099] (3) Phase change rate: Calculate the rate of change of phase over time, used to detect minute motions.

[0100] In this embodiment, before extracting CSI feature information from the plurality of CSI channel information, preprocessing can be performed on the plurality of CSI channel information; wherein, the preprocessing may include any one or more of denoising processing and synchronization processing. By preprocessing the CSI channel information, interference data can be removed, facilitating subsequent data fusion processing.

[0101] Step S303: Fuse multiple CSI feature information to obtain fused feature data; In practical applications, after acquiring multiple CSI feature information, multiple feature information can be fused to obtain fused feature data. Specifically, the fusion process can include weighted fusion or serial fusion.

[0102] Among them, concatenation fusion is to merge features of different dimensions into a single vector, which includes features of all dimensions.

[0103] For example: amplitude feature vector: 2D (mean attenuation, standard deviation); phase feature vector: 2D (main phase shift, phase noise); multipath feature vector: 2D (time delay spread, number of multipaths).

[0104] After concatenation and fusion, a 6-dimensional vector is formed: F = [mean amplitude, standard deviation of amplitude, main phase offset, phase noise, time delay spread, number of multipaths].

[0105] Based on the examples above, we can see that after fusion, F-Cola = [ 1.5, 0.8, 2.0, 1.2, 1.0, 0.5];F_pure water = [ 1.0, 0.5, 1.5, 0.8, 0.5, 0.2].

[0106] In this process, weighted fusion can assign weights based on the importance of different types of features (such as amplitude weight 0.5, phase weight 0.3, multipath weight 0.2), and then calculate the weighted sum of each feature vector to obtain the final fusion vector.

[0107] Step S304: Determine whether the target product can be identified based on the fused feature data; After obtaining the fused feature data, it can be determined whether the target product can be identified. When the fused feature data can find matching fingerprint feature data in the fingerprint database, it is determined that the target product can be identified, and then the product ID corresponding to the fingerprint feature data can be output.

[0108] Step S305: When it is determined that the target product can be identified, the product ID corresponding to the target product is output.

[0109] Step S306: When it is determined that the target product cannot be identified, the CSI feature information is input into a preset deep learning classification model, and the reason for the identification anomaly is output.

[0110] When the fused feature data cannot find matching fingerprint feature data in the fingerprint database, the target product is determined to be unidentifiable. CSI feature extraction uses existing standard methods. If significant discrepancies occur, it is often due to external factors. Therefore, a pre-trained deep learning classification model can be introduced to identify the cause of the anomaly. By inputting the CSI feature information into the preset deep learning classification model, the probability of identifying different causes of anomalies can be output, and the one with the highest probability is determined as the cause of the anomaly.

[0111] In practical applications, different models can be used for different types of CSI feature information. The specific model selection, optimization objectives, and methods are as follows: (1) Amplitude characteristics: Model selection: 1D-CNN and attention mechanism.

[0112] Optimization objective: To capture local decay patterns and long-range dependencies.

[0113] Technical details: Convolution kernel size [3,5,7] multi-scale extraction, attention weights dynamically focus on key subcarriers.

[0114] (2) Phase characteristics: Model selection: LSTM / Transformer.

[0115] Optimization objective: To model the temporal dynamics of phase shift.

[0116] Technical details: Input the phase values ​​of continuous time steps and predict the future phase change trend as an auxiliary task.

[0117] (3) Multipath characteristics: Model selection: Graph Neural Network (GNN).

[0118] Optimization objective: To resolve the topological relationships of multipath paths.

[0119] Technical details: Multipaths are treated as graph nodes, and latency and decay are treated as edge attributes. Path correlation is learned through message passing.

[0120] Reference Figure 3b This document illustrates a product identification process according to an embodiment of this application. The specific process is as follows: CSI channel signals are acquired using WIFIMIMO signals. These CSI channel signals are preprocessed, such as through noise reduction or synchronization. Feature extraction is then performed on the preprocessed data. Amplitude features can be extracted based on amplitude information, phase features based on phase information, and multipath features based on multipath information. After feature extraction, the amplitude features, phase features, and multipath features are fused. The fusion method can be either concatenation or weighted fusion. A large amount of CSI data for known products is collected, and a comprehensive feature vector is extracted and stored as a fingerprint database. Then, the distance between the vector to be identified and the vectors in the fingerprint database can be calculated using Euclidean distance or cosine similarity. The product corresponding to the fingerprint with the smallest distance is selected as the identification result.

[0121] In this embodiment, multiple CSI channel information collected from the target product based on Wi-Fi MIMO signals can be acquired. Then, CSI feature information is extracted from each of the multiple CSI channel information, and these multiple CSI feature information can be fused to obtain fused feature data. Finally, the target product is identified based on the fused feature data to obtain the identification result. This product identification process does not require cloud intervention, thus avoiding network transmission delays, fluctuations in cloud computing power, and processing lag caused by large amounts of video data. It can also cope with scenarios with unstable networks. Furthermore, in this embodiment, product identification based on CSI channel information, compared to product identification using video data, is independent of lighting conditions and has strong robustness to occlusions.

[0122] It should be noted that the product identification method based on Wi-Fi MIMO signals provided in this application embodiment can be executed by a product identification device based on Wi-Fi MIMO signals, or by a control module within that product identification device for executing the product identification method based on Wi-Fi MIMO signals. This application embodiment uses the execution of the product identification method based on Wi-Fi MIMO signals by a product identification device as an example to illustrate the product identification method based on Wi-Fi MIMO signals provided in this application embodiment.

[0123] Reference Figure 4 The diagram shows another product identification device based on Wi-Fi MIMO signals according to an embodiment of this application, which specifically includes the following modules: CSI channel information acquisition module 401 is used to acquire various CSI channel information of target commodities based on Wi-Fi MIMO signals; CSI feature information extraction module 402 is used to extract CSI feature information for the multiple CSI channel information respectively; The feature data fusion determination module 403 is used to fuse multiple CSI feature information to obtain fused feature data; The product recognition module 404 is used to recognize the target product based on the fused feature data and obtain the recognition result.

[0124] In one embodiment of this application, the product identification module 404 may include: The fingerprint feature data acquisition submodule is used to acquire fingerprint feature data of one or more products from a preset fingerprint database; The identification result determination submodule is used to identify the target product based on the fused feature data and the fingerprint feature data, and obtain the identification result.

[0125] In one embodiment of this application, the identification result determination submodule may include: The first feature vector determination unit is used to determine the first feature vector corresponding to the fused feature data; The second feature vector determination unit is used to determine the second feature vector corresponding to the fingerprint feature data; A vector distance data determination unit is used to determine the vector distance data between the first vector data and the second vector data; The identification result determination unit is used to determine the identification of the target product based on the vector distance data and obtain the identification result.

[0126] In one embodiment of this application, the vector distance data determining unit, when determining the vector distance data between the first vector data and the second vector data, is specifically used for: Determine the Euclidean distance between the first vector data and the second vector data; Alternatively, determine the cosine similarity between the first vector data and the second vector data.

[0127] In one embodiment of this application, the product identification module 404 may include: The identification and judgment submodule is used to determine whether the target product can be identified based on the fused feature data; The Product ID submodule is used to output the Product ID corresponding to the target product when it is determined that the target product can be identified.

[0128] In one embodiment of this application, the product identification module 404 may include: The anomaly analysis submodule is used to input the CSI feature information into a preset deep learning classification model and output the reason for the anomaly when it is determined that the target product cannot be identified.

[0129] In one embodiment of this application, the CSI channel information includes any one or more of amplitude information, phase information, and multipath propagation information.

[0130] In this embodiment, multiple CSI channel information collected from the target product based on Wi-Fi MIMO signals can be acquired. Then, CSI feature information is extracted from each of the multiple CSI channel information, and these multiple CSI feature information can be fused to obtain fused feature data. Finally, the target product is identified based on the fused feature data to obtain the identification result. This product identification process does not require cloud intervention, thus avoiding network transmission delays, fluctuations in cloud computing power, and processing lag caused by large amounts of video data. It can also cope with scenarios with unstable networks. Furthermore, in this embodiment, product identification based on CSI channel information, compared to product identification using video data, is independent of lighting conditions and has strong robustness to occlusions.

[0131] The product identification device based on Wi-Fi MIMO signals in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0132] The product identification device based on Wi-Fi MIMO signals in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0133] The product recognition device based on Wi-Fi MIMO signals provided in this application embodiment can achieve... Figures 1 to 3b The various processes implemented by the product identification device based on Wi-Fi MIMO signals in the method embodiments will not be described again here to avoid repetition.

[0134] Optionally, this application embodiment also provides an electronic device, including a processor 1010, a memory 1009, and a program or instructions stored in the memory 1009 and executable on the processor 1010. When the program or instructions are executed by the processor 1010, they implement the various processes of the above-described embodiment of the product identification method based on Wi-Fi MIMO signals and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0135] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0136] Figure 5 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application. The electronic device 1000 includes, but is not limited to, the following components: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, and processor 1010.

[0137] The memory 1009 includes applications and an operating system; the user input unit 1007 may include a touch panel 10071 and other input devices 10072; the input unit 1004 may include an image processor 10041 and a microphone 10042; and the display unit 1006 may include a display panel 10061.

[0138] Those skilled in the art will understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here. This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of the product identification method based on Wi-Fi MIMO signals and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0139] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0140] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described embodiment of the commodity identification method based on Wi-Fi MIMO signals, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0141] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0142] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0144] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A product identification method based on Wi-Fi MIMO signals, characterized in that, The method includes: Acquire multiple CSI channel information of target products based on Wi-Fi MIMO signals; CSI feature information is extracted from each of the multiple CSI channel information; Multiple CSI feature information are fused to obtain fused feature data; The target product is identified based on the fused feature data to obtain the identification result.

2. The method according to claim 1, characterized in that, The process of identifying the target product based on the fused feature data to obtain the identification result includes: Obtain fingerprint feature data of one or more products from a preset fingerprint database; The target product is identified based on the fused feature data and the fingerprint feature data to obtain the identification result.

3. The method according to claim 2, characterized in that, The process of identifying the target product based on the fused feature data and the fingerprint feature data to obtain the identification result includes: Determine the first feature vector corresponding to the fused feature data; Determine the second feature vector corresponding to the fingerprint feature data; Determine the vector distance between the first vector data and the second vector data; Based on the vector distance data, the target product is identified, and the identification result is obtained.

4. The method according to claim 3, characterized in that, The step of determining the vector distance data between the first vector data and the second vector data includes: Determine the Euclidean distance between the first vector data and the second vector data; Alternatively, determine the cosine similarity between the first vector data and the second vector data.

5. The method according to any one of claims 1 to 4, characterized in that, The target product is identified based on the fused feature data to obtain the identification result, including: Based on the fused feature data, it is determined whether the target product can be identified; When it is determined that the target product can be identified, the product ID corresponding to the target product is output.

6. The method according to claim 5, characterized in that, Also includes: When it is determined that the target product cannot be identified, the CSI feature information will be input into a preset deep learning classification model, and the reason for the identification anomaly will be output.

7. The method according to claim 1, characterized in that, The CSI channel information includes one or more of amplitude information, phase information, and multipath propagation information.

8. A product identification device based on Wi-Fi MIMO signals, characterized in that, The device includes: The CSI channel information acquisition module is used to acquire various CSI channel information of the target product based on Wi-Fi MIMO signals; The CSI feature information extraction module is used to extract CSI feature information for the multiple CSI channel information respectively; The fused feature data determination module is used to fuse multiple CSI feature information to obtain fused feature data; The product recognition module is used to recognize the target product based on the fused feature data and obtain the recognition result.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the product identification method based on Wi-Fi MIMO signals as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the product identification method based on Wi-Fi MIMO signals as described in any one of claims 1-7.

Images