Unmanned store customer behavior detection system based on machine vision and cloud computing

Abstract

The application provides a customer behavior detection system for unmanned stores based on machine vision and cloud computing, relates to the field of image data processing, and comprises a demand prediction module, a resource adjustment module, an information acquisition module and a behavior detection module.The demand prediction module is used for predicting future passenger flow based on historical passenger flow and passenger flow influence information of the unmanned store.The resource adjustment module is used for distributing edge computing resources and cloud computing resources based on the future passenger flow.The information acquisition module comprises an image acquisition unit, a shelf state acquisition unit and a personnel positioning unit.The behavior detection module is used for determining a plurality of target image acquisition devices from the image acquisition unit based on customer location information through the edge computing resources, determining a candidate behavior detection type based on shelf state information, extracting corresponding customer behavior features from images collected by the plurality of target image acquisition devices, and generating a customer behavior detection result according to customer behavior features at a plurality of time points through the cloud computing resources.The application has the advantage of realizing automatic detection of customer behavior in unmanned stores.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image data processing, in particular to a customer behavior detection system for unmanned stores based on machine vision and cloud computing. BACKGROUND

[0002] An unmanned store refers to a store that does not need a salesperson to sell and manage in the store. The unmanned store can ensure the normal sales of goods in the store by using high-tech technologies such as radio frequency identification technology, face recognition technology, and mobile payment technology. The unmanned store is favored by customers due to its convenience, unattended nature, and speed. At present, unmanned stores are spreading rapidly in major cities across the country, and will have a growing trend. The area of the unmanned store will also have a growing trend, which may cause a competitive impact on ordinary stores.

[0003] In the prior art, the behavior detection of customers in the unmanned store often only involves whether the payment is correct, and there is no good supervision mechanism for the remaining behaviors, which leads to some customers taking advantage of the loopholes of the unmanned store to perform some illegal operations.

[0004] Therefore, it is necessary to provide a customer behavior detection system for unmanned stores based on machine vision and cloud computing, which is used for automatic detection of customer behavior in unmanned stores. SUMMARY

[0005] The present application provides a customer behavior detection system for unmanned stores based on machine vision and cloud computing, comprising: a demand prediction module for predicting the future customer flow of the unmanned store based on the historical customer flow and customer flow influence information of the unmanned store; a resource adjustment module for adjusting the edge computing resources and cloud computing resources allocated to the unmanned store based on the future customer flow of the unmanned store; an information collection module comprising an image collection unit, a shelf state collection unit, and a personnel positioning unit, wherein the image collection unit comprises a plurality of image collection devices arranged at different positions in the unmanned store; a behavior detection module for determining a plurality of target image collection devices from the plurality of image collection devices based on the customer position information obtained by the personnel positioning unit through the edge computing resources allocated to the unmanned store, determining a candidate behavior detection type based on the shelf state information collected by the shelf state collection unit, extracting customer behavior features corresponding to the candidate behavior detection type from the images collected by the plurality of target image collection devices, and generating a customer behavior detection result according to the customer behavior features corresponding to the candidate behavior detection type at a plurality of time points extracted by the edge computing resources allocated to the unmanned store through the cloud computing resources allocated to the unmanned store.

[0006] Further, the demand prediction module predicts the future passenger flow of the unmanned store based on historical passenger flow and passenger flow influence information of the unmanned store, comprising: determining a plurality of passenger flow influence factors of the unmanned store based on historical passenger flow of the unmanned store, wherein the plurality of passenger flow influence factors include a plurality of target positions, a plurality of target meteorological factors and a plurality of key commodities; obtaining historical passenger flow of the unmanned store in a plurality of historical time periods of a current prediction period; obtaining passenger flow influence information based on the plurality of passenger flow influence factors of the unmanned store; and predicting the future passenger flow of the unmanned store based on the historical passenger flow of the unmanned store in the plurality of historical time periods of the current prediction period and the passenger flow influence information through a passenger flow prediction model.

[0007] Further, the demand prediction module determines a plurality of passenger flow influence factors of the unmanned store based on historical passenger flow of the unmanned store, comprising: determining a plurality of key positions, a plurality of key meteorological factors and a plurality of key commodities based on historical passenger flow of the unmanned store; for any one key position and any one key commodity, determining a sales influence coefficient of the key position on the key commodity based on historical passenger flow information of the key position and historical sales information of the key commodity; determining a plurality of target positions from the plurality of key positions based on the sales influence coefficient of any one key position on any one key commodity; for any one key meteorological factor and any one target position, determining a passenger flow influence coefficient of the key meteorological factor on the target position based on historical meteorological information of the unmanned store and historical passenger flow information of the target position; for any one key meteorological factor and any one key commodity, determining a sales influence coefficient of the key meteorological factor on the key commodity based on historical meteorological information of the unmanned store and historical sales information of the key commodity; and determining a plurality of target meteorological factors from the plurality of key meteorological factors based on the passenger flow influence coefficient of any one key meteorological factor on any one target position and the sales influence coefficient of any one key meteorological factor on any one key commodity, wherein the passenger flow influence information at least includes passenger flow of the plurality of target positions in a plurality of historical time periods of a current prediction period, characteristic values of the plurality of target meteorological factors in the plurality of historical time periods of the current prediction period, and sales information of the plurality of key commodities in the plurality of historical time periods of the current prediction period.

[0008] Further, the demand prediction module determines the plurality of key locations, the plurality of key meteorological factors and the plurality of key commodities based on historical customer flow of the unmanned store, comprising: determining a plurality of candidate locations; for each candidate location, obtaining historical customer flow information of the candidate location and historical customer flow of the unmanned store, and determining a customer flow influence coefficient of the candidate location on the unmanned store; determining the plurality of key locations from the plurality of candidate locations based on the customer flow influence coefficient of each candidate location on the unmanned store; determining a plurality of candidate meteorological factors; for each candidate meteorological factor, obtaining historical meteorological information of the unmanned store and historical customer flow of the unmanned store, and determining a customer flow influence coefficient of the candidate meteorological factor on the unmanned store; determining the plurality of key meteorological factors from the plurality of candidate meteorological factors based on the customer flow influence coefficient of each candidate meteorological factor on the unmanned store; determining a plurality of candidate commodities; for each candidate commodity, obtaining historical sales information of the candidate commodity of the unmanned store and historical customer flow of the unmanned store, and determining a customer flow influence coefficient of the candidate commodity on the unmanned store; determining the plurality of key commodities from the plurality of candidate commodities based on the customer flow influence coefficient of each candidate commodity on the unmanned store.

[0009] Further, the resource adjustment module adjusts the edge computing resources and the cloud computing resources allocated to the unmanned store based on future customer flow of the unmanned store, comprising: determining a plurality of customer abnormal behaviors; for each customer abnormal behavior, determining a customer abnormal behavior occurrence influence coefficient of each target location based on historical customer flow information of each target location and historical customer abnormal behavior records of the unmanned store, determining an associated location of the customer abnormal behavior from the plurality of target locations according to the customer abnormal behavior occurrence influence coefficient of each target location, determining a customer abnormal behavior occurrence influence coefficient of each target meteorological factor based on historical meteorological information of the unmanned store and historical customer abnormal behavior records, determining an associated target meteorological factor of the customer abnormal behavior from the plurality of target meteorological factors according to the customer abnormal behavior occurrence influence coefficient of each target meteorological factor, determining an associated commodity of the customer abnormal behavior based on historical sales information of the key commodity and historical customer abnormal behavior records of the unmanned store, determining prediction auxiliary information of the customer abnormal behavior based on the associated location, the associated target meteorological factor and the associated commodity of the customer abnormal behavior, predicting a future occurrence probability of the customer abnormal behavior based on the prediction auxiliary information of the customer abnormal behavior; determining future edge computing resource demand and cloud computing resource demand of the unmanned store based on future customer flow of the unmanned store and the future occurrence probability of each customer abnormal behavior; adjusting the edge computing resources and the cloud computing resources allocated to the unmanned store based on the future edge computing resource demand and the cloud computing resource demand of the unmanned store.

[0010] Further, the behavior detection module determines a plurality of target image collection devices from the plurality of image collection devices based on the customer position information obtained by the personnel positioning unit through the edge computing resources allocated to the unmanned store, including: determining a key image collection device from the plurality of image collection devices based on the customer position information obtained by the personnel positioning unit; determining a customer posture feature based on the image collected by the key target image collection device; determining an auxiliary image collection device from the plurality of image collection devices based on the customer posture feature, wherein the plurality of target image collection devices includes the key image collection device and the auxiliary image collection device.

[0011] Further, the shelf state acquisition unit includes a sound acquisition assembly, a vibration sensing assembly, and a weight sensing assembly, wherein the sound acquisition assembly includes sound acquisition devices arranged at a plurality of positions of the shelf, the vibration sensing assembly includes vibration sensing devices arranged at a plurality of positions of the shelf, and the weight sensing assembly includes weight sensing devices arranged at the commodity placement positions.

[0012] Further, the behavior detection module determines a candidate behavior detection type based on the shelf state information collected by the shelf state acquisition unit through the edge computing resources allocated to the unmanned store, including: determining an associated shelf state factor of each customer abnormal behavior; determining an abnormal shelf state factor based on the shelf state information collected by the shelf state acquisition unit; determining a real-time occurrence probability of each customer abnormal behavior based on the abnormal shelf state factor and the associated shelf state factor of each customer abnormal behavior; and determining the candidate behavior detection type based on the real-time occurrence probability and the future occurrence probability of each customer abnormal behavior.

[0013] Further, the behavior detection module extracts customer behavior features corresponding to the candidate behavior detection type from the images collected by the plurality of target image collection devices through the edge computing resources allocated to the unmanned store, including: establishing a feature behavior association graph, wherein the feature behavior association graph is used to record key image features of each customer abnormal behavior; and using a feature extraction model to extract customer behavior features corresponding to the candidate behavior detection type from the images collected by the plurality of target image collection devices according to the feature behavior association graph through the edge computing resources allocated to the unmanned store.

[0014] Further, the behavior detection module generates the customer behavior detection result according to the customer behavior features corresponding to the candidate behavior detection types at the multiple time points extracted by the edge computing resource allocated to the unmanned store, by using the cloud computing resource allocated to the unmanned store, including: using the behavior analysis model, generating the multi-frame behavior time sequence features of the customer according to the customer behavior features corresponding to the candidate behavior detection types at the multiple time points extracted by the edge computing resource allocated to the unmanned store; and generating the customer behavior detection result based on the multi-frame behavior time sequence features of the customer.

[0015] Compared with the prior art, the unmanned store customer behavior detection system based on machine vision and cloud computing provided by the application has at least the following beneficial effects:

[0016] 1、The demand prediction module predicts future passenger flow based on historical passenger flow and passenger flow influence information, and can grasp the customer flow trend of the store at different time periods in advance. The resource adjustment module dynamically adjusts the allocation of edge computing resources and cloud computing resources according to future passenger flow. When the passenger flow is large, increase resource allocation to meet the demand of real-time processing and analysis of a large amount of data, and ensure system performance; when the passenger flow is small, reduce resource allocation to reduce operating costs. This on-demand allocation method maximizes the use of resources and avoids waste. The information collection module includes an image collection unit, a shelf state collection unit, and a personnel positioning unit, which can comprehensively collect information in the unmanned store from multiple dimensions. The image collection unit can capture customer behavior and store scenes through multiple image collection devices at different positions; the shelf state collection unit monitors the state of goods on the shelf in real time, such as the number of goods, the placement position, etc.; the personnel positioning unit can accurately locate the customer's position in the store. These rich information provides a solid foundation for subsequent behavior detection and analysis. The personnel positioning unit can accurately obtain customer location information, and the behavior detection module determines multiple target image collection devices from multiple image collection devices based on this information. This precise positioning and device selection mechanism avoids unnecessary image processing, improves image analysis efficiency, and also reduces data transmission volume and network bandwidth pressure. The behavior detection module adopts a collaborative working mode of edge computing and cloud computing. The edge computing resource determines the target image collection device according to the customer location information, determines the candidate behavior detection type based on the shelf state information, and extracts the customer behavior features from the images collected by the target image collection device. The cloud computing resource uses the customer behavior features corresponding to the candidate behavior detection type at multiple time points extracted by the edge computing resource to generate the customer behavior detection result. This hierarchical processing method fully utilizes the real-time performance of edge computing and the powerful computing capability of cloud computing, improving the efficiency and accuracy of behavior detection. By comprehensively considering multiple dimensions of data such as customer location, shelf state, and image information, the behavior detection module can more comprehensively and deeply analyze customer behavior. For example, it can accurately identify customer behaviors such as browsing, picking up, and putting back goods, and realize automatic detection of customer behavior in unmanned stores.

[0017] 2、The method for determining multiple factors affecting the customer flow of the unmanned store considers the influence of individual factors on the customer flow and further analyzes the correlation between the factors. For example, the correlation between the location and the goods, the correlation between the meteorological factors and the location and the goods are considered, so that the prediction model can more comprehensively capture the change rule of the customer flow and avoid prediction deviation caused by ignoring the interaction between the factors. Through hierarchical progressive screening, factors with less or no influence on the customer flow are gradually excluded, and focus is focused on target locations, target meteorological factors and key goods with key influence. This makes the prediction model more concise and efficient, can more accurately reflect the core driving factors of the customer flow, and thus improves the prediction accuracy.

[0018] 3、The traditional resource allocation method has the problems of over-provisioning or under-provisioning, resulting in resource waste or system performance degradation. The resource adjustment module realizes on-demand allocation of resources by accurately predicting resource demand, avoids unnecessary resource occupation, and reduces operating costs. For example, in time periods with low customer flow and low probability of abnormal behavior, the allocation of edge computing resources and cloud computing resources can be appropriately reduced to reduce energy consumption and equipment maintenance costs. Reasonable allocation of resources can improve the utilization rate of resources, so that limited resources can bring the greatest benefit. By optimizing the combination of edge computing resources and cloud computing resources, the resource allocation ratio can be flexibly adjusted according to different situations to better meet the operation needs of the unmanned store, improve the use efficiency of resources, and further reduce operating costs. BRIEF DESCRIPTION OF DRAWINGS

[0019] The present specification will be further illustrated in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbers represent the same structures, wherein:

[0020] Figure 1 is a module schematic diagram of a customer behavior detection system for an unmanned store based on machine vision and cloud computing according to some embodiments of the present specification;

[0021] Figure 2 is a flowchart schematic diagram of determining multiple factors affecting the customer flow of the unmanned store according to some embodiments of the present specification;

[0022] Figure 3 is a schematic diagram of a feature behavior correlation graph according to some embodiments of the present specification. DETAILED DESCRIPTION

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some examples or embodiments of the present specification, and for those skilled in the art, the present specification can also be applied to other similar scenarios without creative labor. Unless it is obvious from the language environment or otherwise stated, the same reference numbers in the drawings represent the same structure or operation.

[0024] Figure 1 is a schematic diagram of a customer behavior detection system of an unmanned store based on machine vision and cloud computing according to some embodiments of the present specification, as shown in Figure 1 The customer behavior detection system of the unmanned store based on machine vision and cloud computing can include a demand prediction module, a resource adjustment module, an information collection module and a behavior detection module, as shown in the following.

[0025] The demand prediction module is configured to predict the future passenger flow of the unmanned store based on the historical passenger flow and passenger flow influence information of the unmanned store.

[0026] Specifically, it includes:

[0027] Based on the historical passenger flow of the unmanned store, a plurality of passenger flow influencing factors of the unmanned store are determined, wherein the plurality of passenger flow influencing factors include a plurality of target positions, a plurality of target meteorological factors and a plurality of key commodities.

[0028] The historical passenger flow of the unmanned store in a plurality of historical time periods of a current prediction period is obtained.

[0029] Based on the plurality of passenger flow influencing factors of the unmanned store, passenger flow influence information is obtained.

[0030] Based on the historical passenger flow of the unmanned store in a plurality of historical time periods of a current prediction period and the passenger flow influence information, the future passenger flow of the unmanned store is predicted by a passenger flow prediction model, wherein the passenger flow prediction model can be a long short-term memory (LSTM) model.

[0031] Figure 2 is a flowchart of determining a plurality of passenger flow influencing factors of an unmanned store according to some embodiments of the present specification, as shown in Figure 2 As preferred, the demand prediction module determines a plurality of passenger flow influencing factors of the unmanned store based on the historical passenger flow of the unmanned store, including:

[0032] Based on the historical passenger flow of the unmanned store, a plurality of key positions, a plurality of key meteorological factors and a plurality of key commodities are determined.

[0033] For any one key location and any one key commodity, based on the historical passenger flow information of the key location and the historical sales information of the key commodity, a sales influence coefficient of the key location on the key commodity is determined;

[0034] Based on the sales influence coefficient of any one key location on any one key commodity, a plurality of target locations are determined from the plurality of key locations;

[0035] For any one key meteorological factor and any one target location, based on the historical meteorological information of the unmanned store and the historical passenger flow information of the target location, a passenger flow influence coefficient of the key meteorological factor on the target location is determined;

[0036] For any one key meteorological factor and any one key commodity, based on the historical meteorological information of the unmanned store and the historical sales information of the key commodity, a sales influence coefficient of the key meteorological factor on the key commodity is determined;

[0037] Based on the passenger flow influence coefficient of any one key meteorological factor on any one target location and the sales influence coefficient of any one key meteorological factor on any one key commodity, a plurality of target meteorological factors are determined from the plurality of key meteorological factors, wherein the passenger flow influence information at least includes the passenger flow of the plurality of target locations in a plurality of historical time periods of a current prediction period, the characteristic values of the plurality of target meteorological factors in the plurality of historical time periods of the current prediction period, and the sales information of the plurality of key commodities in the plurality of historical time periods of the current prediction period.

[0038] As preferred, the demand prediction module determines the plurality of key locations, the plurality of key meteorological factors and the plurality of key commodities based on the historical passenger flow of the unmanned store, comprising:

[0039] A plurality of candidate locations are determined, wherein the candidate location is a location that the passenger flow may have an influence on the passenger flow of the unmanned store, for example, a street, an entrance of a mall, etc.;

[0040] For each candidate location, the historical passenger flow information of the candidate location and the historical passenger flow of the unmanned store are obtained, and a passenger flow influence coefficient of the candidate location on the unmanned store is determined;

[0041] Based on the passenger flow influence coefficient of each candidate location on the unmanned store, a plurality of key locations are determined from the plurality of candidate locations;

[0042] A plurality of candidate meteorological factors are determined;

[0043] For each candidate meteorological factor, the historical meteorological information of the unmanned store and the historical passenger flow of the unmanned store are obtained, and a passenger flow influence coefficient of the candidate meteorological factor on the unmanned store is determined;

[0044] determine a plurality of key weather factors from the plurality of candidate weather factors based on an influence coefficient of each candidate weather factor on the customer flow of the unmanned store;

[0045] determine a plurality of candidate commodities;

[0046] For each candidate commodity, obtain historical sales information of the candidate commodity of the unmanned store and historical customer flow of the unmanned store, and determine an influence coefficient of the candidate commodity on the customer flow of the unmanned store;

[0047] determine a plurality of key commodities from the plurality of candidate commodities based on the influence coefficient of each candidate commodity on the customer flow of the unmanned store.

[0048] Specifically, the Pearson correlation coefficient between the customer flow of the candidate location and the customer flow of the unmanned store can be calculated as the influence coefficient of the candidate location on the customer flow of the unmanned store according to the historical customer flow information of the candidate location and the historical customer flow of the unmanned store, and the candidate location with an influence coefficient greater than a first customer flow influence coefficient threshold can be selected as a key location.

[0049] The manner of screening the key weather factors and the key commodities is similar to the manner of screening the key locations, which will not be repeated here.

[0050] For each key location, a weighted influence coefficient of the key location can be calculated according to the sales influence coefficient of the key location on any one of the key commodities based on the following formula:

[0051]

[0052] wherein, is the weighted influence coefficient of the i-th key location, is the customer flow influence coefficient of the i-th key location on the unmanned store, is the sales influence coefficient of the i-th key location on the j-th key commodity, is the total number of key commodities.

[0053] The weighted influence coefficient of each key location is normalized according to the following formula:

[0054]

[0055] wherein, is the normalized weighted influence coefficient of the i-th key location, is the weighted influence coefficient of the m-th key location, and M is the total number of key locations.

[0056] The key location with a normalized weighted influence coefficient greater than a first weighted influence coefficient threshold is selected as a target location.

[0057] The manner of calculating the influence coefficient of the key meteorological factor on the passenger flow of the target location is similar to that of calculating the sales influence coefficient of the key location on the key commodity, which will not be described here.

[0058] For each key meteorological factor, the weighted influence coefficient of the key meteorological factor can be calculated according to the influence coefficient of the key meteorological factor on the passenger flow of any target location and the sales influence coefficient of the key meteorological factor on any key commodity based on the following formula:

[0059]

[0060] wherein, is the weighted influence coefficient of the kth key meteorological factor, is the passenger flow influence coefficient of the qth target location on the unmanned store, is the passenger flow influence coefficient of the kth key meteorological factor on the qth target location, is the total number of target locations, is the sales influence coefficient of the kth key meteorological factor on the jth key commodity.

[0061] The weighted influence coefficients of each key meteorological factor are normalized, and the key meteorological factor with a normalized weighted influence coefficient greater than a second weighted influence coefficient threshold is taken as a target meteorological factor.

[0062] It can be understood that the demand prediction module not only focuses on the historical passenger flow of the unmanned store itself, but also deeply excavates passenger flow influence factors in multiple dimensions, including location, meteorological factor, and commodity, etc. By comprehensively analyzing the influence of these factors on passenger flow, the change rule of passenger flow can be more comprehensively grasped, and the prediction deviation caused by relying on a single data source can be avoided, thereby improving the accuracy of the prediction result.

[0063] On this basis, the hierarchical screening method first determines candidate factors from multiple dimensions (location, weather, and commodity) to comprehensively cover various aspects that may affect passenger flow. Then, by calculating the influence coefficients of each candidate factor on passenger flow or related sales indicators, it conducts in-depth mining to accurately identify key factors that have a substantial impact on passenger flow, avoiding prediction bias caused by missing important factors. Specifically, for the location, weather factor, and commodity dimensions, the candidate factor set is determined. For each candidate factor, its influence coefficient on the passenger flow of the unmanned store is calculated. For example, in the location dimension, the influence coefficient of the candidate location on the passenger flow of the unmanned store is calculated by obtaining the historical passenger flow information of the candidate location and the historical passenger flow of the unmanned store; in the weather factor dimension, the influence coefficient is calculated using historical weather information and historical passenger flow; and in the commodity dimension, the influence coefficient is calculated based on the historical sales information of the candidate commodity and the historical passenger flow. Based on these influence coefficients, key factors are selected from the candidate factors of each dimension. This step mainly identifies factors that have a relatively obvious impact on passenger flow from numerous possible factors, narrowing the scope of subsequent analysis. After determining the key locations, the association between the key locations and key commodities is further analyzed. The sales influence coefficient of the key location on the key commodity is calculated, which helps to understand which locations have an important promoting effect on the sales of specific commodities. For key weather factors, the association between the key weather factors and the passenger flow of the target location (further selected based on the key locations) and the association between the key weather factors and the sales of the key commodities are analyzed. The influence coefficient of the key weather factor on the passenger flow of the target location and the sales influence coefficient of the key weather factor on the key commodity are calculated. Through this association analysis, the influence mechanism of weather factors on passenger flow in different scenarios can be better understood. Based on the sales influence coefficient of the key location on the key commodity, the target location is determined from the key locations. The target location is a location that not only has an impact on the overall passenger flow but also has an important promoting effect on the sales of specific commodities, better reflecting the synergistic effect between location and commodity sales. Considering the influence coefficient of the key weather factor on the passenger flow of the target location and the sales influence coefficient of the key weather factor on the key commodity, the target weather factor is determined from the key weather factors. This screening method not only considers the influence of individual factors on passenger flow but also analyzes the association between factors. For example, it considers the association between location and commodity, the association between weather factors and location and commodity, enabling the prediction model to more comprehensively capture the variation of passenger flow and avoid prediction bias caused by neglecting the interaction between factors. Through hierarchical screening, factors with less or no impact on passenger flow are gradually excluded, focusing on target locations, target weather factors, and key commodities that have a key impact. This makes the prediction model more concise and efficient, accurately reflecting the core driving factors of passenger flow and improving the accuracy of prediction.

[0064] The resource adjustment module is configured to adjust the edge computing resources and cloud computing resources allocated to the unmanned store based on the future customer flow of the unmanned store.

[0065] Specifically, the method comprises the following steps:

[0066] determining a plurality of customer abnormal behaviors;

[0067] For each customer abnormal behavior, determining an influence coefficient of each target location on the occurrence of the customer abnormal behavior based on historical customer flow information of each target location and historical customer abnormal behavior records of the unmanned store, determining a relevant location of the customer abnormal behavior from the plurality of target locations according to the influence coefficient of each target location on the occurrence of the customer abnormal behavior, determining an influence coefficient of each target meteorological factor on the occurrence of the customer abnormal behavior based on historical meteorological information of the unmanned store and historical customer abnormal behavior records, determining a relevant target meteorological factor of the customer abnormal behavior from the plurality of target meteorological factors according to the influence coefficient of each target meteorological factor on the occurrence of the customer abnormal behavior, determining a relevant product of the customer abnormal behavior based on historical sales information of the key product and historical customer abnormal behavior records of the unmanned store, determining prediction auxiliary information of the customer abnormal behavior based on the relevant location, the relevant target meteorological factor and the relevant product of the customer abnormal behavior, and predicting a future occurrence probability of the customer abnormal behavior based on the prediction auxiliary information of the customer abnormal behavior.

[0068] determining future edge computing resource demand and cloud computing resource demand of the unmanned store based on the future customer flow of the unmanned store and the future occurrence probability of each customer abnormal behavior.

[0069] adjusting the edge computing resources and cloud computing resources allocated to the unmanned store based on the future edge computing resource demand and cloud computing resource demand of the unmanned store.

[0070] Specifically, the plurality of customer abnormal behaviors can at least include:

[0071] 1. Hiding products

[0072] Behavior mode: escaping payment by hiding products (such as putting them in a bag or inside a coat).

[0073] 2. Product switching

[0074] Behavior mode: replacing high-priced products with low-priced product labels or directly replacing the packaging.

[0075] 3. Blocking the camera

[0076] Behavior mode: customers intentionally block the camera with their hands, objects or bodies to escape monitoring.

[0077] 4. Intentionally damaging products

[0078] Behavior pattern: customers damage goods due to personal emotions or malicious intent (e.g., squeezing food packaging, tearing labels).

[0079] 5. Damage equipment

[0080] Behavior pattern: intentionally damaging cameras, code scanning devices, or shelves, causing system failure.

[0081] 6. Loitering observation

[0082] Behavior pattern: customers loiter in the store for a long time but do not shop, which may be for the purpose of mapping or finding system vulnerabilities.

[0083] For each customer abnormal behavior, the nonlinear correlation coefficient between the target location and the customer abnormal behavior can be calculated based on the historical passenger flow of the target location and the number of occurrences of the historical customer abnormal behavior at multiple historical time periods, as the occurrence influence coefficient of the target location on the customer abnormal behavior. The target location with an occurrence influence coefficient greater than a second occurrence influence coefficient threshold can be selected as the associated location of the customer abnormal behavior.

[0084] The calculation of the occurrence influence coefficient of the target meteorological factor on the customer abnormal behavior is similar to the calculation of the occurrence influence coefficient of the target location on the customer abnormal behavior, which will not be repeated here. The screening of the associated target meteorological factor of the customer abnormal behavior and the associated commodity of the customer abnormal behavior is similar to the screening of the associated location of the customer abnormal behavior, which will not be repeated here.

[0085] The prediction auxiliary information of the customer abnormal behavior can include the passenger flow of the associated location of the customer abnormal behavior, the characteristic value of the associated target meteorological factor, and the sales volume of the associated commodity at multiple historical time periods in the current detection period.

[0086] The resource adjustment module can predict the future occurrence probability of each customer abnormal behavior based on the future passenger flow of the unmanned store and the prediction auxiliary information of each customer abnormal behavior through a probability prediction model, where the probability prediction model can be a long short-term memory network model.

[0087] The resource adjustment module can determine the future edge computing resource demand and cloud computing resource demand of the unmanned store based on the future passenger flow of the unmanned store and the future occurrence probability of each customer abnormal behavior through a demand prediction model, where the demand prediction model can be a deep neural network (DNN) model.

[0088] It can be understood that by determining multiple customer abnormal behaviors and analyzing their association with customer abnormal behaviors from multiple dimensions such as target location, target meteorological factors, and key commodities, various factors affecting customer abnormal behaviors are comprehensively considered. This multi-dimensional analysis can more accurately capture the occurrence regularity of customer abnormal behaviors, thereby providing a more accurate basis for resource demand prediction and avoiding resource allocation bias caused by single factor analysis.

[0089] Considering the future customer flow of the unmanned store and the future occurrence probability of each customer abnormal behavior to determine the resource demand makes the resource allocation more in line with the actual operation situation. Customer flow and abnormal behavior probability are interrelated, high customer flow may be accompanied by higher abnormal behavior occurrence risk. By combining these two factors, the resource demand under different conditions can be more accurately evaluated, and precise allocation of resources can be achieved.

[0090] Edge computing resources are close to data sources, which can quickly process data and respond. For unmanned stores, timely processing of customer abnormal behaviors is crucial to ensure operational safety and improve customer experience. By accurately predicting the occurrence probability of customer abnormal behaviors, edge computing resources can be reasonably allocated to ensure that edge computing nodes can respond quickly when abnormal behaviors occur, take timely measures such as starting monitoring and warning, adjusting device status, etc., and improve the real-time performance and reliability of the system.

[0091] Cloud computing resources have strong computing and storage capabilities, which are suitable for processing large-scale data and complex computing tasks. In the resource adjustment module, cloud computing resources are reasonably allocated according to future resource demand, which can ensure that the cloud computing platform can provide sufficient computing support when large amounts of data need to be processed or complex analysis needs to be performed, ensuring the overall performance of the system.

[0092] Traditional resource allocation methods have the problem of over-provisioning or under-provisioning, leading to resource waste or system performance degradation. The resource adjustment module accurately predicts resource demand, realizes on-demand allocation of resources, avoids unnecessary resource occupation, and reduces operating costs. For example, during periods of low customer flow and low abnormal behavior occurrence probability, edge computing resources and cloud computing resources can be appropriately reduced to reduce energy consumption and device maintenance costs.

[0093] Reasonable allocation of resources can improve the utilization rate of resources and maximize the benefits of limited resources. By optimizing the combination of edge computing resources and cloud computing resources and flexibly adjusting the resource allocation ratio according to different situations, the operational needs of unmanned stores can be better met, the use efficiency of resources can be improved, and operating costs can be further reduced.

[0094] The information collection module comprises an image collection unit, a shelf state collection unit and a personnel positioning unit, wherein the image collection unit comprises a plurality of image collection devices arranged at different positions in the unmanned store.

[0095] Preferably, the shelf state collection unit comprises a sound collection assembly, a vibration sensing assembly and a weight sensing assembly, wherein the sound collection assembly comprises sound collection devices arranged at a plurality of positions of the shelf, through which it can be detected whether the customer intentionally damages the goods, the vibration sensing assembly comprises vibration sensing devices arranged at a plurality of positions of the shelf, through which it can be detected whether the customer intentionally damages the shelf, and the weight sensing assembly comprises weight sensing devices arranged at the goods placing positions, through which it can be detected the taking situation of each piece of goods.

[0096] Specifically, in the scenario of an unmanned store, the customer intentionally damaging goods may produce some specific sound signals. For example, when the customer knocks or throws the goods with force, different materials of the goods will produce different frequency and intensity of sound. The sound collection devices are arranged at a plurality of positions of the shelf, which can capture sound information around the shelf in all directions. Through audio analysis algorithms, the collected sound can be feature extracted and pattern recognized. For example, a database containing various normal operation sounds (such as the sound of customers normally picking up and putting down goods) and intentionally damaging goods sounds (such as the sound of glass breaking and the sound of plastic products being hit hard) is established in advance. When the sound collection device detects audio matching the features of the intentionally damaging goods sounds in the database, it can be determined that there may be a customer's behavior of intentionally damaging goods.

[0097] The shelf will vibrate when subjected to external forces, and the vibration characteristics produced by the customer intentionally damaging the shelf (such as shaking or hitting the shelf with force) are different from normal customer operations (such as gently pushing the shelf to pick goods). The vibration sensing devices are arranged at a plurality of positions of the shelf, which can monitor the vibration of each part of the shelf in real time. The vibration data collected by the vibration sensing devices are processed and analyzed, for example, by analyzing the amplitude, frequency, duration and other parameters of the vibration. When the vibration amplitude is too large, the frequency is abnormal or the duration exceeds the normal range, it is determined that there may be a customer's behavior of intentionally damaging the shelf.

[0098] The weight sensing devices are arranged at the goods placing positions, and when the goods are picked up or put back, the weight of the goods placing positions will change. These weight changes can be monitored in real time and judged according to pre-set rules. For example, when a customer picks up a piece of goods, the weight of the goods placing position will decrease, and the decreased weight and the corresponding time are recorded. If the customer eventually purchases the goods, the system will make corresponding records at the settlement link; if the customer does not purchase and puts the goods back, but the position is incorrect, the abnormal situation can also be detected through the weight change.

[0099] The behavior detection module determines a plurality of target image acquisition devices from the plurality of image acquisition devices based on the customer position information obtained by the personnel positioning unit through the edge computing resource allocated to the unmanned store, determines a candidate behavior detection type based on the shelf state information collected by the shelf state collection unit, extracts customer behavior features corresponding to the candidate behavior detection type from the images collected by the plurality of target image acquisition devices, and generates a customer behavior detection result according to the customer behavior features corresponding to the candidate behavior detection type at a plurality of time points extracted by the edge computing resource allocated to the unmanned store through the cloud computing resource allocated to the unmanned store.

[0100] Preferably, the behavior detection module determines a plurality of target image acquisition devices from the plurality of image acquisition devices based on the customer position information obtained by the personnel positioning unit through the edge computing resource allocated to the unmanned store, including:

[0101] Based on the customer position information obtained by the personnel positioning unit, a key image acquisition device is determined from the plurality of image acquisition devices. Specifically, the image acquisition device closest to the customer position can be selected as the key image acquisition device. For example, the behavior detection module can calculate the straight-line distance between each image acquisition device and the customer position point through a distance algorithm such as the Euclidean distance algorithm, and select the image acquisition device with the smallest distance value as the key image acquisition device.

[0102] Based on the image collected by the key target image acquisition device, the customer posture feature is determined. Specifically, the positions of various key joint points of the human body in the image can be identified through a human body posture estimation model, such as the head, shoulder, elbow, hand, hip, knee, and foot. By connecting and analyzing these key joint points, a customer posture skeleton diagram can be constructed to determine the customer's posture feature, such as standing, bending, squatting, and reaching out.

[0103] Based on the customer posture feature, an auxiliary image acquisition device is determined from the plurality of image acquisition devices. The plurality of target image acquisition devices includes the key image acquisition device and the auxiliary image acquisition device. Specifically, according to the customer's posture feature, the layout and perspective information of the image acquisition devices in the unmanned store are combined to select auxiliary image acquisition devices that can provide additional useful information from the plurality of image acquisition devices. For example, when the key image acquisition device shows that the customer is in a bent-over posture to view the bottom layer of the shelf, the perspectives of other image acquisition devices around are analyzed, and those image acquisition devices that can capture the customer's bent-over area or the customer's hand movements from different angles are selected as auxiliary image acquisition devices. These auxiliary devices can provide a more comprehensive perspective and supplement the details that the key device may not be able to capture, such as the contact between the customer's hand and the goods, the movements of other parts of the customer's body, etc.

[0104] As preferred, the behavior detection module determines the candidate behavior detection type based on the shelf state information collected by the shelf state collection unit through the edge computing resource allocated to the unmanned store, including:

[0105] Determine the associated shelf state factors of each customer abnormal behavior, for example, hiding goods can be associated with the weight of the goods, goods switching can be associated with the weight of the goods, and damaged goods can be associated with shelf vibration, sound, etc.

[0106] Based on the shelf state information collected by the shelf state collection unit, determine the abnormal shelf state factors, specifically, the abnormal shelf state factors can be determined based on the shelf state information collected by the shelf state collection unit through the abnormal judgment model, wherein the abnormal judgment model can be a convolutional neural network (CNN) model;

[0107] Based on the abnormal shelf state factors and the associated shelf state factors of each customer abnormal behavior, determine the real-time occurrence probability of each customer abnormal behavior, specifically, the real-time occurrence probability of each customer abnormal behavior can be determined based on the abnormal shelf state factors and the associated shelf state factors of each customer abnormal behavior through the probability prediction model, wherein the probability prediction model can be a convolutional neural network model;

[0108] Based on the real-time occurrence probability and the future occurrence probability of each customer abnormal behavior, determine the candidate behavior detection type, specifically, for each customer abnormal behavior, the real-time occurrence probability and the future occurrence probability of the customer abnormal behavior can be weighted to determine the comprehensive occurrence probability of the customer abnormal behavior, and the customer abnormal behavior with a comprehensive occurrence probability greater than a comprehensive occurrence probability threshold is taken as the candidate behavior detection type.

[0109] As preferred, the behavior detection module extracts the customer behavior features corresponding to the candidate behavior detection type from the images collected by the multiple target image collection devices through the edge computing resource allocated to the unmanned store, including:

[0110] Establish a feature behavior association graph;

[0111] Through the edge computing resource allocated to the unmanned store, use the feature extraction model to extract the customer behavior features corresponding to the candidate behavior detection type from the images collected by the multiple target image collection devices according to the feature behavior association graph, wherein the feature extraction model can be a YOLOv8 model.

[0112] Specifically, Figure 3 is a schematic diagram of the feature behavior association graph according to some embodiments of the present specification, such as Figure 3As shown, the feature behavior correlation map is used to record the key image features of each customer abnormal behavior, for example:

[0113] The customer behavior features corresponding to hiding goods can include:

[0114] 1. Body part blocking action: When the customer puts the goods into the bag, there will be an action of the hand stretching into the bag and blocking the goods; if hidden in the coat, there will be image features of the hand operating in the coat, such as the local bulge of the coat and the hand action can be vaguely seen. For example, in the image, it can be seen that the customer holds the goods with one hand, and the other hand quickly puts the goods into the side pocket of the coat, and the coat at this position appears obvious bulge.

[0115] 2. Abnormal position of goods: The goods are originally placed on the shelf, and after hiding, the goods disappear in the normal field of view, but the outline or color similar to the goods appears in the specific part of the customer's body (such as the bag or the coat). For example, the snacks originally placed neatly on the shelf disappear after the customer passes by, and at the same time, it is observed that there is a color and shape similar to the snack packaging on one side of the customer's backpack.

[0116] The customer behavior features corresponding to goods switching can include:

[0117] 1. Label operation action: The customer will have the action of tearing off the original label or pasting a new label. In the image, it can be seen that the customer pinches the edge of the product label with his fingers and tears it off with force, or holds the new label close to the product for pasting operation.

[0118] 2. Comparison of differences in appearance of goods: There are obvious differences in appearance before and after switching, such as packaging color, pattern, shape, etc. By comparing the appearance features of the goods in the images before and after switching, it can be found that high-priced goods are replaced by low-priced goods labels or packaging. For example, the originally exquisite imported chocolate packaging becomes the packaging of ordinary domestic chocolate after switching.

[0119] 3. Environmental attention action: In the image, the customer's eyes occasionally glance towards the direction of the camera, or observe whether there are other customers around who notice his behavior.

[0120] The customer behavior features corresponding to blocking the camera can include:

[0121] 1. Blocking object action: When the customer deliberately blocks the camera with his hand, object or body, the action of the blocking object will be clearly shown in the image. For example, the customer stretches his hand to block the camera with a piece of paper.

[0122] 2. Body posture adjustment: In order to better block the camera, the customer may adjust the body posture, such as standing on tiptoe, turning sideways, etc. In the image, the change in posture of the customer can be captured, such as slightly leaning forward and stretching the arm upwards to block the camera.

[0123] 3. Occluded area features: The area of the camera that is occluded will appear as a distinct black or blurry area in the image, contrasting sharply with the normal surveillance footage. By analyzing the changes in brightness and clarity of different areas in the image, it can be determined whether the camera is being occluded.

[0124] The customer behavior features corresponding to intentional damage of merchandise can include:

[0125] 1. Damage actions: When a customer forcefully squeezes, throws, or tears a product, there will be clear action features in the image. For example, a customer may forcefully squeeze a food package, causing it to deform or break, or throw a product high into the air and then drop it on the ground.

[0126] 2. Merchandise damage state: Damaged merchandise will exhibit states such as breakage, deformation, and contents spilling in the image. For example, after a beverage bottle is broken, liquid will splash everywhere, and after a food package is torn open, food will scatter on the ground.

[0127] 3. Expression and emotional performance: Some customers may exhibit anger or dissatisfaction when intentionally damaging merchandise, and facial expressions can assist in determining their behavior intentions. In the image, the customer may be seen with a furrowed brow and a fierce gaze while performing the action of damaging the merchandise.

[0128] The customer behavior features corresponding to damage of equipment can include:

[0129] 1. Damage equipment actions: When a customer forcefully shakes, hits, or knocks a camera, code scanning device, or shelf, there will be corresponding action records in the image. For example, a customer may forcefully shake a camera stand, causing the camera to shake, or hit a code scanning device with a fist.

[0130] 2. Equipment damage state: Damaged equipment will exhibit signs of damage in the image, such as a broken camera lens, a broken code scanning device shell, or a deformed shelf. By comparing the normal state of the equipment with the image after it has been damaged, the damage to the equipment can be clearly seen.

[0131] The customer behavior features corresponding to loitering and observation can include:

[0132] 1. Stopping and observing actions: Customers may stand in front of certain shelves for a long time, carefully observing the merchandise, but not immediately taking it. In the image, the customer can be seen standing in front of the shelf, leaning slightly forward, and staring at the merchandise, sometimes picking up the merchandise to examine it and then putting it back in its original position.

[0133] 2. Head turning actions: To observe the surrounding environment and the movements of other customers, customers who are loitering and observing will frequently turn their heads. The image can capture the action of the customer's head turning left and right, indicating that they are alert to the surrounding situation.

[0134] As preferred, the behavior detection module generates the customer behavior detection result based on the customer behavior features corresponding to the candidate behavior detection types at multiple time points extracted by the edge computing resource allocated to the unmanned store, through the cloud computing resource allocated to the unmanned store, including:

[0135] Through the cloud computing resource allocated to the unmanned store, the customer multi-frame behavior time sequence features are generated based on the customer behavior features corresponding to the candidate behavior detection types at multiple time points extracted by the edge computing resource allocated to the unmanned store, using the behavior analysis model.

[0136] Based on the customer multi-frame behavior time sequence features, the customer behavior detection result is generated.

[0137] Specifically, after receiving the customer behavior features corresponding to the candidate behavior detection types at multiple time points transmitted by the edge computing, the cloud computing resource inputs these features into the behavior analysis model. The model comprehensively analyzes these features, considering the correlation and changes between different time points, thereby generating the customer multi-frame behavior time sequence features. The multi-frame behavior time sequence features can more comprehensively reflect the dynamic process of customer behavior, containing the evolution information of behavior in the time dimension. For example, for the behavior of hiding goods, the multi-frame behavior time sequence features can reflect the feature changes of the entire process from picking up the goods, observing the surrounding environment to hiding the goods in the bag. After obtaining the customer multi-frame behavior time sequence features, the cloud computing resource will classify and identify these features using the behavior analysis model. The model will determine which type the current customer behavior belongs to according to the various normal and abnormal behavior patterns learned in advance. For example, the multi-frame behavior time sequence features are matched with known behavior patterns of hiding goods, goods switching, normal shopping, etc., and the similarity is calculated to determine the specific category of customer behavior. According to the results of behavior classification and identification, the cloud computing resource generates the customer behavior detection result. The detection result can include the type of behavior (such as normal behavior, hiding goods behavior, etc.), the confidence of the behavior (indicating the reliability of the model's judgment of the behavior), and the time and location of the behavior, etc. After generating the detection result, the cloud computing resource will feed it back to the management system of the unmanned store, so that the management personnel can take appropriate measures in a timely manner, such as warning abnormal behavior, preventing customers from violating the rules, etc.

[0138] Finally, it should be understood that the embodiments described in the specification are only used to illustrate the principles of the embodiments of the specification. Other variations can also belong to the scope of the specification. Therefore, as an example but not limitation, alternative configurations of the embodiments of the specification can be considered consistent with the teachings of the specification. Accordingly, the embodiments of the specification are not limited to the embodiments explicitly introduced and described in the specification.

Claims

1. A system for detecting customer behavior in an unmanned store based on machine vision and cloud computing, characterized in that, Comprise: A demand prediction module for predicting future customer flow of the unmanned store based on historical customer flow and customer flow influence information of the unmanned store; A resource adjustment module for adjusting edge computing resources and cloud computing resources allocated to the unmanned store based on future customer flow of the unmanned store; An information collection module comprising an image collection unit, a shelf state collection unit and a personnel positioning unit, wherein the image collection unit comprises a plurality of image collection devices arranged at different positions in the unmanned store; A behavior detection module for determining a plurality of target image collection devices from the plurality of image collection devices based on customer location information obtained by the personnel positioning unit through the edge computing resources allocated to the unmanned store, determining a candidate behavior detection type based on shelf state information collected by the shelf state collection unit, extracting customer behavior features corresponding to the candidate behavior detection type from images collected by the plurality of target image collection devices, and generating a customer behavior detection result through the cloud computing resources allocated to the unmanned store according to customer behavior features corresponding to the candidate behavior detection type at a plurality of time points extracted by the edge computing resources allocated to the unmanned store; The demand prediction module predicts future customer flow of the unmanned store based on historical customer flow and customer flow influence information of the unmanned store, comprising: Based on the historical customer flow of the unmanned store, a plurality of customer flow influence factors of the unmanned store are determined, wherein the plurality of customer flow influence factors comprise a plurality of target positions, a plurality of target meteorological factors and a plurality of key goods; Obtain the historical customer flow of the unmanned store in a plurality of historical time periods of the current prediction period; Based on the plurality of customer flow influence factors of the unmanned store, obtain customer flow influence information; Through a customer flow prediction model, based on the historical customer flow of the unmanned store in a plurality of historical time periods of the current prediction period and the customer flow influence information, predict the future customer flow of the unmanned store; The demand prediction module determines a plurality of customer flow influence factors of the unmanned store based on historical customer flow of the unmanned store, comprising: Based on the historical customer flow of the unmanned store, a plurality of key positions, a plurality of key meteorological factors and a plurality of key goods are determined; For any one key position and any one key good, based on historical passenger flow information of the key position and historical sales information of the key good, a sales influence coefficient of the key position on the key good is determined; Based on the sales influence coefficient of any one key position on any one key good, a plurality of target positions are determined from the plurality of key positions; For any one key meteorological factor and any one target position, based on historical meteorological information of the unmanned store and historical passenger flow information of the target position, a passenger flow influence coefficient of the key meteorological factor on the target position is determined; For any one key meteorological factor and any one key good, based on historical meteorological information of the unmanned store and historical sales information of the key good, a sales influence coefficient of the key meteorological factor on the key good is determined; The demand prediction module determines the plurality of key locations, the plurality of key meteorological factors and the plurality of key commodities based on the historical customer flow of the unmanned store, and includes: 2.The machine vision and cloud computing based unmanned store customer behavior detection system of claim 1, wherein, determining a plurality of candidate locations; for each candidate location, obtaining historical customer flow information of the candidate location and historical customer flow of the unmanned store, and determining a customer flow influence coefficient of the candidate location on the unmanned store; determining a plurality of key locations from the plurality of candidate locations based on the customer flow influence coefficient of each candidate location on the unmanned store; determining a plurality of candidate meteorological factors; for each candidate meteorological factor, obtaining historical meteorological information of the unmanned store and historical customer flow of the unmanned store, and determining a customer flow influence coefficient of the candidate meteorological factor on the unmanned store; determining a plurality of key meteorological factors from the plurality of candidate meteorological factors based on the customer flow influence coefficient of each candidate meteorological factor on the unmanned store; determining a plurality of candidate commodities; for each candidate commodity, obtaining historical sales information of the candidate commodity of the unmanned store and historical customer flow of the unmanned store, and determining a customer flow influence coefficient of the candidate commodity on the unmanned store; determining a plurality of key commodities from the plurality of candidate commodities based on the customer flow influence coefficient of each candidate commodity on the unmanned store. The resource adjustment module adjusts the edge computing resources and cloud computing resources allocated to the unmanned store based on the future customer flow of the unmanned store, and includes: 3.The machine vision and cloud computing based unmanned store customer behavior detection system of claim 1, wherein, determining a plurality of customer abnormal behaviors; for each customer abnormal behavior, determining an occurrence influence coefficient of each target location on the customer abnormal behavior based on historical customer flow information of each target location and historical customer abnormal behavior records of the unmanned store, determining an associated location of the customer abnormal behavior from the plurality of target locations according to the occurrence influence coefficient of each target location on the customer abnormal behavior, determining an occurrence influence coefficient of each target meteorological factor on the customer abnormal behavior based on historical meteorological information of the unmanned store and historical customer abnormal behavior records, determining an associated target meteorological factor of the customer abnormal behavior from the plurality of target meteorological factors according to the occurrence influence coefficient of each target meteorological factor on the customer abnormal behavior, determining an associated commodity of the customer abnormal behavior based on historical sales information of the key commodity and historical customer abnormal behavior records of the unmanned store, determining prediction auxiliary information of the customer abnormal behavior based on the associated location, the associated target meteorological factor and the associated commodity of the customer abnormal behavior, and predicting a future occurrence probability of the customer abnormal behavior based on the prediction auxiliary information of the customer abnormal behavior; determining future edge computing resource demand and cloud computing resource demand of the unmanned store based on future customer flow of the unmanned store and future occurrence probability of each customer abnormal behavior; ​ Adjust the allocation of edge computing resources and cloud computing resources to the unmanned store based on future edge computing resource requirements and cloud computing resource requirements of the unmanned store. 4.The machine vision and cloud computing based unmanned store customer behavior detection system according to any one of claims 1-3, wherein, The behavior detection module determines a plurality of target image acquisition devices from the plurality of image acquisition devices based on the customer location information obtained by the personnel positioning unit through the edge computing resources allocated to the unmanned store, including: Determine a key image acquisition device from the plurality of image acquisition devices based on the customer location information obtained by the personnel positioning unit; Determine customer posture features based on the images acquired by the key target image acquisition device; Determine an auxiliary image acquisition device from the plurality of image acquisition devices based on the customer posture features, wherein the plurality of target image acquisition devices includes the key image acquisition device and the auxiliary image acquisition device. 5.The machine vision and cloud computing based unmanned store customer behavior detection system of claim 3, wherein, The shelf state acquisition unit includes a sound acquisition assembly, a vibration sensing assembly, and a weight sensing assembly, wherein the sound acquisition assembly includes sound acquisition devices arranged at a plurality of positions of the shelf, the vibration sensing assembly includes vibration sensing devices arranged at a plurality of positions of the shelf, and the weight sensing assembly includes weight sensing devices arranged at the commodity placement positions. 6.The machine vision and cloud computing based customer behavior detection system of unmanned store according to claim 5, wherein, The behavior detection module determines a candidate behavior detection type based on the shelf state information acquired by the shelf state acquisition unit through the edge computing resources allocated to the unmanned store, including: Determine the associated shelf state factors of each customer abnormal behavior; Determine abnormal shelf state factors based on the shelf state information acquired by the shelf state acquisition unit; Determine the real-time occurrence probability of each customer abnormal behavior based on the abnormal shelf state factors and the associated shelf state factors of each customer abnormal behavior; Determine the candidate behavior detection type based on the real-time occurrence probability and the future occurrence probability of each customer abnormal behavior. 7.The machine vision and cloud computing based unmanned store customer behavior detection system of claim 6, wherein, The behavior detection module extracts customer behavior features corresponding to the candidate behavior detection type from the images acquired by the plurality of target image acquisition devices through the edge computing resources allocated to the unmanned store, including: Establish a feature behavior association graph, wherein the feature behavior association graph is used to record the key image features of each customer abnormal behavior; Use the feature extraction model to extract customer behavior features corresponding to the candidate behavior detection type from the images acquired by the plurality of target image acquisition devices according to the feature behavior association graph through the edge computing resources allocated to the unmanned store. 8.The machine vision and cloud computing based unmanned store customer behavior detection system of claim 7, wherein, The behavior detection module generates a customer behavior detection result based on the customer behavior features corresponding to the candidate behavior detection type at a plurality of time points extracted by the edge computing resources allocated to the unmanned store through the cloud computing resources allocated to the unmanned store, including: Use the behavior analysis model to generate multi-frame behavior time series features of the customer based on the customer behavior features corresponding to the candidate behavior detection type at a plurality of time points extracted by the edge computing resources allocated to the unmanned store through the cloud computing resources allocated to the unmanned store; Generate the customer behavior detection result based on the multi-frame behavior time series features of the customer.

Images