Artificial intelligence-based store retail management system and method

Abstract

The application relates to the field of intelligent management, and specifically discloses a store retail management system and method based on artificial intelligence, which collects a regional heat map in a store through an infrared camera, acquires sales data of products, and respectively performs image feature analysis and context encoding on the regional heat map in the store and the sales data of the products by using image analysis and natural language processing technologies based on deep learning, so as to intelligently obtain a judgment result of whether the layout of the products needs to be adjusted. In this way, the system can help understand the attention and interest points of customers to different products, automatically analyze data and adjust the layout of the products, reduce the need for manual decision-making, help the store adapt to market changes more quickly, enhance the competitive advantage, and improve the operation efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent management, and more specifically, to a store retail management system and method based on artificial intelligence. BACKGROUND

[0002] The retail industry is a highly competitive industry, and stores need to continuously optimize management and operations to improve competitiveness and market share. Moreover, consumer demand and shopping habits are constantly changing, and stores need to adjust product mix, pricing strategy, and service quality according to consumer demand.

[0003] However, traditional methods often rely on the subjective experience and intuition of store managers, resulting in inaccurate or inappropriate decisions. Secondly, the store layout in traditional methods is often static, with a long adjustment period, and cannot adapt to market changes and customer demand in a timely manner.

[0004] Therefore, a store retail management system based on artificial intelligence is expected. SUMMARY

[0005] To solve the above technical problems, the present application is proposed. The embodiments of the present application provide a store retail management system and method based on artificial intelligence, which collects the area heat map in the store through an infrared camera and obtains the sales data of the products, and uses deep learning-based image analysis and natural language processing technology to analyze the image features and context encoding of the area heat map and the sales data of the products in the store, respectively, to intelligently obtain the judgment result of whether the layout of the goods needs to be adjusted. In this way, the system can help understand the customer's attention and interest points for different products, automatically analyze the data and adjust the layout of the goods, reducing the need for manual decision-making, while helping the store to adapt to market changes more quickly, enhancing the competitive advantage, and improving operational efficiency.

[0006] According to one aspect of the present application, a store retail management system based on artificial intelligence is provided, comprising:

[0007] a store area heat map collection module for collecting the area heat map in the store through an infrared camera;

[0008] a product sales data acquisition module for acquiring sales data of products;

[0009] a target saliency capture module for performing area heat map saliency capture on the area heat map in the store to obtain an area saliency heat map;

[0010] a heat map area saliency feature extraction module for sequentially performing heat map feature extraction and semantic understanding on the area saliency heat map to obtain a heat map area saliency semantic understanding feature vector;

[0011] a sales data semantic understanding module configured to perform sales data semantic understanding on the sales data of the product to obtain a product sales data semantic understanding feature vector;

[0012] a feature fusion module configured to fuse the product sales data semantic understanding feature vector and the heat map region saliency semantic understanding feature vector to obtain a store retail overall situation semantic representation vector;

[0013] a judgment result generation module configured to obtain a judgment result based on the store retail overall situation semantic representation vector.

[0014] According to another aspect of the present application, a store retail management method based on artificial intelligence is provided, which comprises:

[0015] capturing a regional heat map in the store by an infrared camera;

[0016] obtaining sales data of a product;

[0017] performing regional heat map saliency capture on the regional heat map in the store to obtain a region saliency heat map;

[0018] performing heat map feature extraction and semantic understanding on the region saliency heat map in sequence to obtain a heat map region saliency semantic understanding feature vector;

[0019] performing sales data semantic understanding on the sales data of the product to obtain a product sales data semantic understanding feature vector;

[0020] fusing the product sales data semantic understanding feature vector and the heat map region saliency semantic understanding feature vector to obtain a store retail overall situation semantic representation vector;

[0021] a judgment result generation module configured to obtain a judgment result based on the store retail overall situation semantic representation vector.

[0022] Compared with the prior art, the store retail management system and method based on artificial intelligence provided by the present application can capture a regional heat map in the store by an infrared camera and obtain sales data of a product, and can use deep learning-based image analysis and natural language processing technology to perform image feature analysis and context encoding on the regional heat map in the store and the sales data of the product, respectively, so as to intelligently obtain a judgment result of whether the layout of the product needs to be adjusted. In this way, the system can help understand the attention and interest points of customers to different products, automatically analyze the data and adjust the layout of the product, reduce the need for manual decision-making, and help the store adapt to market changes more quickly, enhance the competitive advantage, and improve the operation efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0023] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description thereof taken in conjunction with the accompanying drawings, in which: The accompanying drawings provide exemplary embodiments of the application and serve as an aid in understanding the application. They constitute a part of this specification and include exemplary embodiments of the application, together with the description, to explain the principles of the application. They are not intended to limit the application, but to provide illustrative examples thereof. In the drawings, like reference numerals refer to like elements or steps throughout.

[0024] Figure 1 A block diagram of an artificial intelligence-based store retail management system according to an embodiment of the present application.

[0025] Figure 2 An architectural diagram of an artificial intelligence-based store retail management system according to an embodiment of the present application.

[0026] Figure 3 A block diagram of a heat map region saliency feature extraction module of an artificial intelligence-based store retail management system according to an embodiment of the present application.

[0027] Figure 4 A flowchart of an artificial intelligence-based store retail management method according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all, embodiments of the present disclosure are shown. Understanding that these drawings depict only some embodiments of the present disclosure and are not therefore to be considered to be limiting of its scope, the present disclosure will be described with additional specificity and detail through the use of the drawings.

[0029] It should be understood that the various steps of the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present disclosure is not limited in this regard.

[0030] It should be noted that the terms "first", "second", "third", etc. used in the embodiments of the present application are merely to distinguish similar objects, and do not represent a specific order of the objects. Understandably, the "first", "second", "third" can be interchanged in a specific order or sequence as allowed. It should be understood that the objects distinguished by "first", "second", "third" can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0031] As used in this disclosure and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0032] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0033] The retail industry is highly competitive, and stores must continuously optimize their management and operations to enhance their competitiveness and market share. Furthermore, as consumer demands and shopping habits constantly evolve, stores must adjust their product mix, pricing strategies, and service quality to meet these needs.

[0034] However, because traditional methods often rely on the subjective experience and intuition of store managers, decisions can be inaccurate or unresponsive to market changes. Furthermore, traditional store layouts are often static, requiring long adjustment cycles and failing to adapt promptly to market changes and customer demands.

[0035] Therefore, there is a desire for an AI-based store retail management system that uses infrared cameras to capture store area heat maps and obtain product sales data. Deep learning-based image analysis and natural language processing techniques are then used to perform image feature analysis and context encoding on these heat maps and product sales data, respectively, to intelligently determine whether product layout adjustments are necessary. In this way, the system can help understand customers' attention and interests in different products, automatically analyzing data and adjusting product layouts, reducing the need for manual decision-making. This helps stores adapt to market changes more quickly, enhance their competitive advantage, and improve operational efficiency.

[0036] Figure 1 This is a block diagram of an artificial intelligence-based store retail management system according to an embodiment of the present application. Figure 2 FIG is a schematic diagram of the architecture of an artificial intelligence-based store retail management system according to an embodiment of the present application. Figure 1 and Figure 2As shown, the store retail management system 100 based on artificial intelligence according to the embodiment of the application comprises: a store area heat map acquisition module 110, configured to acquire a store area heat map in the store through an infrared camera; a product sales data acquisition module 120, configured to acquire product sales data; a target saliency capturing module 130, configured to perform area heat map saliency capturing on the store area heat map to obtain an area saliency heat map; a heat map area saliency feature extraction module 140, configured to sequentially perform heat map feature extraction and semantic understanding on the area saliency heat map to obtain a heat map area saliency semantic understanding feature vector; a sales data semantic understanding module 150, configured to perform sales data semantic understanding on the product sales data to obtain a product sales data semantic understanding feature vector; a feature fusion module 160, configured to fuse the product sales data semantic understanding feature vector and the heat map area saliency semantic understanding feature vector to obtain a store retail overall situation semantic representation vector; and a judgment result generation module 170, configured to obtain a judgment result based on the store retail overall situation semantic representation vector.

[0037] In the embodiment of the application, the store area heat map acquisition module 110 is configured to acquire a store area heat map in the store through an infrared camera. It can be understood that, considering that the store area heat map is data acquired through an infrared camera, it shows the customer flow and customer distribution in different areas of the store, and such a heat map is usually represented in a color-coded manner, with the depth of color or the change of temperature reflecting the customer density. Based on this, in order to understand the customer flow in each area of the store in detail and adjust the commodity layout according to the customer flow and interest area, in the technical solution of the application, the store area heat map is acquired through an infrared camera, so that the walking path, stay time and hot area of customers in the store can be analyzed to optimize the store layout and product placement, thereby improving sales and customer satisfaction.

[0038] It is worth mentioning that the infrared camera is a device that uses infrared light to form images. It can capture images in low-light environments. Unlike ordinary cameras, this camera usually contains an infrared filter that blocks infrared light from entering the sensor during the day, so that the camera only captures visible light. However, at night or in low light conditions, the filter stops working and allows infrared light to enter the sensor, thereby forming an image through the infrared reflection of objects. Infrared cameras are widely used in security monitoring because they can provide clear images in completely dark environments, which is particularly important for 24-hour monitoring. In addition, infrared cameras are also used in other fields, such as wildlife observation, astronomical observation, and medical diagnosis, etc.

[0039] In the embodiments of the present application, the product sales data acquisition module 120 is configured to acquire sales data of the product. Accordingly, considering that the sales data of the product refers to statistical information related to product sales, these data can provide in-depth understanding of the performance of the product in the market. Specifically, the sales data of the product includes sales volume, sales trend, customer purchase data, etc. Based on this, in the technical solution of the present application, the sales data of the product is acquired and analyzed and processed in combination with the regional heat map in the store, so that the sales strategy can be formulated or adjusted, the product portfolio can be optimized, the sales efficiency can be improved, and the competitiveness can be enhanced.

[0040] In the embodiments of the present application, the target saliency capturing module 130 is configured to perform regional heat map saliency capturing on the regional heat map in the store to obtain a regional saliency heat map. Specifically, in the embodiments of the present application, the target saliency capturing module is configured to input the regional heat map in the store into a Faster R-CNN-based regional heat map saliency capturer to perform target saliency capturing to obtain the regional saliency heat map. Accordingly, considering that the regional heat map in the store contains hot regions and important products in the store, for example, the customer flow in different regions in the store, that is, which regions attract a large number of customers and which regions are relatively few. Therefore, in order to focus on the key salient regions in the regional heat map in the store, facilitate subsequent feature analysis and processing of the salient regions, in the technical solution of the present application, the regional heat map in the store is input into the Faster R-CNN-based regional heat map saliency capturer to perform target saliency capturing to obtain the regional saliency heat map. Specifically, the Faster R-CNN is a popular target detection algorithm that can effectively detect targets in an image and accurately locate them. Therefore, through the Faster R-CNN-based regional heat map saliency capturer, the system can help identify and locate the products or hot regions in different regions in the store, that is, those visually eye-catching regions, so as to capture the salient regions in the regional heat map in the store, which can more focused on important regions and products in the store, so as to more accurately analyze the sales and layout effects, and provide strong decision support for whether to adjust the product layout.

[0041] In the embodiments of the present application, the heat map region saliency feature extraction module 140 is configured to sequentially perform heat map feature extraction and semantic understanding on the regional saliency heat map to obtain a heat map region saliency semantic understanding feature vector. Figure 3 The block diagram of the heat map region saliency feature extraction module in the artificial intelligence-based store retail management system according to the embodiments of the present application. Specifically, in the embodiments of the present application, as shown in Figure 3As shown, the heat map region saliency feature extraction module 140 includes a salient region feature extraction unit 141 configured to input the region saliency heat map into a salient region feature extraction module based on a MobileNets model to obtain a heat map region saliency feature map, and a salient region semantic feature understanding unit 142 configured to input the heat map region saliency feature map into a salient region semantic feature understanding module based on a PVT model to obtain the heat map region saliency semantic understanding feature vector.

[0042] Specifically, the salient region feature extraction unit 141 is configured to input the region saliency heat map into a salient region feature extraction module based on a MobileNets model to obtain a heat map region saliency feature map. It should be understood that, considering that the region saliency heat map reflects the salient regions in the store, in order to extract and mine the salient heat point features about the customer flow and commodity shelf information from the region saliency heat map, in the technical solution of the present application, the region saliency heat map is input into a salient region feature extraction module based on a MobileNets model to obtain a heat map region saliency feature map. Specifically, the MobileNet model is designed for mobile and edge devices, has the characteristics of lightweight, can reduce the consumption of computing resources while maintaining accuracy, and it can effectively extract feature information in images, that is, through the salient region feature extraction module based on a MobileNets model, the salient heat map feature region of the region saliency heat map can be better understood, thereby obtaining a more accurate heat map region saliency feature map, which further helps the store to optimize commodity layout, improve operation efficiency and customer satisfaction.

[0043] It is worth mentioning that the salient region feature extraction module based on a MobileNets model is a deep learning application that uses MobileNet architecture to process and analyze image data, especially to extract key features from region saliency heat maps. Specifically, the MobileNet is a convolutional neural network (CNN) optimized for mobile and embedded devices, which is known for its lightweight and high efficiency while maintaining good performance. In particular, MobileNets aim to reduce model size and speed up inference while maintaining high accuracy. They use depthwise separable convolutions as building blocks, which first apply independently to each input channel, then use a 1x1 pointwise convolution to combine the results. Moreover, MobileNet can learn multi-scale feature representations from low to high levels, which helps to capture different levels of features of salient regions, such as areas with high customer flow or areas of particular customer interest, thereby optimizing store layout and marketing strategies.

[0044] Specifically, the salient region semantic feature understanding unit 142 is configured to input the heat map region saliency feature map into a PVT model-based salient region semantic feature understanding module to obtain the heat map region saliency semantic understanding feature vector. Accordingly, considering that different heat map region saliency features in the heat map region saliency feature map have different scale semantic feature information about hotspot saliency, for example, different region saliency features can reflect the intensity of customers, and high-density regions can indicate particularly popular goods or promotional activities, or customers stay longer in certain regions, which can indicate that they are interested in certain goods or are making decisions. Therefore, in the technical solution of the present application, the heat map region saliency feature map is input into the PVT model-based salient region semantic feature understanding module to obtain the heat map region saliency semantic understanding feature vector. It is worth mentioning that the PVT model combines a vision transformer (Vision Transformer) and a pyramid structure, which can effectively extract features from different scales, and at the same time, uses the attention mechanism in the Transformer architecture, which can effectively capture the correlation information between different regions in the image. That is, through the semantic understanding and multi-scale salient feature capture of the heat map region saliency feature map by the PVT model-based salient region semantic feature understanding module, an in-depth semantic understanding of the salient region can be provided, including the function, purpose or association with customer behavior of the region, and at the same time of feature extraction, important regions and semantic information are focused on, which helps to comprehensively understand the region heat map and product situation in the store.

[0045] In the embodiments of the present application, the sales data semantic understanding module 150 is configured to perform sales data semantic understanding on the sales data of the product to obtain a product sales data semantic understanding feature vector. Specifically, in the embodiments of the present application, the sales data semantic understanding module is configured to input the sales data of the product into an RNN-based sales data semantic understanding model to obtain the product sales data semantic understanding feature vector. Accordingly, it is considered that the sales data of the product contains information about the sales of each product and the product name, etc. Therefore, in order to capture and extract the semantic information of the product from the product sales data, in the technical solutions of the present application, the sales data of the product is input into the RNN-based sales data semantic understanding model to obtain the product sales data semantic understanding feature vector. In detail, in the sales data, information about time series is usually involved, such as the trend of the sales volume changing over time, and the RNN is a neural network model suitable for processing sequence data, which can capture the time-dependent relationship and sequence information in the data, and can retain and utilize the past information when processing sequence data. Therefore, by inputting the sales data of the product into the RNN-based sales data semantic understanding model, the sequence features of the sales data can be better understood, the long-term dependence in the sales data can be captured, and the meaning and trend of the sales data can be more deeply understood, so as to extract the potential patterns and rules in the sales data, and provide strong support for the decision making and sales optimization of the system.

[0046] In the embodiments of the present application, the feature fusion module 160 is configured to fuse the product sales data semantic understanding feature vector and the heat map region saliency semantic understanding feature vector to obtain a store retail overall situation semantic representation vector. Specifically, in the embodiments of the present application, the feature fusion module is configured to: perform vector splicing on the product sales data semantic understanding feature vector and the heat map region saliency semantic understanding feature vector to obtain the store retail overall situation semantic representation vector. It can be understood that, considering that the product sales data semantic understanding feature vector reflects the inherent semantic information and feature representation of the product sales data, and contains information about product sales trends, sales volume changes, product popularity, etc. The heat map region saliency semantic understanding feature vector reflects the saliency features and semantic information of different regions in the store, and contains information about the heat, customer flow, product placement, etc. in each region in the store, helping the system to understand the layout and hot spot regions in the store. Therefore, in order to better fuse the two pieces of information to comprehensively show the retail management situation in the entire store, in the technical solutions of the present application, the product sales data semantic understanding feature vector and the heat map region saliency semantic understanding feature vector are spliced to obtain the store retail overall situation semantic representation vector. That is, the product sales data and the store heat map region information are from different data sources, and respectively express different aspects of product sales and store layout. By splicing the feature vectors of the two aspects, the information of different information sources can be comprehensively utilized to obtain a more comprehensive and integrated store retail overall situation representation, so that the system can more comprehensively capture the features of product sales data and heat map region information, which helps to improve the understanding and analysis ability of the system for the overall situation of the store, and provides more abundant information for optimizing the commodity layout.

[0047] In the embodiments of the present application, the judgment result generation module 170 is configured to obtain a judgment result based on the store retail overall situation semantic representation vector. Specifically, in the embodiments of the present application, the judgment result generation module is configured to: input the store retail overall situation semantic representation vector into a commodity layout adjuster based on a classifier to obtain a judgment result of whether the layout of the commodity needs to be adjusted. That is, by performing classification processing on the store retail overall situation semantic representation vector, a judgment result of whether the layout of the commodity needs to be adjusted is intelligently obtained. In this way, the system can help to understand the customer's attention and interest points for different products, to automatically analyze data and intelligently adjust the layout, reduce the need for manual decision-making, and help the store to adapt to market changes more quickly, enhance the competitive advantage, and improve the operation efficiency.

[0048] In a preferred embodiment, the judgment result generation module includes: a feature significance improvement unit, which is used to perform category probability description significance improvement on the store retail overall situation semantic representation vector based on the classification domain to obtain an optimization factor; a feature weighting unit, which is used to weight the store retail overall situation semantic representation vector with the optimization factor as the weight to obtain an optimized store retail overall situation semantic representation vector; and a layout adjustment unit, which is used to input the optimized store retail overall situation semantic representation vector into a classifier-based product layout adjuster to obtain a judgment result of whether the layout of the product needs to be adjusted.

[0049] Specifically, the feature significance improvement unit is used to perform significance improvement of the semantic representation vector of the overall retail situation of the store based on the category probability description of the classification domain to obtain an optimization factor. In particular, in the technical solution of the present application, the regional heat map and product sales data come from different data sources, representing the customer behavior patterns and sales performance in the store respectively, and have different data characteristics and distributions. Different models such as Faster R-CNN, MobileNets, PVT model and RNN are used to extract features. These models may capture differences in data dimensions and feature weights, resulting in inconsistent feature vector distribution. Specifically, the regional saliency capturer may be based on visual saliency rather than actual customer behavior, which may cause the captured salient areas to deviate from the actual customer interest areas. The salient region semantic feature understanding module and the product sales data semantic understanding model may be based on different semantic understanding frameworks, resulting in differences in the feature vectors after understanding at the semantic level, which in turn leads to poor manifold geometric consistency of the overall feature distribution of the semantic representation vector of the overall store retail situation. In particular, if the feature distribution of the semantic representation vector of the overall store retail situation is inconsistent, the classifier-based product layout adjuster may find it difficult to learn the mapping relationship between stable features and product layout adjustment requirements during training, resulting in a decrease in the performance of the classifier-based product layout adjuster. In addition, the inconsistency of the semantic representation vector of the overall store retail situation may cause the classification results to have high uncertainty in some cases, affecting the decision-making quality of the product layout adjustment. Based on this, the semantic representation vector of the overall store retail situation is significantly improved based on the category probability description of the classification domain.

[0050] More specifically, the feature saliency improvement unit is configured to: multiply the store retail overall situation semantic representation vector and a classification weight matrix of the classifier-based commodity layout adjuster to obtain a store retail overall situation semantic representation weight feature vector; concatenate the store retail overall situation semantic representation vector and the store retail overall situation semantic representation weight feature vector to obtain a store retail overall situation semantic representation-weight information joint vector; pass the store retail overall situation semantic representation-weight information joint vector through a first fully connected layer and then through a sigmoid function to obtain a first activation output value; add the store retail overall situation semantic representation vector and the store retail overall situation semantic representation weight feature vector by position to obtain a store retail overall situation semantic representation-weight information sum vector; pass the store retail overall situation semantic representation-weight information sum vector through a second fully connected layer and then through a sigmoid function to obtain a second activation output value; calculate the mean of the first activation output value and the second activation output value, and subtract 1 from the mean to obtain a first weighting coefficient; calculate the mean of the first activation output value and the second activation output value to obtain a second weighting coefficient; calculate a natural exponential function value by raising the feature values of the store retail overall situation semantic representation weight feature vector to a power to obtain a first exponential store retail overall situation semantic representation weight feature vector; calculate a natural exponential function value by raising the feature values of the store retail overall situation semantic representation vector to a power to obtain a second exponential store retail overall situation semantic representation vector; and based on the first weighting coefficient and the second weighting coefficient, calculate the weighted sum of the first exponential store retail overall situation semantic representation weight feature vector and the second exponential store retail overall situation semantic representation vector to obtain a weighted sum feature vector, and calculate the two-norm of the weighted sum feature vector to obtain the optimization factor.

[0051] In the preferred embodiment described above, specifically, the feature saliency improvement unit is configured to: improve the saliency of the store retail overall situation semantic representation vector based on the category probability description in the classification domain according to a saliency improvement formula to obtain the optimization factor; wherein the saliency improvement formula is:

[0052]

[0053] wherein v c represents the store retail overall situation semantic representation vector, M represents a classification weight matrix of the classifier-based commodity layout adjuster, represents matrix multiplication, wherein represents a position-wise addition, concat represents a concatenation function, W1 represents a first weight matrix of a first fully connected layer, b1 represents a first bias vector of the first fully connected layer, t1 represents a first activation output value, W2 represents a second weight matrix of a second fully connected layer, b2 represents a second bias vector of the second fully connected layer, t2 represents a second activation output value, sigmoid represents a sigmoid function, ||·||2 represents a two-norm of a vector, and a represents an optimization factor.

[0054] That is, in the technical solution of the present application, the manifold geometry consistency of the overall feature distribution of the store retail overall situation semantic representation vector is poor, which leads to poor class label regression normativity of the store retail overall situation semantic representation vector relative to the classifier when it is classified and regressed by the classifier-based commodity layout adjuster, affecting the accuracy of the classification result. Therefore, in the technical solution of the present application, the class probability description of the store retail overall situation semantic representation vector based on the classification domain is significantly improved, which performs auxiliary analysis on the feature attribute layout of the store retail overall situation semantic representation vector by using the classification weight matrix of the classifier-based commodity layout adjuster, and performs backward correlation prediction based on the statistical norm of the difference features in the class high-dimensional space between the auxiliary analysis result and the original store retail overall situation semantic representation vector, so as to improve the description significance of the improved store retail overall situation semantic representation vector on the class probability of the predetermined class of the classifier-based commodity layout adjuster. In this way, the geometric consistency of the store retail overall situation semantic representation vector is maintained during the migration transformation process to the class label regression domain of the classifier-based commodity layout adjuster, which not only helps to reduce the regression deviation in the classification process, but also improves the generalization ability of the classifier-based commodity layout adjuster to new data, and finally realizes more accurate classification results.

[0055] In summary, the artificial intelligence-based store retail management system 100 based on the embodiments of the present application is illustrated, which collects the area heat map in the store by an infrared camera and obtains the sales data of the products, and respectively performs image feature analysis and context encoding on the area heat map in the store and the sales data of the products by using deep learning-based image analysis and natural language processing technology, so as to intelligently obtain the judgment result of whether the layout of the commodities needs to be adjusted. In this way, the system can help to understand the customer's attention and interest points for different products, automatically analyze the data and commodity layout adjustment, reduce the need for manual decision-making, and at the same time help the store to adapt to market changes faster, enhance the competitive advantage, and improve the operation efficiency.

[0056] Figure 4 The flowchart of the artificial intelligence-based store retail management method according to the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the method comprises the following steps: Figure 4As shown, the store retail management method based on artificial intelligence according to the embodiment of the present application comprises: S110, collecting a regional heat map in a store by an infrared camera; S120, obtaining sales data of a product; S130, performing regional heat map saliency capture on the regional heat map in the store to obtain a regional saliency heat map; S140, sequentially performing heat map feature extraction and semantic understanding on the regional saliency heat map to obtain a heat map regional saliency semantic understanding feature vector; S150, performing sales data semantic understanding on the sales data of the product to obtain a product sales data semantic understanding feature vector; S160, fusing the product sales data semantic understanding feature vector and the heat map regional saliency semantic understanding feature vector to obtain a store retail overall situation semantic representation vector; and S170, obtaining a judgment result based on the store retail overall situation semantic representation vector.

[0057] Here, those skilled in the art can understand that the specific operations of each step in the above store retail management method based on artificial intelligence have been described in detail above with reference to the description of the store retail management system based on artificial intelligence of the embodiment of the present application, and therefore, the repeated description thereof will be omitted. Figures 1 to 3

[0058] In the above embodiment, all or part of the above embodiment can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiment can be implemented in the form of a computer program product. The computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiment of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as digital video disc (DVD)) or semiconductor media (such as solid state disk (SSD)) and the like.

[0059] ​Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0060] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0061] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0062] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0063] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit of the technical solutions of the present application.

Claims

1. An artificial intelligence-based store retail management system, characterized by, The method comprises the following steps: a store area thermal map acquisition module is configured to acquire a store area thermal map by using an infrared camera; a product sales data acquisition module is configured to acquire product sales data; a target saliency capturing module is configured to capture the saliency of the store area thermal map to obtain a region saliency thermal map; a thermal map region saliency feature extraction module is configured to sequentially perform thermal map feature extraction and semantic understanding on the region saliency thermal map to obtain a thermal map region saliency semantic understanding feature vector; a sales data semantic understanding module is configured to perform sales data semantic understanding on the product sales data to obtain a product sales data semantic understanding feature vector; a feature fusion module is configured to fuse the product sales data semantic understanding feature vector and the thermal map region saliency semantic understanding feature vector to obtain a store retail overall situation semantic representation vector; a judgment result generation module is configured to obtain a judgment result based on the store retail overall situation semantic representation vector; The judgment result generation module comprises: a feature saliency improvement unit is configured to improve the saliency of the store retail overall situation semantic representation vector based on the category probability description of the classification domain to obtain an optimization factor; a feature weighting unit is configured to weight the store retail overall situation semantic representation vector by using the optimization factor as a weight to obtain an optimized store retail overall situation semantic representation vector; a layout adjustment unit is configured to input the optimized store retail overall situation semantic representation vector into a classifier-based commodity layout adjuster to obtain a judgment result of whether the commodity layout needs to be adjusted; The feature saliency improvement unit is configured to: multiply the store retail overall situation semantic representation vector by a classification weight matrix of the classifier-based commodity layout adjuster to obtain a store retail overall situation semantic representation weight feature vector; concatenate the store retail overall situation semantic representation vector and the store retail overall situation semantic representation weight feature vector to obtain a store retail overall situation semantic representation-weight information joint vector; pass the store retail overall situation semantic representation-weight information joint vector through a first fully connected layer and then through a sigmoid function to obtain a first activation output value; add the store retail overall situation semantic representation vector and the store retail overall situation semantic representation weight feature vector by position to obtain a store retail overall situation semantic representation-weight information sum vector; pass the store retail overall situation semantic representation-weight information sum vector through a second fully connected layer and then through a sigmoid function to obtain a second activation output value; calculate the mean of the first activation output value and the second activation output value, and subtract the mean from 1 to obtain a first weighting coefficient; calculate the mean of the first activation output value and the second activation output value to obtain a second weighting coefficient; calculate the natural exponential function value by raising the feature value of the store retail overall situation semantic representation weight feature vector to the power to obtain a first exponential store retail overall situation semantic representation weight feature vector; calculating a natural exponential function value by raising the eigenvalue of the store retail overall situation semantic representation vector to the power to obtain a second exponential store retail overall situation semantic representation vector; based on the first weighting coefficient and the second weighting coefficient, calculating a weighted sum of the first exponential store retail overall situation semantic representation weight feature vector and the second exponential store retail overall situation semantic representation vector to obtain a weighted sum feature vector, and calculating the two norm of the weighted sum feature vector to obtain the optimization factor. 2.The AI-based store retail management system of claim 1, wherein, The target saliency capture module is used to: input the area heat map in the store into a Faster R-CNN-based area heat map saliency capturer for target saliency capture to obtain the area saliency heat map. 3.The AI-based store retail management system of claim 2, wherein, The heat map area saliency feature extraction module comprises: The salient region feature extraction unit is used to input the area saliency heat map into a MobileNets model-based salient region feature extraction module to obtain a heat map area saliency feature map; The salient region semantic feature understanding unit is used to input the heat map area saliency feature map into a PVT model-based salient region semantic feature understanding module to obtain the heat map area saliency semantic understanding feature vector. 4.The AI-based store retail management system of claim 3, wherein The sales data semantic understanding module is used to: input the product sales data into an RNN-based sales data semantic understanding model to obtain the product sales data semantic understanding feature vector. 5.The AI-based store retail management system of claim 4, wherein, The feature fusion module is used to: vector splicing the product sales data semantic understanding feature vector and the heat map area saliency semantic understanding feature vector to obtain the store retail overall situation semantic representation vector. 6.The AI-based store retail management system of claim 5, wherein, The judgment result generation module is used to: input the store retail overall situation semantic representation vector into a classifier-based commodity layout adjuster to obtain a judgment result of whether the commodity layout needs to be adjusted.

7. An artificial intelligence-based store retail management method of the artificial intelligence-based store retail management system according to any one of claims 1 to 6, characterized by, comprises: acquiring an area heat map in a store through an infrared camera; acquiring product sales data; performing area heat map saliency capture on the area heat map in the store to obtain an area saliency heat map; sequentially performing heat map feature extraction and semantic understanding on the area saliency heat map to obtain a heat map area saliency semantic understanding feature vector; performing sales data semantic understanding on the product sales data to obtain a product sales data semantic understanding feature vector; fusing the product sales data semantic understanding feature vector and the heat map area saliency semantic understanding feature vector to obtain a store retail overall situation semantic representation vector; a judgment result generation module for obtaining a judgment result based on the store retail overall situation semantic representation vector. 8.The AI-based store retail management method of claim 7, wherein, sequentially performing heat map feature extraction and semantic understanding on the area saliency heat map to obtain a heat map area saliency semantic understanding feature vector, comprising: inputting the area saliency heat map into a MobileNets model-based salient region feature extraction module to obtain a heat map area saliency feature map; inputting the heat map area saliency feature map into a PVT model-based salient region semantic feature understanding module to obtain the heat map area saliency semantic understanding feature vector.

Images