An unattended management system based on AI algorithm

Abstract

The present invention discloses an unattended management system based on AI algorithm, and specifically relates to the technical field of unattended management. The present invention provides a commodity AI recognition method, which can quickly and accurately identify and settle commodities selected by users, reduce user waiting time, and to a certain extent avoid the situation where unattended stores are blocked due to users waiting for settlement for too long, thereby increasing the sales volume of commodities and the turnover of stores. A commodity inventory adjustment method is provided, which is in line with sales trends and can reduce the replenishment frequency to a certain extent, reduce the economic losses caused by commodity shortages due to untimely replenishment, and can also avoid large accumulation of commodities to a certain extent. A store business quality evaluation method is provided, which can judge whether the store business quality and unattended management effect within a preset business cycle meet expectations based on the evaluation results, and provide data support for the adjustment and optimization of subsequent management strategies.

Description

Technical Field

[0001] The present invention relates to the field of unmanned management technology, and more specifically, to an unmanned management system based on an AI algorithm. Background Art

[0002] With the maturity and development of AI technology, many traditional manual tasks are gradually being replaced by AI, and the concept of unmanned stores is gradually turning from theory into reality.

[0003] The existing unmanned management system monitors the user's behavior dynamic information in the unmanned store through the camera. After the user selects the product, the QR code of the selected product is scanned in the self-service checkout machine to obtain the product price information. After all the scans are completed, the payment method selection will appear. After the user confirms the payment method, the product fee will be deducted through the payment code presented by the user.

[0004] However, the existing unmanned management system still has some problems: the existing unmanned management system uses a self-service checkout machine to identify and settle selected goods, and a long waiting time is still required when there is a large flow of people in an unmanned store, and the speed of commodity identification and settlement processing still needs to be improved. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an unattended management system based on an AI algorithm to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: an unattended management system based on AI algorithm, comprising:

[0007] User product order creation module: creates a user's product order with the user's mobile phone number as the order name, automatically adds the product name, model, unit price and weight selected by the user through AI recognition or manual recognition to the product order, and numbers the products in the order in which the recognition results are generated;

[0008] Intelligent monitoring module: The high-definition cameras are deployed to monitor the behavior of users who select products by themselves in unattended stores after scanning the code. When users select products, multi-view images of the selected products are collected and automatically transmitted to the AI ​​recognition module;

[0009] AI recognition module: After receiving the multi-view images of the selected products, it processes them to generate a three-dimensional depth map model of the selected products, identifies the selected products based on the set AI recognition algorithm, and forwards the multi-view images of the selected products that fail to be identified to the manual recognition module;

[0010] Manual identification module: For products that AI fails to identify, manual identification is performed based on product videos and the name, model, unit price and weight of the selected products are output to the user product order creation module;

[0011] Automatic inventory update module: automatically updates the remaining inventory quantity information of goods after the commodity transaction is completed, and sends the name of the out-of-stock commodity, the quantity of the out-of-stock commodity to be replenished, and the address of the out-of-stock store to remind the supplier to replenish the goods when the remaining inventory quantity is lower than the preset value;

[0012] Intelligent alarm module: an alarm is issued when an abnormal operation occurs during the user's selection of goods or when the user leaves the store with the goods to be paid before completing the payment and the vertical distance from the store door exceeds the set safety distance;

[0013] Store business data collection module: collects sales data and product sales service data of goods within a preset business cycle and transmits the collected data to the store business data processing module;

[0014] Store business data processing module: processes the collected store business data to obtain the expected inventory of each commodity in the next preset business cycle and the average commodity out-of-stock rate coefficient, AI recognition availability coefficient, AI recognition accuracy coefficient, average user waiting time, inventory accuracy coefficient and automatic alarm accuracy coefficient of the current preset business cycle;

[0015] Store business quality evaluation module: used to calculate the store's business quality index and compare it with the set expected business quality index to determine whether the store's business quality and unattended management effect meet expectations.

[0016] Preferably, the AI ​​recognition module includes a multi-perspective product image receiving unit, an image preprocessing unit, a product 3D model generating unit, a model data acquisition unit, a model matching unit and a recognition result output unit. The multi-perspective product image receiving unit is used to receive the multi-perspective image of the selected product; the image preprocessing unit sequentially performs image correction, image correspondence search and image consistency constraint processing on the received multi-perspective image of the selected product; the product 3D model generating unit inputs the multi-perspective depth image of the selected product obtained after preprocessing into a qualified multi-perspective stereoscopic 3D reconstruction deep learning model to obtain a 3D depth map model of the selected product; the model data acquisition unit respectively collects the outer contour area of ​​each color area in the 3D depth map model of the selected product and the values ​​of each pixel point on the three channels of R, G and B, the product material The outer contour area of ​​each color area in the physical model and the values ​​of each pixel point on the three channels of R, G, and B, as well as the number of color areas in the physical model of the product material; the model matching unit, based on the collected model data, will select the three-dimensional depth map model of the product and the product material physical model in the product material physical model database to perform color area outer contour area matching, color area color matching, and spatial geometry matching in turn to obtain the final matching result and finally calculate the model matching coefficient, and determine the identified target product or recognition failure based on the final matching result and the final calculated model matching coefficient; the recognition result output unit will output the information of the successfully identified target product to the user product order creation module, and the target product information includes the product name, model, unit price and weight, and forward the multi-view images of the selected product that failed to be recognized to the manual recognition module.

[0017] Preferably, the specific steps of performing color region outer contour area matching in the model matching unit are as follows:

[0018] A1. Calculate the maximum matching coefficient αwij between the outer contour area Aci of the i-th color region in the selected product 3D depth map model and the outer contour area of ​​the color region of the j-th product material entity model. The specific formula is:

[0019] , Asj1, Asj2, ..., Asjn are the outer contour areas of the 1st, 2nd, ..., nth color regions of the jth product material entity model respectively;

[0020] A2. Compare the calculated maximum matching coefficient with the set outer contour area matching threshold. If the calculated maximum matching coefficient is greater than or equal to the set threshold, it is marked that the color area of ​​the product material physical model corresponding to the maximum matching coefficient successfully matches the outer contour area of ​​the color area in the selected product 3D depth map model, and retain the product material physical models that successfully match the outer contour area of ​​each color area in the selected product 3D depth map model.

[0021] Preferably, the specific steps of performing color area color matching in the model matching unit are as follows:

[0022] A3. Calculate the average color difference coefficient αyij between the i-th color area in the three-dimensional depth map model of the product material and the i-th color area in the corresponding outer contour area of ​​the retained j-th product material entity model that successfully matches. The specific formula is:

[0023] , Nxij is the number of overlapping pixels between the i-th color area in the j-th commodity material physical model whose outer contour area matches successfully and the i-th color area in the commodity material 3D depth map model, Rcijk, Gcijk, Bcijk, Rsijk, Gsijk, Bsijk are the values ​​of the k-th pixel in the i-th color area in the commodity material 3D depth map model on the R, G, B channels and the values ​​of the k-th pixel in the i-th color area in the j-th commodity material physical model whose outer contour area matches successfully on the R, G, B channels;

[0024] A4. Compare the calculated average color difference coefficient with the set color difference threshold. If the calculated value is less than or equal to the set threshold, it is marked that the color area in the physical model of the product material corresponding to the average color difference coefficient successfully matches the color area in the three-dimensional depth map model of the selected product material, and the physical model of the product material that successfully matches the color of each color area in the three-dimensional depth map model of the selected product is retained.

[0025] Preferably, the specific steps of performing spatial geometric matching in the model matching unit and determining the recognition result based on the matching result are as follows:

[0026] A5. Overlap and ablate the product material 3D depth map model and the jth product material entity model that is retained, and record the volume Vwj of the remaining product material 3D depth map model, the volume Vtj of the remaining product material entity model, the volume Vc of the complete product material 3D depth map model, and the volume Vs of the complete product material entity model after overlapping and ablation. The specific calculation formula of the spatial geometric matching coefficient αphj is: ;

[0027] A6. Compare the calculated spatial geometry matching coefficient with the set spatial geometry matching threshold. If the calculated value is greater than or equal to the set threshold, it is marked that the commodity material entity model corresponding to the spatial geometry matching coefficient successfully matches the spatial geometry of the selected commodity material 3D depth map model, and retain the commodity material entity model that successfully matches the spatial geometry of the selected commodity 3D depth map model.

[0028] A7. Calculate the model matching coefficient αpcj between the jth product material entity model remaining after three rounds of matching and the selected product 3D depth map model. The specific formula is:

[0029] , Ns is the number of color regions of the selected product 3D depth map model;

[0030] A8. If there are multiple product material entity models remaining after three rounds of matching, the product material entity model corresponding to the maximum value of the calculated model matching coefficient is the target product finally identified. If there is only one product material entity model remaining after three rounds of matching, the product material entity model is the target product finally identified. If there are no product material entity models remaining after three rounds of matching, the identification fails.

[0031] Preferably, the sales data of the goods within the preset business cycle collected by the store business data collection module are specifically the total inventory and actual sales volume of each commodity; the collected commodity sales service data within the preset business cycle include the normal sales time and out-of-stock time of each commodity, the number of goods identified by AI during commodity identification, the number of goods identified manually, the number of goods correctly identified by AI during commodity identification, the completion time of user pickup for each commodity order and the issuance time of the order to be paid, the number of correct and incorrect commodity inventory updates, the number of correct alarm issuances and the number of incorrect alarm issuances.

[0032] Preferably, the store business data processing module includes a store business data receiving unit, a commodity sales data processing unit, a commodity inventory adjustment unit, a store business data processing unit and a processed business data transmission unit. The specific process of data processing is as follows:

[0033] Store business data receiving unit: used to receive the data collected by the store business data collection module;

[0034] Commodity sales data processing unit: Calculate the sales coefficient αgi of the i-th commodity based on the total inventory Mci and actual sales volume Mgi of the i-th commodity in the preset business cycle. The specific formula is:

[0035] ;

[0036] Commodity inventory adjustment unit: Calculate the expected inventory Myi of the i-th commodity based on the sales coefficient αgi and the actual sales volume Mgi of the i-th commodity. The specific formula is:

[0037] , apply the calculated expected inventory Myi of the i-th commodity to the next preset business cycle;

[0038] Store business data processing unit: Calculate the average commodity out-of-stock rate coefficient Xqe based on the out-of-stock duration Tqi and the normal selling duration Tsi of the i-th commodity in the preset business cycle. The specific formula is:

[0039] , Na is the number of commodity types within the preset business cycle. The AI ​​recognition availability coefficient Xsb is calculated based on the number of commodities recognized by AI in the preset business cycle, Mpa, and the number of commodities recognized manually, Mpb. The specific formula is:

[0040] , the AI ​​recognition accuracy coefficient Xsz is calculated based on the number of commodities Mpz correctly recognized by AI during the preset business cycle and the number of commodities Mpa recognized by AI. The specific formula is: , based on the user's pickup completion time Twi and the order to be paid Tfi in the i-th product order within the preset business cycle, the average waiting time Tde of the user is calculated. The specific formula is: , Nd is the total number of commodity orders within the preset business cycle, and the inventory accuracy coefficient Xkz is calculated based on the correct number of commodity inventory updates Mka and the incorrect number of commodity inventory updates Mkb within the preset business cycle. The specific formula is: , based on the number of correct alarms Mfa and the number of false alarms Mfb in the preset business cycle, the automatic alarm accuracy coefficient Xbz is calculated. The specific formula is: ;

[0041] Processed business data transmission unit: outputs the calculated average commodity out-of-stock rate coefficient, AI recognition availability coefficient, AI recognition accuracy coefficient, user average waiting time, inventory accuracy coefficient and automatic alarm accuracy coefficient to the store business quality evaluation module.

[0042] Preferably, the store business quality evaluation module includes a data receiving unit, a data secondary processing unit, a business quality index calculation unit and a business quality evaluation unit. The data receiving unit is used to receive various parameters transmitted by the store business data processing module; the data secondary processing unit calculates the AI ​​recognition application reliability coefficient Ysl based on the AI ​​recognition availability coefficient Xsb and the AI ​​recognition accuracy coefficient Xsz. The specific formula is: , based on the inventory accuracy coefficient Xkz and the automatic alarm accuracy coefficient Xbz, the automation processing reliability coefficient Ycl is calculated. The specific formula is:

[0043] The specific formula for calculating the business quality index ZC by the business quality index calculation unit based on the average commodity out-of-stock rate coefficient Xqe, the average waiting time of users Tde, the AI ​​recognition application reliability coefficient Ysl, and the automation processing reliability coefficient Ycl is: , e is a natural constant; the business quality evaluation unit compares the calculated business quality index with the set expected business quality index. If the calculated value is greater than or equal to the set value, the business quality and unattended management effect of the store within the preset business cycle are in line with expectations; if the calculated value is less than the set value, the business quality and unattended management effect of the store within the preset business cycle are not in line with expectations.

[0044] Technical effects and advantages of the present invention:

[0045] The present invention sets an AI recognition module to receive a multi-view image of a selected commodity, processes the image to generate a three-dimensional depth map model of the selected commodity, identifies the selected commodity based on a set AI recognition algorithm, and forwards the multi-view image of the selected commodity that fails to be identified to manual recognition, thereby providing a commodity AI recognition method, which can quickly and accurately identify and settle the commodity selected by the user, reduce the waiting time of the user, and to a certain extent avoid the situation where the unattended store is blocked due to the long waiting time for the settlement of the user, and can increase the in-and-out flow of users of the unattended store, and increase the sales volume of the commodity and the turnover of the store; the store business data collection module, the store business data processing module, and the store business quality evaluation module are set to provide a commodity inventory adjustment method on the one hand, which is in line with the sales trend and can reduce the replenishment frequency to a certain extent, reduce the economic loss of commodity shortage caused by untimely replenishment, and can also avoid a large amount of commodity accumulation to a certain extent, and on the other hand, provide a store business quality evaluation method, which can judge whether the store business quality and the unattended management effect within a preset business cycle meet expectations based on the evaluation results, and provide data support for the adjustment and optimization of subsequent management strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a system structure block diagram of the present invention.

[0047] Figure 2 This is a flowchart of the commodity AI recognition of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] like Figure 1 The embodiment shown provides an unattended management system based on AI algorithm, including a user identity recognition module, a user product order creation module, an intelligent monitoring module, an AI recognition module, a manual recognition module, a cost accounting module, an inventory automatic update module, an intelligent alarm module, a store business data collection module, a store business data processing module, a store business quality evaluation module and a database. The user identity recognition module is connected to the user product order creation module, the intelligent monitoring module is connected to the AI ​​recognition module and the intelligent alarm module, the AI ​​recognition module is connected to the manual recognition module and the user product order creation module, the manual recognition module is connected to the user product order creation module, the user product order creation module is connected to the cost accounting module, the cost accounting module is connected to the inventory automatic update module and the intelligent alarm module, the user product order creation module, the AI ​​recognition module, the manual recognition module, the inventory automatic update module, the intelligent alarm module are connected to the store business data collection module, the store business data collection module, the store business data processing module and the store business quality evaluation module are connected in sequence, and all modules in the system are connected to the database.

[0050] The user identity recognition module backs up the user identity information after the user scans the QR code attached in the unattended store and is authorized, collects the user's facial information to review the user identity information, and if the review result is the same user, jumps to the user product order creation module.

[0051] Furthermore, the user identity recognition module includes a user identity information acquisition unit, a user identity information review unit, an information notification unit and a user identity information clearing unit. The user identity information acquisition unit acquires the user identity information through a QR code scanned by the user; the user identity information review unit collects the user's facial information and reviews the acquired user identity information based on the collected user's facial information. If the review result belongs to the same user, the review is passed, otherwise the review is failed; the information notification unit sends a user identity authentication passed prompt to the user product order creation module when the review is passed, and sends an identity authentication failed reminder to the user when the review is failed; the identity information clearing unit clears the backed up user identity information after the user order is completed.

[0052] What needs to be specifically explained in this embodiment is that when the user scans the code and backs up the user identity information after authorization, the backed up user identity information includes the user ID card account, the user ID card photo, the payment software and bank card account bound to the user ID card, and the mobile phone number bound to the payment software and bank card.

[0053] The user product order creation module creates a product order for the user using the user's mobile phone number as the order name, automatically adds the product name, model, unit price and weight selected by the user through AI recognition or manual recognition to the product order, and numbers the products in the order in which the recognition results are generated.

[0054] The intelligent monitoring module monitors the behavior of users who select goods by themselves in unattended stores after scanning the code through the deployed high-definition cameras, and collects multi-view images of the selected goods when the users select the goods and automatically transmits them to the AI ​​recognition module;

[0055] After receiving the multi-view images of the selected product, the AI ​​recognition module processes them to generate a three-dimensional depth map model of the selected product, identifies the selected product based on the set AI recognition algorithm, and forwards the multi-view images of the selected product that fail to be identified to the manual recognition module.

[0056] Furthermore, the AI ​​recognition module includes a multi-perspective product image receiving unit, an image preprocessing unit, a product three-dimensional model generating unit, a model data acquisition unit, a model matching unit and a recognition result output unit. The multi-perspective product image receiving unit is used to receive the multi-perspective image of the selected product; the image preprocessing unit sequentially performs image correction, image correspondence search and image consistency constraint processing on the received multi-perspective image of the selected product; the product three-dimensional model generating unit inputs the multi-perspective depth image of the selected product obtained after preprocessing into a qualified multi-perspective stereoscopic three-dimensional reconstruction deep learning model to obtain a three-dimensional depth map model of the selected product; the model data acquisition unit respectively collects the outer contour area of ​​each color area in the three-dimensional depth map model of the selected product and the values ​​of each pixel point on the three channels of R, G and B, the product material The outer contour area of ​​each color area in the physical model and the values ​​of each pixel point on the three channels of R, G, and B, as well as the number of color areas in the physical model of the product material; the model matching unit, based on the collected model data, will select the three-dimensional depth map model of the product and the product material physical model in the product material physical model database to perform color area outer contour area matching, color area color matching, and spatial geometry matching in turn to obtain the final matching result and finally calculate the model matching coefficient, and determine the identified target product or recognition failure based on the final matching result and the final calculated model matching coefficient; the recognition result output unit will output the information of the successfully identified target product to the user product order creation module, and the target product information includes the product name, model, unit price and weight, and forward the multi-view images of the selected product that failed to be recognized to the manual recognition module.

[0057] What needs to be specifically explained in this embodiment is that the main purpose of image correction for the multi-view images of commodity materials is to allow the epipolar lines corresponding to the pixels to be above the horizontal scanning lines passing through the pixels, thereby determining the baseline. Image correction is a prior art and will not be specifically elaborated here. After image correction, it is necessary to traverse all pixels on the epipolar lines of one image to search for the best pixels that can match the pixels on another image, and then establish a corresponding relationship based on this. By establishing a corresponding relationship in this way, the original search on the entire image can be changed to a search on the epipolar lines, that is, the search process is changed from the original two-dimensional search to a one-dimensional search, and the processing efficiency is significantly improved. When performing pixel matching, the luminosity can be used The consistency measurement method is used to measure the similarity between pixels, so as to use this as a basis for judging whether the two pixels are the same point in the real world. When pixels are directly used for photometric consistency matching or window-based matching, after comparing the results of the algorithm output with the results of the benchmark, it can be found that the depth map output in this way is not smooth. Consistency constraints can be imposed to make the algorithm obtain a smooth depth map. Consistency constraints include uniqueness constraints, order constraints, and smoothness constraints. The uniqueness constraint can ensure that any pixel point on a given image has at most one matching point in another image. The order constraint can ensure that the selected corresponding points maintain the same order in the two images. The smoothness constraint ensures that the similarity measure between adjacent pixels should change slowly.

[0058] What needs to be specifically explained in this embodiment is that the construction, training and optimization process of the multi-view three-dimensional reconstruction deep learning model is an existing technology and will not be elaborated here.

[0059] Specifically, in this embodiment, a product material physical model is created for each product material stored in the unattended store and stored in the product material physical model database. When the product type is updated, the product material physical model database will also update the product material physical model accordingly.

[0060] Furthermore, the specific steps of matching the outer contour area of ​​the color region in the model matching unit are as follows:

[0061] A1. Calculate the maximum matching coefficient αwij between the outer contour area Aci of the i-th color region in the selected product 3D depth map model and the outer contour area of ​​the color region of the j-th product material entity model. The specific formula is: , Asj1, Asj2, ..., Asjn are the outer contour areas of the 1st, 2nd, ..., nth color regions of the jth product material entity model respectively;

[0062] A2. Compare the calculated maximum matching coefficient with the set outer contour area matching threshold. If the calculated maximum matching coefficient is greater than or equal to the set threshold, it is marked that the color area of ​​the product material physical model corresponding to the maximum matching coefficient successfully matches the outer contour area of ​​the color area in the selected product 3D depth map model, and retain the product material physical models that successfully match the outer contour area of ​​each color area in the selected product 3D depth map model.

[0063] Furthermore, the specific steps of performing color matching of color regions in the model matching unit are as follows:

[0064] A3. Calculate the average color difference coefficient αyij between the i-th color area in the three-dimensional depth map model of the product material and the i-th color area in the corresponding outer contour area of ​​the retained j-th product material entity model that successfully matches. The specific formula is: , Nxij is the number of overlapping pixels between the i-th color area in the j-th commodity material physical model whose outer contour area matches successfully and the i-th color area in the commodity material 3D depth map model, Rcijk, Gcijk, Bcijk, Rsijk, Gsijk, Bsijk are the values ​​of the k-th pixel in the i-th color area in the commodity material 3D depth map model on the R, G, B channels and the values ​​of the k-th pixel in the i-th color area in the j-th commodity material physical model whose outer contour area matches successfully on the R, G, B channels;

[0065] A4. Compare the calculated average color difference coefficient with the set color difference threshold. If the calculated value is less than or equal to the set threshold, it is marked that the color area in the physical model of the product material corresponding to the average color difference coefficient successfully matches the color area in the three-dimensional depth map model of the selected product material, and the physical model of the product material that successfully matches the color of each color area in the three-dimensional depth map model of the selected product is retained.

[0066] Furthermore, the specific steps of performing spatial geometric matching in the model matching unit and determining the recognition result based on the matching result are as follows:

[0067] A5. Overlap and ablate the product material 3D depth map model and the jth product material entity model that is retained, and record the volume Vwj of the remaining product material 3D depth map model, the volume Vtj of the remaining product material entity model, the volume Vc of the complete product material 3D depth map model, and the volume Vs of the complete product material entity model after overlapping and ablation. The specific calculation formula of the spatial geometric matching coefficient αphj is: ;

[0068] A6. Compare the calculated spatial geometry matching coefficient with the set spatial geometry matching threshold. If the calculated value is greater than or equal to the set threshold, it is marked that the commodity material entity model corresponding to the spatial geometry matching coefficient successfully matches the spatial geometry of the selected commodity material 3D depth map model, and retain the commodity material entity model that successfully matches the spatial geometry of the selected commodity 3D depth map model.

[0069] A7. Calculate the model matching coefficient αpcj between the jth product material entity model remaining after three rounds of matching and the selected product 3D depth map model. The specific formula is: , Ns is the number of color regions of the selected product 3D depth map model;

[0070] A8. If there are multiple product material entity models remaining after three rounds of matching, the product material entity model corresponding to the maximum value of the calculated model matching coefficient is the target product finally identified. If there is only one product material entity model remaining after three rounds of matching, the product material entity model is the target product finally identified. If there are no product material entity models remaining after three rounds of matching, the identification fails.

[0071] The manual identification module performs manual identification on the products that AI fails to identify based on the product video and outputs the name, model, unit price and weight of the selected products to the user product order creation module.

[0072] The fee calculation module automatically calculates the fee for the user's product order after the user completes product selection and generates an electronic ticket and sends it to the user's mobile phone. After the user confirms that it is correct, click OK and select a payment method to complete the deduction.

[0073] The automatic inventory update module automatically updates the remaining inventory quantity information of the goods after the commodity transaction is completed, and sends the out-of-stock commodity name, the number of out-of-stock commodities to be replenished, and the out-of-stock store address to the supplier to remind the supplier to replenish the goods when the remaining inventory quantity is lower than the preset value.

[0074] The intelligent alarm module issues an alarm when an abnormal operation occurs during the user's selection of goods or when the user leaves the store with the goods to be paid without completing the payment and the vertical distance from the store door exceeds the set safety distance.

[0075] The store business data collection module collects the sales data and the commodity selling service data of the commodity within a preset business cycle and transmits the collected data to the store business data processing module.

[0076] Furthermore, the sales data of goods within the preset business cycle collected by the store business data collection module specifically include the total inventory and actual sales volume of each product; the collected product sales service data within the preset business cycle include the normal sales time and out-of-stock time of each product, the number of goods identified by AI during product identification, the number of goods identified manually, the number of goods correctly identified by AI during product identification, the completion time of user pickup for each product order and the issuance time of the order to be paid, the number of correct and incorrect product inventory updates, the number of correct alarm issuances, and the number of incorrect alarm issuances.

[0077] In this embodiment, it should be specifically explained that the user pick-up completion time refers to the time when the last item in the user's product order is added, and the payment order issuance time refers to the time when the user receives the order electronic ticket.

[0078] The store business data processing module processes the collected store business data to obtain the expected inventory of each commodity in the next preset business cycle and the average commodity out-of-stock rate coefficient, AI recognition availability coefficient, AI recognition accuracy coefficient, user average waiting time, inventory accuracy coefficient and automatic alarm accuracy coefficient of the current preset business cycle.

[0079] Furthermore, the store business data processing module includes a store business data receiving unit, a commodity sales data processing unit, a commodity inventory adjustment unit, a store business data processing unit and a processed business data transmission unit. The specific process of data processing is as follows:

[0080] Store business data receiving unit: used to receive the data collected by the store business data collection module;

[0081] Commodity sales data processing unit: Calculate the sales coefficient αgi of the i-th commodity based on the total inventory Mci and actual sales volume Mgi of the i-th commodity in the preset business cycle. The specific formula is: ;

[0082] Commodity inventory adjustment unit: Calculate the expected inventory Myi of the i-th commodity based on the sales coefficient αgi and the actual sales volume Mgi of the i-th commodity. The specific formula is: , apply the calculated expected inventory Myi of the i-th commodity to the next preset business cycle;

[0083] What needs to be specifically explained in this embodiment is that each time the commodity sales data within a preset business cycle is processed, a new expected inventory of each category of commodities will be generated and applied to the next preset business cycle. This not only conforms to the sales trend but also can reduce the replenishment frequency to a certain extent, reduce the economic losses caused by commodity shortages due to untimely replenishment, and can also avoid large accumulation of commodities to a certain extent.

[0084] Store business data processing unit: Calculate the average commodity out-of-stock rate coefficient Xqe based on the out-of-stock duration Tqi and the normal selling duration Tsi of the i-th commodity in the preset business cycle. The specific formula is: , Na is the number of commodity types within the preset business cycle. The AI ​​recognition availability coefficient Xsb is calculated based on the number of commodities recognized by AI in the preset business cycle, Mpa, and the number of commodities recognized manually, Mpb. The specific formula is: , the AI ​​recognition accuracy coefficient Xsz is calculated based on the number of commodities Mpz correctly recognized by AI during the preset business cycle and the number of commodities Mpa recognized by AI. The specific formula is: , based on the user's pickup completion time Twi and the order to be paid Tfi in the i-th product order within the preset business cycle, the average waiting time Tde of the user is calculated. The specific formula is: , Nd is the total number of commodity orders within the preset business cycle, and the inventory accuracy coefficient Xkz is calculated based on the correct number of commodity inventory updates Mka and the incorrect number of commodity inventory updates Mkb within the preset business cycle. The specific formula is: , based on the number of correct alarms Mfa and the number of false alarms Mfb in the preset business cycle, the automatic alarm accuracy coefficient Xbz is calculated. The specific formula is: ;

[0085] Processed business data transmission unit: outputs the calculated average commodity out-of-stock rate coefficient, AI recognition availability coefficient, AI recognition accuracy coefficient, user average waiting time, inventory accuracy coefficient and automatic alarm accuracy coefficient to the store business quality evaluation module.

[0086] The store business quality evaluation module is used to calculate the store's business quality index and compare it with the set expected business quality index to determine whether the store's business quality and unattended management effect meet expectations.

[0087] Further, the store business quality evaluation module includes a data receiving unit, a data secondary processing unit, a business quality index calculation unit and a business quality evaluation unit. The data receiving unit is used to receive various parameters transmitted by the store business data processing module; the data secondary processing unit calculates the AI ​​recognition application reliability coefficient Ysl based on the AI ​​recognition availability coefficient Xsb and the AI ​​recognition accuracy coefficient Xsz. The specific formula is: , based on the inventory accuracy coefficient Xkz and the automatic alarm accuracy coefficient Xbz, the automation processing reliability coefficient Ycl is calculated. The specific formula is: The specific formula for calculating the business quality index ZC by the business quality index calculation unit based on the average commodity out-of-stock rate coefficient Xqe, the average waiting time of users Tde, the AI ​​recognition application reliability coefficient Ysl, and the automation processing reliability coefficient Ycl is: , e is a natural constant; the business quality evaluation unit compares the calculated business quality index with the set expected business quality index. If the calculated value is greater than or equal to the set value, the business quality and unattended management effect of the store within the preset business cycle are in line with expectations; if the calculated value is less than the set value, the business quality and unattended management effect of the store within the preset business cycle are not in line with expectations.

[0088] The database is used to store data information of all modules in the system.

[0089] It should be specifically noted in this embodiment that the set values ​​used are selected based on actual needs and are not limited to specific values ​​here.

[0090] This embodiment provides an unattended management method based on an AI algorithm, comprising the following steps:

[0091] S1: After the user scans the QR code attached in the unattended store and passes the authorization, the user's identity information is backed up, and the user's face information is collected to verify the user's identity information. If the verification result is the same user, the user is redirected to the user's product order;

[0092] S2: Create a product order for the user with the user's mobile phone number as the order name, automatically add the product name, model, unit price and weight selected by the user through AI recognition or manual recognition to the product order, and use the order in which the recognition results are generated as the order in which the products are arranged and numbered;

[0093] S3: HD cameras are deployed to monitor the user's behavior of selecting products in unmanned stores after scanning the code, and multi-view images of the selected products are collected when the user selects the products;

[0094] S4: Process the multi-view images of the selected product to generate a three-dimensional depth map model of the selected product, and identify the selected product based on a set AI recognition algorithm;

[0095] S5: For products that AI fails to identify, manual identification is performed based on the product video and the name, model, unit price and weight of the selected product are output to the user's product order;

[0096] S6: After the user has finished selecting the product, the user's product order is automatically calculated and an electronic ticket is generated and sent to the user's mobile phone. After the user confirms that it is correct, click OK and select the payment method to complete the deduction;

[0097] S7: After the commodity transaction is completed, the remaining commodity inventory quantity information is automatically updated, and when the remaining commodity inventory quantity is lower than the preset value, the out-of-stock commodity name, the out-of-stock commodity quantity to be replenished, and the out-of-stock store address are sent to the supplier to remind the supplier to replenish the goods;

[0098] S8: An alarm is issued when an abnormal operation occurs during the user's selection of goods or when the user leaves the store with the goods to be paid before completing the payment and the vertical distance from the store door exceeds the set safety distance;

[0099] S9: Collect the total inventory and actual sales volume of each product within the preset business cycle, the normal sales time and out-of-stock time of each product, the number of products identified by AI during product identification, the number of products identified manually, the number of products correctly identified by AI during product identification, the time when the user picks up each product order and the time when the order to be paid is issued, the number of correct and incorrect product inventory updates, the number of correct alarm issuances, and the number of incorrect alarm issuances;

[0100] S10: Processing the collected store business data to obtain the expected inventory of each commodity in the next preset business cycle and the average commodity out-of-stock rate coefficient, AI recognition availability coefficient, AI recognition accuracy coefficient, average user waiting time, inventory accuracy coefficient and automatic alarm accuracy coefficient of the current preset business cycle, calculating the AI ​​recognition application reliability coefficient based on the AI ​​recognition availability coefficient and the AI ​​recognition accuracy coefficient, calculating the automation processing reliability coefficient based on the inventory accuracy coefficient and the automatic alarm accuracy coefficient, and calculating the business quality index based on the average commodity out-of-stock rate coefficient, the average user waiting time, the AI ​​recognition application reliability coefficient and the automation processing reliability coefficient;

[0101] S11: Compare the calculated business quality index with the set expected business quality index. If the calculated value is greater than or equal to the set value, the store's business quality and unmanned management effect within the preset business cycle are in line with expectations. If the calculated value is less than the set value, the store's business quality and unmanned management effect within the preset business cycle are not in line with expectations.

[0102] like Figure 2 This embodiment provides a commodity AI recognition flow chart.

[0103] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An unattended management system based on AI algorithm, characterized by: include: User product order creation module: creates a user's product order with the user's mobile phone number as the order name, automatically adds the product name, model, unit price and weight selected by the user through AI recognition or manual recognition to the product order, and numbers the products in the order in which the recognition results are generated; Intelligent monitoring module: The high-definition cameras are deployed to monitor the behavior of users who select products by themselves in unattended stores after scanning the code. When users select products, multi-view images of the selected products are collected and automatically transmitted to the AI ​​recognition module; AI recognition module: After receiving the multi-view images of the selected products, it processes them to generate a three-dimensional depth map model of the selected products, identifies the selected products based on the set AI recognition algorithm, and forwards the multi-view images of the selected products that fail to be identified to the manual recognition module; The AI ​​recognition module includes a multi-view commodity image receiving unit, an image preprocessing unit, a commodity three-dimensional model generating unit, a model data collecting unit, a model matching unit and a recognition result output unit. The multi-view commodity image receiving unit is used to receive the multi-view images of the selected commodity; the image preprocessing unit sequentially performs image correction, image correspondence search and image consistency constraint processing on the received multi-view images of the selected commodity; The product 3D model generation unit inputs the preprocessed multi-view depth image of the selected product into a qualified multi-view stereoscopic 3D reconstruction deep learning model to obtain a 3D depth map model of the selected product; the model data acquisition unit respectively acquires the outer contour area of ​​each color region in the 3D depth map model of the selected product and the value of each pixel point on the three channels of R, G, and B, the outer contour area of ​​each color region in the product material physical model and the value of each pixel point on the three channels of R, G, and B, and the number of color regions in the product material physical model; The model matching unit performs color region outer contour area matching, color region color matching, and spatial geometry matching on the selected product 3D depth map model and the product material physical model in the product material physical model database based on the collected model data, and obtains the final matching result and finally calculates the model matching coefficient, and determines the identified target product or the identification failure based on the final matching result and the final calculated model matching coefficient; The recognition result output unit outputs the target product information that has been successfully recognized to the user product order creation module, and the target product information includes product name, model, unit price and weight, and forwards the multi-view images of the selected products that have failed to be recognized to the manual recognition module; The specific steps of performing color region outer contour area matching in the model matching unit are as follows: A1. Calculate the maximum matching coefficient αwij between the outer contour area Aci of the i-th color region in the selected product 3D depth map model and the outer contour area of ​​the color region of the j-th product material entity model. The specific formula is: Asj1, Asj2, ..., Asjn are the outer contour areas of the 1st, 2nd, ..., nth color regions of the jth product material entity model respectively; A2. Compare the calculated maximum matching coefficient with the set outer contour area matching threshold. If the calculated maximum matching coefficient is greater than or equal to the set threshold, it is marked that the color area of ​​the product material physical model corresponding to the maximum matching coefficient successfully matches the outer contour area of ​​the color area in the selected product 3D depth map model, and retain the product material physical model that successfully matches the outer contour area of ​​each color area in the selected product 3D depth map model; The specific steps of performing color matching of color regions in the model matching unit are as follows: A3. Calculate the average color difference coefficient αyij between the i-th color area in the three-dimensional depth map model of the product material and the i-th color area in the corresponding outer contour area of ​​the retained j-th product material entity model that successfully matches. The specific formula is: Nxij is the number of overlapping pixels between the i-th color area in the j-th product material physical model whose outer contour area matches successfully and the i-th color area in the product material 3D depth map model; Rcijk, Gcijk, Bcijk, Rsijk, Gsijk, and Bsijk are the values ​​of the k-th pixel in the i-th color area in the product material 3D depth map model on the R, G, and B channels and the values ​​of the k-th pixel in the i-th color area in the j-th product material physical model whose outer contour area matches successfully on the R, G, and B channels; A4. Compare the calculated average color difference coefficient with the set color difference threshold. If the calculated value is less than or equal to the set threshold, it is marked that the color area in the physical model of the product material corresponding to the average color difference coefficient successfully matches the color area in the three-dimensional depth map model of the selected product material, and the physical model of the product material that successfully matches the color of each color area in the three-dimensional depth map model of the selected product is retained; The specific steps of performing spatial geometric matching in the model matching unit and determining the recognition result based on the matching result are as follows: A5. Overlap and ablate the product material 3D depth map model and the jth product material entity model that is retained, and record the volume Vwj of the remaining product material 3D depth map model, the volume Vtj of the remaining product material entity model, the volume Vc of the complete product material 3D depth map model, and the volume Vs of the complete product material entity model after overlapping and ablation. The specific calculation formula of the spatial geometric matching coefficient αphj is: A6. Compare the calculated spatial geometry matching coefficient with the set spatial geometry matching threshold. If the calculated value is greater than or equal to the set threshold, it is marked that the commodity material entity model corresponding to the spatial geometry matching coefficient successfully matches the spatial geometry of the selected commodity material 3D depth map model, and retain the commodity material entity model that successfully matches the spatial geometry of the selected commodity 3D depth map model. A7. Calculate the model matching coefficient αpcj between the jth product material entity model remaining after three rounds of matching and the selected product 3D depth map model. The specific formula is: Ns is the number of color regions of the selected product 3D depth map model; A8. If there are multiple product material entity models remaining after three rounds of matching, the product material entity model corresponding to the maximum value of the calculated model matching coefficient is the target product finally identified. If there is only one product material entity model remaining after three rounds of matching, the product material entity model is the target product finally identified. If there are no product material entity models remaining after three rounds of matching, the identification fails. Manual identification module: For products that AI fails to identify, manual identification is performed based on product videos and the name, model, unit price and weight of the selected products are output to the user product order creation module; Automatic inventory update module: automatically updates the remaining inventory quantity information of goods after the commodity transaction is completed, and sends the name of the out-of-stock commodity, the quantity of the out-of-stock commodity to be replenished, and the address of the out-of-stock store to remind the supplier to replenish the goods when the remaining inventory quantity is lower than the preset value; Intelligent alarm module: an alarm is issued when an abnormal operation occurs during the user's selection of goods or when the user leaves the store with the goods to be paid before completing the payment and the vertical distance from the store door exceeds the set safety distance; Store business data collection module: collects sales data and product sales service data of goods within a preset business cycle and transmits the collected data to the store business data processing module; The commodity sales service data collected by the store business data collection module within the preset business cycle includes the normal sales time and out-of-stock time of each commodity, the number of commodities identified by AI during commodity identification, the number of commodities identified manually, the number of commodities correctly identified by AI during commodity identification, the completion time of user pickup for each commodity order and the issuance time of the order to be paid, the number of correct and incorrect commodity inventory updates, the number of correct alarm issuances and the number of incorrect alarm issuances; Store business data processing module: processes the collected store business data to obtain the expected inventory of each commodity in the next preset business cycle and the average commodity out-of-stock rate coefficient, AI recognition availability coefficient, AI recognition accuracy coefficient, average user waiting time, inventory accuracy coefficient and automatic alarm accuracy coefficient of the current preset business cycle; The store business data processing module includes a store business data receiving unit, a store business data processing unit and a processed business data transmission unit, wherein the store business data receiving unit is used to receive the data collected by the store business data collection module; The store business data processing unit calculates the average commodity out-of-stock rate coefficient Xqe based on the out-of-stock duration Tqi and the normal selling duration Tsi of the i-th commodity in the preset business cycle. The specific formula is: Na is the number of commodity types within the preset business cycle. The AI ​​recognition availability coefficient Xsb is calculated based on the number of commodities recognized by AI in the preset business cycle, Mpa, and the number of commodities recognized manually, Mpb. The specific formula is: The AI ​​recognition accuracy coefficient Xsz is calculated based on the number of correctly identified products Mpz and the number of products recognized by AI during the preset business cycle. The specific formula is: The average waiting time Tde of users is calculated based on the completion time Twi of user pickup in the i-th product order within the preset business cycle and the time Tfi of the order to be paid. The specific formula is: Nd is the total number of product orders within the preset business cycle. The inventory accuracy coefficient Xkz is calculated based on the correct number of product inventory updates Mka and the incorrect number of product inventory updates Mkb within the preset business cycle. The specific formula is: The automatic alarm accuracy coefficient Xbz is calculated based on the number of correct alarms Mfa and the number of false alarms Mfb within the preset business cycle. The specific formula is: After processing, the business data transmission unit outputs the calculated average commodity out-of-stock rate coefficient, AI recognition availability coefficient, AI recognition accuracy coefficient, user average waiting time, inventory accuracy coefficient and automatic alarm accuracy coefficient to the store business quality evaluation module; Store business quality evaluation module: used to calculate the store's business quality index and compare it with the set expected business quality index to determine whether the store's business quality and unattended management effect meet expectations; The store business quality evaluation module includes a data receiving unit, a data secondary processing unit, a business quality index calculation unit and a business quality evaluation unit. The data receiving unit is used to receive various parameters transmitted by the store business data processing module; the data secondary processing unit calculates the AI ​​recognition application reliability coefficient Ysl based on the AI ​​recognition availability coefficient Xsb and the AI ​​recognition accuracy coefficient Xsz. The specific formula is: The automation processing reliability coefficient Ycl is calculated based on the inventory accuracy coefficient Xkz and the automatic alarm accuracy coefficient Xbz. The specific formula is: The specific formula for calculating the business quality index ZC by the business quality index calculation unit based on the average commodity out-of-stock rate coefficient Xqe, the average user waiting time Tde, the AI ​​recognition application reliability coefficient Ysl, and the automation processing reliability coefficient Ycl is: e is a natural constant; the business quality evaluation unit compares the calculated business quality index with the set expected business quality index. If the calculated value is greater than or equal to the set value, the store's business quality and unmanned management effects within the preset business cycle are in line with expectations; if the calculated value is less than the set value, the store's business quality and unmanned management effects within the preset business cycle are not in line with expectations.

2. According to claim 1, an unattended management system based on AI algorithm is characterized in that: The sales data of commodities within a preset business cycle collected by the store business data collection module specifically include the total inventory and actual sales volume of each commodity.

3. The unattended management system based on AI algorithm according to claim 1, characterized in that: The store business data processing module also includes a commodity sales data processing unit and a commodity inventory adjustment unit. The specific process of data processing is as follows: Commodity sales data processing unit: Calculate the sales coefficient αgi of the i-th commodity based on the total inventory Mci and the actual sales volume Mgi of the i-th commodity in the preset business cycle. The specific formula is: Commodity inventory adjustment unit: Calculate the expected inventory Myi of the i-th commodity based on the sales coefficient αgi and the actual sales volume Mgi of the i-th commodity. The specific formula is: The calculated expected inventory quantity Myi of the ith commodity is applied to the next preset business cycle.

Images