Intelligent push method and system for online commodities

Abstract

The invention is suitable for the technical field of commodity pushing, and particularly relates to an intelligent pushing method and system for online commodities, and the method comprises the steps: collecting the portrait information of a customer, and binding the portrait information of the customer with a mobile device of the customer; detecting the taking state of the commodity, acquiring parameter data of the commodity, and performing image acquisition to obtain two groups of monitoring images; calling a top view image, identifying a sheltered area based on the top view image, and determining a relationship between the sheltered area and the commodity number; and determining a corresponding relationship between the commodity and the customer according to the front view image, and sending corresponding parameter data. According to the invention, by monitoring the commodity and determining the customer who takes out the commodity when the commodity is taken out, the parameter data corresponding to the commodity is retrieved and sent to the corresponding mobile device through the local area network, so that the user can further understand the commodity information in an online manner and can purchase the commodity in an online manner, and the user experience is improved. And the shopping convenience of the user is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of product push, and in particular relates to an intelligent push method and system for online products. Background Art

[0002] Unmanned stores are a new retail model that leverages advanced technologies such as artificial intelligence, the Internet of Things, and automated payment systems to create a fully automated shopping experience without human cashiers. Customers simply enter the store using a mobile app or biometrics, select their desired items, and leave. The items selected throughout the shopping process are automatically recorded and charged to the customer's account upon departure. This model not only improves shopping efficiency and saves customers time, but also reduces labor costs for merchants and offers consumers a more convenient and intelligent shopping experience.

[0003] In current unmanned stores, although customers can directly see the product itself, the relevant information of the product cannot be directly obtained by customers. Customers' incomplete understanding of the product information has become a major shortcoming of unmanned stores. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent push method for online products, aiming to solve the problem that in current unmanned stores, although customers can directly see the product itself, the relevant information of the product cannot be directly obtained by the customers, and the customers have incomplete understanding of the information about the product, which has become a major shortcoming of unmanned stores.

[0005] The present invention is implemented as follows: a method for intelligently pushing online products, the method comprising:

[0006] Collecting customer portrait information, storing the customer portrait information in a local area network, binding the customer portrait information to the customer's mobile device, and establishing a data connection with the mobile device. The unmanned store is equipped with a smart shelf, which is used to place goods and monitor the access status of goods. The smart shelf is equipped with an image acquisition device;

[0007] Detecting the access status of the product, obtaining parameter data of the product when the product is accessed, and controlling the image acquisition device in the corresponding area to acquire images, thereby obtaining two sets of monitoring images, namely, a front view image and a top view image;

[0008] Retrieving a bird's-eye view image, identifying an occluded area based on the bird's-eye view image, and determining a relationship between the occluded area and the product number;

[0009] The correspondence between the product and the customer is determined based on the front view image, and the corresponding parameter data is sent to the customer's mobile device.

[0010] Preferably, the step of detecting the use status of the commodity, obtaining parameter data of the commodity when the commodity is used, and controlling the image acquisition device in the corresponding area to perform image acquisition to obtain two sets of monitoring images specifically includes:

[0011] Detect the access status of the goods and determine the corresponding product number when detecting that the goods have left the smart shelf;

[0012] Retrieving parameter data of the corresponding product from a preset product information database according to the product number, the parameter data also includes information of similar online products;

[0013] An image acquisition instruction is sent to the image acquisition device to control the two groups of image acquisition devices to synchronously acquire images, thereby obtaining two groups of monitoring images, namely a front view image and a top view image.

[0014] Preferably, the steps of retrieving the overhead image, identifying the blocked area based on the overhead image, and determining the relationship between the blocked area and the product number specifically include:

[0015] Retrieving the overhead image, identifying ground markers contained in the overhead image, and determining product numbers corresponding to ground markers that do not appear in the overhead image, wherein the ground markers are set on the ground and have a one-to-one correspondence with products on the smart shelf;

[0016] The target detection algorithm is used to identify the human body area in the overhead image, and each human body area is regarded as an occlusion area;

[0017] The occluded areas in the overhead image are divided and the product numbers corresponding to the different occluded areas are determined.

[0018] Preferably, the step of determining the correspondence between the product and the customer based on the front view image and sending corresponding parameter data to the customer's mobile device specifically includes:

[0019] Determine the location of the corresponding blocked area according to the product number of the product being taken, and determine the position of the human portrait in the blocked area in the front view image;

[0020] According to the position of the portrait, the image of the corresponding area in the front view image is retrieved to perform portrait recognition and identify the customer who takes the product;

[0021] According to the binding relationship between the customer and the mobile device, the parameter data of the product is sent to the mobile device via the local area network.

[0022] Preferably, the commodity is provided with a sensor, and the sensor is used to interact with the smart shelf.

[0023] Another object of the present invention is to provide an intelligent push system for online products, the system comprising:

[0024] a device connection module, for collecting customer portrait information, storing the customer portrait information in a local area network, binding the customer portrait information to the customer's mobile device, and establishing a data connection with the mobile device; a smart shelf is provided in the unmanned store, and the smart shelf is used to place goods and monitor the access status of goods, and an image acquisition device is provided on the smart shelf;

[0025] The image acquisition module is used to detect the access status of the product. When the product is accessed, it obtains the parameter data of the product and controls the image acquisition device in the corresponding area to perform image acquisition to obtain two sets of monitoring images, namely the front view image and the top view image;

[0026] A number matching module is used to retrieve the overhead image, identify the occluded area based on the overhead image, and determine the relationship between the occluded area and the product number;

[0027] The data interaction module is used to determine the corresponding relationship between the product and the customer based on the front view image and send corresponding parameter data to the customer's mobile device.

[0028] Preferably, the graphics acquisition module includes:

[0029] The number recognition unit is used to detect the access status of the product and determine the corresponding product number when it is detected that the product has left the smart shelf;

[0030] A parameter data retrieving unit, configured to retrieve parameter data of a corresponding product from a preset product information database according to the product number, wherein the parameter data also includes information of similar online products;

[0031] The image acquisition unit is used to send image acquisition instructions to the image acquisition device, control the two groups of image acquisition devices to synchronously acquire images, and obtain two groups of monitoring images, namely the front view image and the top view image.

[0032] Preferably, the number matching module includes:

[0033] A product identification unit is configured to retrieve the overhead image, identify ground markers contained in the overhead image, and determine product numbers corresponding to ground markers that do not appear in the overhead image. The ground markers are set on the ground and correspond one-to-one with products on the smart shelf.

[0034] The occlusion region division unit is used to identify the human body region in the overhead image using a target detection algorithm, and each human body region is regarded as an occlusion region;

[0035] The region matching unit is used to divide the occluded regions in the overhead image and determine the product numbers corresponding to different occluded regions.

[0036] Preferably, the data interaction module includes:

[0037] A portrait position locating unit, configured to determine the position of a corresponding blocked area according to the product number of the product being taken, and to determine the position of the portrait in the blocked area in the front-view image;

[0038] The customer recognition unit is used to retrieve the image of the corresponding area in the front view image according to the position of the portrait, perform portrait recognition, and identify the customer who takes the product;

[0039] The product push unit is used to send the parameter data of the product to the mobile device through the local area network according to the binding relationship between the customer and the mobile device.

[0040] Preferably, the commodity is provided with a sensor, and the sensor is used to interact with the smart shelf.

[0041] The present invention provides an intelligent push method for online products. The method monitors the products and determines the customer who takes out the products when the products are taken out, thereby retrieving parameter data corresponding to the products and sending it to the corresponding mobile device through a local area network, so that users can further understand the product information online and purchase them online, which greatly improves the convenience of shopping for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a method for intelligently pushing online products provided by an embodiment of the present invention;

[0043] Figure 2 A flowchart of the steps of detecting the use status of a commodity provided by an embodiment of the present invention, obtaining parameter data of the commodity when the commodity is used, and controlling the image acquisition device in the corresponding area to perform image acquisition to obtain two sets of monitoring images;

[0044] Figure 3 A flowchart of the steps of retrieving a bird's-eye view image, identifying an occluded area based on the bird's-eye view image, and determining a relationship between the occluded area and a product number, provided by an embodiment of the present invention;

[0045] Figure 4 A flowchart of the steps of determining the correspondence between a product and a customer based on a front-view image and sending corresponding parameter data to the customer's mobile device according to an embodiment of the present invention;

[0046] Figure 5 This is an architectural diagram of an intelligent online product push system provided by an embodiment of the present invention;

[0047] Figure 6 This is an architectural diagram of a graphics acquisition module provided in an embodiment of the present invention;

[0048] Figure 7 An architectural diagram of a number matching module provided in an embodiment of the present invention;

[0049] Figure 8 An architectural diagram of a data interaction module provided in an embodiment of the present invention;

[0050] Figure 9 A schematic plan view of an unmanned store provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0052] like Figure 1 FIG. 1 is a flow chart of a method for intelligently pushing online products according to an embodiment of the present invention, the method comprising:

[0053] S100, collect the customer's portrait information, store the customer's portrait information in the local area network, bind the customer's portrait information to the customer's mobile device, and establish a data connection with the mobile device. The unmanned store is equipped with a smart shelf, which is used to place goods and monitor the access status of goods. The smart shelf is equipped with an image acquisition device.

[0054] In this step, the customer's portrait information is collected. When the customer enters the unmanned store, he / she is authenticated by scanning the code. After the authentication is completed, the customer accesses the local area network of the unmanned store through the mobile device. At this time, the mobile device can establish a data connection with the unmanned store through the local area network. The local area network is equipped with a data processing center. The unmanned store is equipped with smart shelves. There are multiple reading devices on the smart shelves. The goods are equipped with corresponding sensors. The sensors correspond to the reading devices. When the sensor is an NFC coil, the reading device is an NFC reading device. When the sensor is an RFID electronic tag, the reading device is an RFID reading device. The two When a product is placed on a smart shelf, the sensor on the product can be detected by the corresponding reading device. When the product leaves the smart shelf, the reading device cannot detect the sensor and determines that the product has been taken off. Two image acquisition devices are set on the smart shelf. One image acquisition device is set on the top of the smart shelf for downward shooting. The other image acquisition device is set in the middle of the smart shelf for capturing portraits of customers who take the product. The shooting direction is toward the front of the smart shelf. When the product leaves, the corresponding image acquisition device performs image acquisition. In order to facilitate the positioning of customers, corresponding ground signs are set on the ground.

[0055] S200, detecting the use status of the product. When the product is used, obtaining parameter data of the product, and controlling the image acquisition device in the corresponding area to perform image acquisition to obtain two sets of monitoring images, which are a front view image and a top view image.

[0056] In this step, the access status of the goods is detected, and the data processing center accesses each reading device at a preset time interval, and determines the goods placed at each position on the smart shelf based on the information read by the reading device. When the goods are taken down, the sensor is disconnected from the reading device. At this time, a query is performed based on the product number of the goods, and the corresponding parameter data is retrieved from the preset product information database. The parameter data of the goods at least includes the parameter information of the goods themselves, such as the size, model, fabric, instructions and usage of the goods, and can also include other online goods similar to the goods. The above data is temporarily stored in the data processing center. At the same time, the image acquisition device at the corresponding position on the smart shelf is controlled to work according to the position of the corresponding reading device, so as to obtain a group of overhead images and a group of front view images. The overhead image is taken from the top down to determine the position of the person, and the front view image is used to collect the user's facial information to determine the customer's identity.

[0057] S300: Retrieve a bird's-eye view image, identify an occluded area based on the bird's-eye view image, and determine a relationship between the occluded area and the product number.

[0058] In this step, the overhead image is retrieved. The overhead image is taken from above the customer and is used to determine the distribution of people inside the unmanned store. After the overhead image and the front view image are collected, they are sent to the data processing center through the local area network for processing. The position of the people in the overhead image is determined by the preset people recognition algorithm. Each person will cover a certain area of ​​the picture area. The area covered by the person is defined as the blocked area. The position of the product on the smart shelf can be determined according to the product number. When the customer takes the product, the ground sign will be blocked. The customer's position can be determined according to the position of the blocked ground sign. Based on this, the position distribution of different people can be determined, and the position of the product on the smart shelf can be determined according to the ground sign. Therefore, the position of the customer who takes the product can be preliminarily determined.

[0059] S400: Determine the correspondence between the product and the customer based on the front view image, and send corresponding parameter data to the customer's mobile device.

[0060] In this step, a front view image is obtained. Each front view image can cover a limited number of products on the smart shelf, and the distribution of the positions of the products on the smart shelf is uniform. Therefore, based on the position of the taken product on the smart shelf, a portrait at the corresponding position in the front view image is selected for image recognition, and the portrait is compared with the portrait photo taken when the customer enters the unmanned store to determine the identity of the customer. Based on this, the identity of the customer who took the product in the overhead view image is determined. When it is determined by the overhead view image that the customer has entered the checkout area of ​​the unmanned store, the corresponding product settlement information is displayed on the cash register display screen. After determining the correspondence between the product and the customer, the parameter data of the product is sent to the customer's mobile device via the local area network. The customer can view the relevant information of the product on the mobile device or place an order on the mobile device. The ordering method can be online mail or online payment and offline pickup. At the same time, other products related to the product are displayed on the mobile device, realizing intelligent online push of similar products and displaying the location of the pushed product to facilitate the customer to find the corresponding pushed product in the unmanned store.

[0061] like Figure 2 As shown, as a preferred embodiment of the present invention, the steps of detecting the use status of a commodity, obtaining parameter data of the commodity when the commodity is used, and controlling the image acquisition device in the corresponding area to perform image acquisition to obtain two sets of monitoring images specifically include:

[0062] S201, detecting the access status of the product, and determining the corresponding product number when detecting that the product has left the smart shelf.

[0063] In this step, the access status of the goods is detected and each reading device is polled at a preset time interval. When the reading device reads the sensor, the sensor information is recorded. The sensor stores the product number. When the product number can be read from the reading device, it is determined that the product has not been taken. Conversely, when the product number cannot be read, it is determined that the product has been taken away.

[0064] S202: Retrieve parameter data of the corresponding product from a preset product information database according to the product number, wherein the parameter data also includes information of similar online products.

[0065] S203: Send an image acquisition instruction to the image acquisition device to control the two groups of image acquisition devices to synchronously acquire images, thereby obtaining two groups of monitoring images, namely, a front view image and a top view image.

[0066] In this step, data query is performed based on the product number, and parameter information of each product is recorded in the product information database. The parameter information can be displayed through the online product details page, and users can view and place orders on the details page, providing synchronous online shopping services. When the product is taken down, an image acquisition instruction is sent to the image acquisition device, such as Figure 9 As shown, for the image acquisition device that captures the front view image, according to the position of the product on the smart shelf, the image acquisition device in the corresponding area is called to capture the image, and the top view image is captured simultaneously to obtain two sets of monitoring images.

[0067] like Figure 3 As shown, as a preferred embodiment of the present invention, the steps of retrieving the overhead image, identifying the occluded area based on the overhead image, and determining the relationship between the occluded area and the product number specifically include:

[0068] S301, retrieve the overhead image, identify the ground markers contained in the overhead image, and determine the product numbers corresponding to the ground markers that do not appear in the overhead image. The ground markers are set on the ground and have a one-to-one correspondence with the products on the smart shelf.

[0069] In this step, a bird's-eye view image is retrieved. In the bird's-eye view image, various ground signs can be seen. Ground signs can be marked with text, patterns, or QR codes. Ground signs are unique, and their identities are determined by image recognition. The setting position of the ground sign is fixed, and its position in the unmanned store can be determined. When a customer stands on the ground sign, the ground sign will be blocked. There is a correspondence between the ground sign and the goods on the smart shelf.

[0070] S302: Using a target detection algorithm to identify human body regions in the overhead image, each human body region is considered as an occlusion region.

[0071] In this step, a target detection algorithm is used to identify human body areas in the overhead image. The target detection algorithm can be a YOLO (You Only Look Once) model or an SSD (Single Shot MultiBox Detector) model, so as to quickly determine the location of the customer in the overhead image. The location blocked by the customer is the blocked area. Due to the overhead perspective, the identity of the customer cannot be determined.

[0072] S303: Divide the occluded areas in the overhead image and determine the product numbers corresponding to the different occluded areas.

[0073] In this step, the occlusion area in the overhead image is divided, and different occlusion areas represent different customers. The goods in front of the customer can be determined based on the ground signs blocked by the occlusion area. If there is a product among the above-mentioned goods that has been taken, it is determined that the product has been taken away by the user. If there is no ground sign in the occlusion area, it is determined that the customer is away from the smart shelf and has not performed any operation of taking or placing goods.

[0074] like Figure 4 As shown, as a preferred embodiment of the present invention, the step of determining the correspondence between the product and the customer based on the front view image and sending the corresponding parameter data to the customer's mobile device specifically includes:

[0075] S401: Determine the position of a corresponding blocking area according to the product number of the product to be taken, and determine the position of the portrait of the blocking area in the front-view image.

[0076] S402: Retrieve an image of a corresponding area in the front view image according to the position of the portrait, perform portrait recognition, and identify the customer who takes the product.

[0077] In this step, the location of the corresponding blocked area is determined according to the product number of the taken product. Since the position of the product on the smart shelf is known, the position of the ground mark corresponding to the product is also known. Based on this, the matching relationship between the taken product and the customer in the overhead image can be determined, and the position of the customer in the front view image is determined according to the position distribution of the products on the smart shelf. Specifically, if the smart shelf is divided into multiple cargo intervals, each cargo interval has 10 products placed on it, and one cargo interval has ten products numbered 1-10 placed on it, the corresponding image acquisition device is A. When product No. 8 in the cargo interval is taken, a set of front view images will be generated. The front view image is divided into ten equal parts according to the order of the product arrangement to obtain ten blocks. The block numbers are the same as the product numbers. Block No. 8 and its adjacent blocks No. 7 and No. 9 are selected, and portrait recognition is performed on blocks No. 7, No. 8 and No. 9 to extract portrait information. Based on the extracted portrait information, it is compared with the portrait photo of the customer when he enters, so as to determine the customer who takes the current product.

[0078] S403: According to the binding relationship between the customer and the mobile device, the parameter data of the product is sent to the mobile device via the local area network.

[0079] In this step, based on the binding relationship between the customer and the mobile device, the data processing center sends the prepared commodity parameter data to the corresponding mobile device through the local area network, and the user can view it through the mobile device. When multiple customers visit the unmanned store at the same time, multiple customers are allowed to be bound to the same mobile device for group access. When settling, all settlements are made through the mobile device to enhance the user experience. By performing image recognition on the front view image, the customer in the front view image and the overhead view image can be matched, thereby determining the customer's location in real time. When the customer enters the corresponding settlement area, quick settlement can be performed and an order can be generated quickly. A portrait recognition device can be further set up in the settlement area to further confirm the user's identity or implement facial payment.

[0080] like Figure 5 As shown, an embodiment of the present invention further provides an intelligent push system for online products, the system comprising:

[0081] The device connection module 100 is used to collect the customer's portrait information, store the customer's portrait information in the local area network, bind the customer's portrait information to the customer's mobile device, and establish a data connection with the mobile device. The unmanned store is equipped with a smart shelf, which is used to place goods and monitor the access status of the goods. The smart shelf is equipped with an image acquisition device.

[0082] In this system, the device connection module 100 collects the customer's portrait information. When the customer enters the unmanned store, the customer is authenticated by scanning the code. After the authentication is completed, the customer accesses the local area network of the unmanned store through the mobile device. At this time, the mobile device can establish a data connection with the unmanned store through the local area network. The local area network is equipped with a data processing center. The unmanned store is equipped with smart shelves. There are multiple reading devices on the smart shelves. The goods are equipped with corresponding sensors. The sensors correspond to the reading devices. When the sensor is an NFC coil, the reading device is an NFC reading device. When the sensor is an RFID electronic tag, the reading device is an RFID reader. The two devices are used in combination. When a product is placed on the smart shelf, the sensor on the product can be detected by the corresponding reading device. When the product leaves the smart shelf, the reading device cannot detect the sensor and determines that the product is taken away. Two image acquisition devices are set on the smart shelf. One image acquisition device is set on the top of the smart shelf for downward shooting. The other image acquisition device is set in the middle of the smart shelf for collecting portraits of customers who take the products. The shooting direction is toward the front of the smart shelf. When the product leaves, the corresponding image acquisition device performs image acquisition. In order to facilitate the positioning of customers, corresponding ground signs are set on the ground.

[0083] The image acquisition module 200 is used to detect the status of the goods being taken. When the goods are taken, the parameter data of the goods are obtained, and the image acquisition device in the corresponding area is controlled to perform image acquisition to obtain two sets of monitoring images, which are front view images and top view images respectively.

[0084] In this system, the graphic acquisition module 200 detects the access status of the goods, and the data processing center accesses each reading device at a preset time interval, and determines the goods placed at each position on the smart shelf based on the information read by the reading device. When the goods are taken down, the sensor is disconnected from the reading device. At this time, a query is performed based on the product number of the goods, and the corresponding parameter data is retrieved from the preset product information database. The parameter data of the goods at least includes the parameter information of the goods themselves, such as the size, model, fabric, instructions and usage of the goods, and can also include other online goods similar to the goods. The above data is temporarily stored in the data processing center. At the same time, the image acquisition device at the corresponding position on the smart shelf is controlled to work according to the position of the corresponding reading device, so as to obtain a group of overhead images and a group of front view images. The overhead image is taken from the top down to determine the position of the person, and the front view image is used to collect the user's facial information to determine the customer's identity.

[0085] The serial number matching module 300 is used to retrieve the overhead image, identify the occluded area based on the overhead image, and determine the relationship between the occluded area and the product serial number.

[0086] In this system, the number matching module 300 retrieves the overhead image, which is taken from above the customer and is used to determine the distribution of people inside the unmanned store. After the overhead image and the front view image are collected, they are sent to the data processing center through the local area network for processing. The position of the people in the overhead image is determined by the preset people recognition algorithm. Each person will cover a certain area of ​​the picture area. The area covered by the person is defined as the blocked area. The position of the product on the smart shelf can be determined according to the product number. When the customer takes the product, the ground sign will be blocked. The customer's position can be determined according to the position of the blocked ground sign. Based on this, the position distribution of different people can be determined, and the position of the product on the smart shelf can be determined according to the ground sign. Therefore, the position of the customer who takes the product can be preliminarily determined.

[0087] The data interaction module 400 is used to determine the correspondence between the product and the customer based on the front view image and send corresponding parameter data to the customer's mobile device.

[0088] In this system, the data interaction module 400 obtains a front view image. Each front view image can only cover a limited number of products on the smart shelf, and the products are evenly distributed on the smart shelf. Therefore, based on the position of the taken product on the smart shelf, a portrait at the corresponding position in the front view image is selected for image recognition and compared with a portrait photo taken when the customer enters the unmanned store to determine the customer's identity. Based on this, the identity of the customer who took the product in the overhead image is determined. When the overhead image determines that the customer has entered the checkout area of ​​the unmanned store, the corresponding product settlement information is displayed on the cashier display screen. After determining the correspondence between the product and the customer, the parameter data of the product is sent to the customer's mobile device via the local area network. The customer can view the relevant information of the product on the mobile device and place an order on the mobile device. The order placement method can be online mail or online payment and offline pickup. At the same time, other products related to the product are displayed on the mobile device, realizing intelligent online push of similar products and displaying the location of the pushed products to facilitate the customer to find the corresponding pushed products in the unmanned store.

[0089] like Figure 6 As shown, as a preferred embodiment of the present invention, the graphics acquisition module 200 includes:

[0090] The number recognition unit 201 is used to detect the use status of the product and determine the corresponding product number when it is detected that the product has left the smart shelf.

[0091] In this module, the number recognition unit 201 detects the status of the goods being taken and polls each reading device at a preset time interval. When the reading device reads the sensor, it records the information of the sensor. The sensor stores the product number. When the product number can be read from the reading device, it is determined that the product has not been taken. Conversely, when the product number cannot be read, it is determined that the product has been taken away.

[0092] The parameter data retrieving unit 202 is configured to retrieve parameter data of a corresponding product from a preset product information database according to the product number. The parameter data also includes information of similar online products.

[0093] The image acquisition unit 203 is used to send an image acquisition instruction to the image acquisition device, control the two groups of image acquisition devices to synchronously acquire images, and obtain two groups of monitoring images, namely a front view image and a top view image.

[0094] In this module, the parameter data retrieval unit 202 performs data query based on the product number, and records the parameter information of each product in the product information database. The parameter information can be displayed through the online product details page, and the user can view and place an order on the details page, providing a synchronous online shopping service. When the product is taken down, an image acquisition instruction is sent to the image acquisition device, such as Figure 9 As shown, for the image acquisition device that captures the front view image, according to the position of the product on the smart shelf, the image acquisition device in the corresponding area is called to capture the image, and the top view image is captured simultaneously to obtain two sets of monitoring images.

[0095] like Figure 7 As shown, as a preferred embodiment of the present invention, the number matching module 300 includes:

[0096] The product identification unit 301 is used to retrieve the overhead image, identify the ground signs contained in the overhead image, and determine the product numbers corresponding to the ground signs that do not appear in the overhead image. The ground signs are set on the ground and correspond one-to-one to the products on the smart shelf.

[0097] In this module, the product identification unit 301 retrieves the overhead image. In the overhead image, various ground signs can be seen. The ground signs can be marked by text, patterns or QR codes. The ground signs are unique. The identity of the ground signs is determined by image recognition. The setting position of the ground signs is fixed, and their position in the unmanned store can be determined. When a customer stands on the ground sign, the ground sign will be blocked. There is a correspondence between the ground signs and the products on the smart shelves.

[0098] The occlusion region division unit 302 is configured to identify human body regions in the overhead image using a target detection algorithm, with each human body region being considered as an occlusion region.

[0099] In this module, the occlusion area division unit 302 uses a target detection algorithm to identify the human body area in the overhead image. The target detection algorithm can be a YOLO (You Only Look Once) model or an SSD (Single Shot MultiBoxDetector) model, so as to quickly determine the location of the customer in the overhead image. The location blocked by the customer is the occlusion area. Due to the use of a bird's-eye view, the identity of the customer cannot be determined.

[0100] The region matching unit 303 is configured to divide the occluded regions in the overhead image and determine the product numbers corresponding to the different occluded regions.

[0101] In this module, the area matching unit 303 divides the occlusion area in the overhead image. Different occlusion areas represent different customers. The goods in front of the customer can be determined based on the ground signs blocked by the occlusion areas. If there is a product among the above-mentioned goods that has been taken, it is determined that the product has been taken away by the user. If there is no ground sign in the occlusion area, it is determined that the customer is away from the smart shelf and has not performed any operation of taking or placing goods.

[0102] like Figure 8 As shown, as a preferred embodiment of the present invention, the data interaction module 400 includes:

[0103] The portrait position locating unit 401 is used to determine the position of the corresponding blocking area according to the product number of the taken product, and determine the portrait position of the blocking area in the front view image.

[0104] The customer recognition unit 402 is used to retrieve an image of a corresponding area in the front view image according to the position of the portrait, perform portrait recognition, and identify the customer who takes the product.

[0105] In this module, the portrait position positioning unit 401 determines the location of the corresponding occlusion area according to the product number of the taken product. Since the position of the product on the smart shelf is known, the position of the ground mark corresponding to the product is also known. Based on this, the matching relationship between the taken product and the customer in the overhead image can be determined, and the position of the customer in the front view image is determined according to the position distribution of the products on the smart shelf. Specifically, if the smart shelf is divided into multiple cargo intervals, each cargo interval contains 10 products, and one cargo interval contains ten products numbered 1-10, the corresponding image acquisition device is A. When product No. 8 in the cargo interval is taken, a set of front view images will be generated. The front view image is divided into ten equal parts according to the order of the product arrangement to obtain ten blocks. The block numbers are the same as the product numbers. Block No. 8 and its adjacent blocks No. 7 and No. 9 are selected, and portrait recognition is performed on blocks No. 7, No. 8 and No. 9 to extract portrait information. Based on the extracted portrait information, it is compared with the portrait photo of the customer when entering, so as to determine the customer who takes the current product.

[0106] The product push unit 403 is used to send the parameter data of the product to the mobile device via the local area network according to the binding relationship between the customer and the mobile device.

[0107] In this module, the product push unit 403 sends the prepared product parameter data to the corresponding mobile device through the local area network according to the binding relationship between the customer and the mobile device. The data processing center can view the data through the mobile device. When multiple customers visit the unmanned store at the same time, multiple customers are allowed to be bound to the same mobile device for group access. When settling, all settlements are made through the mobile device to improve the user experience. By performing image recognition on the front view image, the customer in the front view image and the overhead view image can be matched, so as to determine the customer's location in real time. When the customer enters the corresponding settlement area, quick settlement can be performed and an order can be generated quickly. A portrait recognition device can be further set in the settlement area to further confirm the user's identity or realize face payment.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for intelligently pushing online products, characterized in that: The method comprises: Collecting customer portrait information, storing the customer portrait information in a local area network, binding the customer portrait information to the customer's mobile device, and establishing a data connection with the mobile device. The unmanned store is equipped with a smart shelf, which is used to place goods and monitor the access status of goods. The smart shelf is equipped with an image acquisition device; Detecting the access status of the product, obtaining parameter data of the product when the product is accessed, and controlling the image acquisition device in the corresponding area to acquire images, thereby obtaining two sets of monitoring images, namely, a front view image and a top view image; Retrieving a bird's-eye view image, identifying an occluded area based on the bird's-eye view image, and determining a relationship between the occluded area and the product number; The correspondence between the product and the customer is determined based on the front view image, and the corresponding parameter data is sent to the customer's mobile device.

2. The method for intelligently pushing online products according to claim 1, characterized in that: The step of detecting the use status of the commodity, obtaining parameter data of the commodity when the commodity is used, and controlling the image acquisition device in the corresponding area to perform image acquisition to obtain two sets of monitoring images specifically includes: Detect the access status of the goods and determine the corresponding product number when detecting that the goods have left the smart shelf; Retrieving parameter data of the corresponding product from a preset product information database according to the product number, the parameter data also includes information of similar online products; An image acquisition instruction is sent to the image acquisition device to control the two groups of image acquisition devices to synchronously acquire images, thereby obtaining two groups of monitoring images, namely a front view image and a top view image.

3. The method for intelligently pushing online products according to claim 1, characterized in that: The steps of retrieving the overhead image, identifying the blocked area based on the overhead image, and determining the relationship between the blocked area and the product number specifically include: Retrieving the overhead image, identifying ground markers contained in the overhead image, and determining product numbers corresponding to ground markers that do not appear in the overhead image, wherein the ground markers are set on the ground and have a one-to-one correspondence with products on the smart shelf; The target detection algorithm is used to identify the human body area in the overhead image, and each human body area is regarded as an occlusion area; The occluded areas in the overhead image are divided and the product numbers corresponding to the different occluded areas are determined.

4. The method for intelligently pushing online products according to claim 1, characterized in that: The step of determining the correspondence between the product and the customer based on the front view image and sending corresponding parameter data to the customer's mobile device specifically includes: Determine the location of the corresponding blocked area according to the product number of the product being taken, and determine the position of the human portrait in the blocked area in the front view image; According to the position of the portrait, the image of the corresponding area in the front view image is retrieved to perform portrait recognition and identify the customer who takes the product; According to the binding relationship between the customer and the mobile device, the parameter data of the product is sent to the mobile device via the local area network.

5. The method for intelligently pushing online products according to claim 1, characterized in that: The commodity is provided with a sensor, which is used to interact with the smart shelf.

6. An intelligent push system for online products, characterized in that: The system comprises: a device connection module, for collecting customer portrait information, storing the customer portrait information in a local area network, binding the customer portrait information to the customer's mobile device, and establishing a data connection with the mobile device; a smart shelf is provided in the unmanned store, and the smart shelf is used to place goods and monitor the access status of goods, and an image acquisition device is provided on the smart shelf; The image acquisition module is used to detect the access status of the product. When the product is accessed, it obtains the parameter data of the product and controls the image acquisition device in the corresponding area to perform image acquisition to obtain two sets of monitoring images, namely the front view image and the top view image; A number matching module is used to retrieve the overhead image, identify the occluded area based on the overhead image, and determine the relationship between the occluded area and the product number; The data interaction module is used to determine the corresponding relationship between the product and the customer based on the front view image and send corresponding parameter data to the customer's mobile device.

7. The intelligent push system for online products according to claim 6, characterized in that: The graphics acquisition module includes: The number recognition unit is used to detect the access status of the product and determine the corresponding product number when it is detected that the product has left the smart shelf; A parameter data retrieving unit, configured to retrieve parameter data of a corresponding product from a preset product information database according to the product number, wherein the parameter data also includes information of similar online products; The image acquisition unit is used to send image acquisition instructions to the image acquisition device, control the two groups of image acquisition devices to synchronously acquire images, and obtain two groups of monitoring images, namely the front view image and the top view image.

8. The intelligent push system for online products according to claim 6, characterized in that: The number matching module includes: A product identification unit is configured to retrieve the overhead image, identify ground markers contained in the overhead image, and determine product numbers corresponding to ground markers that do not appear in the overhead image. The ground markers are set on the ground and correspond one-to-one with products on the smart shelf. The occlusion region division unit is used to identify the human body region in the overhead image using a target detection algorithm, and each human body region is regarded as an occlusion region; The region matching unit is used to divide the occluded regions in the overhead image and determine the product numbers corresponding to different occluded regions.

9. The intelligent push system for online products according to claim 6, characterized in that: The data interaction module includes: A portrait position locating unit, configured to determine the position of a corresponding blocked area according to the product number of the product being taken, and to determine the position of the portrait in the blocked area in the front-view image; The customer recognition unit is used to retrieve the image of the corresponding area in the front view image according to the position of the portrait, perform portrait recognition, and identify the customer who takes the product; The product push unit is used to send the parameter data of the product to the mobile device through the local area network according to the binding relationship between the customer and the mobile device.

10. The intelligent push system for online products according to claim 6, characterized in that: The commodity is provided with a sensor, which is used to interact with the smart shelf.

Images