Artificial intelligence model data updating method for intelligent vending machine

Abstract

The invention discloses an artificial intelligence model data updating method for an intelligent vending machine. The artificial intelligence model data updating method comprises the steps of obtaining an actual purchase video which is determined by a commodity recognition algorithm or manual auditing and comprises untrained new commodities; the actual purchase video is marked with target commodity SKU information. Performing new commodity sub-graph interception on the actual purchase video; storing the commodity local sub-graph into a new commodity sub-graph to-be-processed database; when the commodity local sub-graphs in the new commodity sub-graph to-be-processed database exceed a threshold value, starting verification of all the commodity local sub-graphs in the new commodity sub-graph to-be-processed database; screening the commodity local sub-graphs in the new commodity sub-graph to-be-processed database to obtain new commodity replacement pictures, and verifying the recognition accuracy of the screened new commodity replacement pictures; and after the verification is passed, replacing the screened new commodity replacement picture into the original new commodity sample picture library so as to replace the commodity sample picture of the untrained commodity in the commodity recognition algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of smart vending machines, and in particular to a method for updating artificial intelligence model data for smart vending machines. Background Art

[0002] With the continuous development of vending machine technology, open-door vending cabinets have gradually become a more common vending machine in the field of unmanned vending. In the field of vending machines, compared with the old-fashioned method of selecting goods by pressing buttons, the purchase method in which consumers actively open the door to take goods can improve consumers' shopping experience. At the same time, it also reduces the requirements for the hardware complexity of vending machines, and different types of goods can be arranged in the same vending cabinet.

[0003] At present, during the operation of open-door vending machines, visual recognition is generally used to identify consumers' purchasing behavior, thereby determining the quantity and price of the items purchased by consumers. Based on this information, consumption settlement and deduction are completed on the remote management platform. In the process of realizing the above-mentioned shopping settlement, accurately identifying the type and quantity of goods actually taken by the purchaser is an essential technical link to achieve accurate settlement of the purchase process.

[0004] However, in actual operation, the operator of the smart vending machine often needs to add new products to the vending machine. When this demand arises, the operator generally provides sample pictures of the newly added products, and the product recognition algorithm completes the update training of the algorithm model based on the newly added product pictures, so that the algorithm can be updated to accurately identify the newly added products. However, for the algorithm side, the frequent product updates and continuous updating of the recognition algorithm greatly increase the training time of the algorithm model, which causes the update speed of the algorithm model to lag far behind the speed of new products, resulting in a bad user experience for consumers in the operation of smart vending.

[0005] It can be seen that in response to the demand for updating and upgrading the product recognition algorithm when new products are launched, a new data updating method is needed in the prior art to achieve rapid updating and deployment of the product recognition algorithm. Summary of the invention

[0006] The technical problem to be solved by the present invention is to quickly update and deploy a product recognition algorithm model for new products after the smart vending machine updates the products, so that the product recognition algorithm can quickly and accurately recognize the new products.

[0007] The technical purpose to be achieved by the present invention is to provide a method for training unidentified commodities applicable to intelligent vending machines, the method comprising the following steps:

[0008] An actual purchase video containing untrained new products determined by a product recognition algorithm or manual review is obtained; the product recognition algorithm recognizes the product SKU in the actual purchase video based on standard pictures of products that have undergone supervised learning and sample pictures of new products that have not undergone supervised learning; the actual purchase video is annotated with target product SKU information.

[0009] Capturing new product sub-images from the actual purchase video; and storing the product partial sub-images in a new product sub-image to-be-processed database;

[0010] When the number of commodity local subgraphs in the new commodity subgraph to-be-processed database exceeds a threshold, the verification of all commodity local subgraphs in the new commodity subgraph to-be-processed database is initiated;

[0011] The local sub-images of the products in the new product sub-image to be processed database are screened to obtain new product replacement images, and the recognition accuracy of the screened new product replacement images is verified; after the verification, the above-mentioned screened new product replacement images are replaced into the original new product sample image library to replace the product sample images of untrained products in the product recognition algorithm.

[0012] In one embodiment, the actual purchase video containing untrained new products determined by a product recognition algorithm or manual review means: the products in the actual purchase video identified by the product recognition algorithm are untrained new products; or, the recognition results output by the product recognition algorithm have a low confidence level and require manual review, and the recognition results are determined to be untrained new products through manual review.

[0013] In one embodiment, the extracting new product sub-images from the actual purchase video includes: using a target detection model to detect a local area of ​​a product on each frame of the actual purchase video;

[0014] Perform binary classification on all product local areas to remove non-product local areas;

[0015] The reserved local area of ​​the product is intercepted to generate a local sub-graph of the product;

[0016] Generate a feature matrix of the local sub-graph of the product according to the local sub-graph of the product; and generate a feature matrix of all product pictures according to all product pictures; the all product pictures are the standard pictures of the product and the sample pictures of the new product;

[0017] REID classification is performed based on the feature matrix of the local sub-graph of the product and the feature matrix of all product images to obtain the product SKU classification result of each image in the local sub-graph of the product.

[0018] The new product sub-graph to be finally intercepted is determined based on the classification results.

[0019] In one embodiment, the method of determining the new product sub-graph to be finally intercepted based on the classification results includes: based on the SKU information corresponding to each product local sub-graph given in the classification results, screening out the SKU information that is the same as the target SKU information, and retaining the product local sub-graphs that exceed the threshold based on the REID classification confidence value of each product local sub-graph.

[0020] In one embodiment, obtaining the new product replacement image includes: calculating the feature matrix of all product local sub-images in the new product sub-image to be processed database;

[0021] Calculate the cosine distance between each row of the feature matrix of all product local sub-graphs and all other rows; delete the images whose cosine distance is greater than the first cosine distance threshold τ1;

[0022] The remaining pictures whose cosine distance in the local sub-pictures of the product is greater than the second cosine distance threshold τ2 are selected as replacement pictures for the new product, and the pictures are used as comparison pictures for the new product.

[0023] In one embodiment, the verification of the recognition accuracy of the screened new product replacement pictures includes: performing REID classification on the new product reference pictures based on the new product replacement pictures to obtain a first accuracy; performing REID classification on the new product reference pictures based on the new product sample pictures to obtain a second accuracy; when the first accuracy is greater than the second accuracy, the verification is passed.

[0024] Compared with the prior art, one or more embodiments of the present invention may have the following advantages:

[0025] The present invention can quickly replace the sample images of new products that have not been supervised and used by the product recognition model by processing the data of the actual purchase videos of untrained products generated in the actual operation of the smart vending machine, so that the product recognition model can be quickly applied to the accurate recognition of new products. This reduces the workload of the smart vending machine for updating the artificial intelligence algorithm after new products are launched.

[0026] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0028] Figure 1 It is a flowchart of a method for updating artificial intelligence model data of an intelligent vending machine according to a first embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of the process of capturing new product sub-images from an actual purchase video according to the first embodiment of the present invention;

[0030] Figure 3 is a schematic diagram of the verification process flow of the first embodiment of the present invention;

[0031] Figure 4 is a structural block diagram of an electronic device according to a second embodiment of the present invention;

[0032] Figure 5 It is a structural diagram of a computer system of an artificial intelligence model data updating method according to a second embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention may be a system, method and / or computer program product at any possible level of integrated technical detail. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon for causing a processor to perform various aspects of the present invention.

[0034] A computer readable storage medium may be a tangible device that can retain and store instructions used by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes the following: a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination of the above. A computer readable storage medium, as used herein, should not be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted by a wire.

[0035] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0036] The computer-readable program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages ​​and process programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer by any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., by using the Internet of an Internet service provider). In certain embodiments, the electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) may execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit, thereby performing aspects of the present invention.

[0037] Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It is understood that each block of the flowchart illustration and / or block diagram, and the combination of blocks in the flowchart illustration and / or block diagram, can be implemented by computer-readable program instructions.

[0038] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing device to produce a machine so that the instructions executed by the processor of the computer or other programmable data processing device create instructions for implementing the flowcharts and / or block diagrams. Figure 1These computer-readable program instructions may also be stored in a computer-readable storage medium capable of booting a computer, a programmable data processing device and / or other device that operates in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes a manufactured product that includes instructions that implement aspects of the functions / actions specified in the flowchart and / or block diagram blocks.

[0039] Computer readable program instructions may also be loaded into a computer, other programmable data processing device, or other device to cause a series of operating steps to be performed on the computer, other programmable devices or other devices for producing a computer-implemented process, so that the instructions executed on the computer, other programmable device or other device implement the functions / actions specified in the flowchart and / or block diagram blocks.

[0040] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementation architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions indicated in the blocks may occur out of the order indicated in the figure. For example, in practice, two blocks shown in succession may be completed as a step, executed concurrently, substantially concurrently, with partial or full temporal overlap, or the blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block of the block diagram and / or flowchart illustration, as well as combinations of blocks in the block diagram and / or flowchart illustration, may be implemented by a dedicated hardware-based system that performs the specified functions or actions or executes a combination of dedicated hardware and computer instructions.

[0041] Example 1

[0042] like Figure 1 As shown, the artificial intelligence model data updating method of the smart vending machine of this embodiment includes the following steps:

[0043] Step S100, obtaining the actual purchase video of the untrained new product determined by the product recognition model or manual review. The actual purchase video of the untrained new product is generated during the actual operation of the smart vending machine. During the actual operation of the smart vending machine, when the untrained new product is updated to the smart vending machine, the operator of the smart vending machine will provide the algorithm side of the product recognition settlement of the smart vending machine with sample pictures of the untrained new product for the algorithm side to update and train the product recognition model. Therefore, after the algorithm side obtains the sample image of the untrained new product, the entire product image database of the product recognition algorithm model will contain two parts of images. One part is the standard product images (trained_gallery) (i.e., the product standard image gallery) that have completed supervised learning of the algorithm model. These standard images correspond to clear and unique product SKU information. The other part is the new product sample images (extra_gallery) (i.e., the new product sample image gallery) provided by the operator. These new product sample images have not undergone supervised learning of the algorithm model, but each of them corresponds to clear and unique product SKU information (i.e., the product SKU specified by the operator).

[0044] The purpose of this embodiment is to update the image data of the new product sample image library through the data update method of the present invention so that it can be consistent with the images in the product standard image library. Specifically, the new product sample images are not the actual sales behavior images generated by the smart vending machine in actual operation. They are generally pictures taken directly by the operator of the new product. The picture structure and product angle are inconsistent with the video pictures when the actual purchase behavior occurs on the smart vending machine. This will greatly reduce the recognition accuracy when using, for example, the REID model for product recognition in the future. Therefore, the recognition accuracy of the product recognition algorithm can be effectively improved by replacing the new product sample images.

[0045] In the process of obtaining the actual purchase video of the untrained new product determined by the product recognition model or manual review, there are two main ways to generate the actual purchase video of the untrained new product. The first is that the product in the actual purchase video is identified by the product recognition algorithm as an untrained new product SKU, that is, this recognition result is obtained by the product recognition algorithm based on the above-mentioned new product sample image library. The second is the recognition result output by the product recognition algorithm with a low confidence that needs to be manually reviewed, and the result is determined to be an untrained new product through manual review. The actual purchase video of the untrained new product obtained by the above two methods is the original data of the image data updating method of the present invention. It should be noted here that the actual purchase video is marked with the determined target SKU information, that is, whether it is identified by the product recognition model or manually marked by manual review, the actual purchase video will be marked with the product SKU information corresponding to the video, that is, the target SKU information.

[0046] Step S200: extract new product sub-images from the actual purchase video. The process of extracting new product sub-images from the actual purchase video is as follows: Figure 2 shown.

[0047] First, use the target detection model to detect the local area of ​​the product for each frame of the actual purchase video. The local area detection of the product can use the target detection algorithm of the yolo series. Use the target detection algorithm to detect each frame of the actual purchase video of the input untrained new product. If there is a purchased product in the frame image, the target detection algorithm will output the rectangular box bbox of the purchased product. bbox = [(x1, y1), (x2, y2)], where (x1, y1) and (x2, y2) correspond to the upper left and lower right corner coordinates of the rectangular box of the purchased product in the current frame image. It should be noted that when there are multiple products in a frame image, the target detection model will obtain multiple rectangular boxes bbox.

[0048] Subsequently, a binary classification test is performed on all local areas of the goods to delete the non-goods local areas. Using the binary classification detection model, according to the binary classification category and the pre-set screening threshold, it is distinguished whether the currently detected local area of ​​the goods is a non-goods image. The binary classification model used may be VGG, ResNeXt or DenseNet. If it is a non-goods image, the information of the local area of ​​the goods is discarded. The confidence output by the binary classification detection model refers to the possibility that the current image is a product. Only when the confidence exceeds the pre-set threshold, it is considered that the local area of ​​the goods currently detected contains goods for sale.

[0049] Then, all the local areas of the products are intercepted to generate the local sub-images of the products. That is, the local areas of the products retained after the binary classification screening of the local areas of the products are intercepted to generate multiple local sub-images of the products. imgs .

[0050] Then, the first feature matrix (query_feature) of the local subgraph of the product is generated according to the local subgraph of the product. That is, the first feature matrix B corresponding to the local subgraph of the current product is calculated using the reid model. p×n It is worth noting that p is the number of local sub-graphs obtained during detection, and n is the dimension size set in advance when the product recognition algorithm performs supervised learning. Each row of the first feature matrix corresponds to a feature matrix of a local sub-graph of a product.

[0051] Subsequently, a second feature matrix (gallery_feature) about all product images is generated based on all product images, namely, all images in the product standard image library and the new product sample image library, which include both product standard images and new product sample images.

[0052] Use the reid model to calculate all the product images covered by the current order to obtain the second feature matrix A m×n . Where m is the number of images of all sku that the current smart vending machine supports selling. n is the dimension size that the algorithm sets in advance when performing supervised learning. Each row of the matrix corresponds to the eigenvalue of a sku.

[0053] For example: Where m is the number of all pictures in the gallery. i Represents the image address of the i-th product. That is, one sku has multiple product images.

[0054] Next, REID classification is performed based on the first feature matrix and the second feature matrix. That is, the cosine distance between each row of the first feature matrix and the second feature matrix is ​​calculated to obtain a cosine distance matrix C p×m . Get the cosine distance matrix C p×m The minimum coordinate of each column is (i, j), where i is the image index of the current product standard image and j is the image index of the product local sub-image. Since each product standard image has corresponding SKU information, the product SKU classification result of each image in the product local sub-image is obtained.

[0055] Then, the new product sub-graph to be intercepted is determined according to the REID classification result, which includes filtering out the SKU information corresponding to each product partial sub-graph given in the REID classification result, and retaining the product partial sub-graphs that exceed the threshold according to the REID classification confidence value of each product partial sub-graph.

[0056] Finally, the retained product partial sub-graphs are stored in the new product sub-graph to-be-processed database.

[0057] Step S300: When the number of commodity local sub-graphs in the new commodity sub-graph to be processed database exceeds the threshold, the verification of all commodity local sub-graphs in the new commodity sub-graph to be processed database is started. Otherwise, the accumulation of commodity local sub-graphs continues. In this embodiment, the threshold of the number of pictures can be set to 200.

[0058] Step S400: verify the local product sub-graph in the new product sub-graph to be processed database. The verification process is as follows: Figure 3 As shown, including:

[0059] Calculate the feature matrix of all product local subgraphs in the new product subgraph to be processed database. Use the reid model to obtain the feature matrix of all product local subgraphs Where d1 is the number of local subgraphs of the product to be verified.

[0060] Then, delete the misdetected images in the local sub-image of the product, that is, calculate the feature matrix The cosine distance value between each row of the matrix and all other rows, that is, the similarity between the current image and all other images is calculated. Images with cosine distances greater than the first cosine distance threshold τ1 are identified as images with large feature differences and then deleted. In this embodiment, the first cosine distance threshold τ1 is set to τ1 = 0.9.

[0061] Then, among the retained images, the images with cosine distance values ​​greater than the second cosine distance threshold τ2 are selected as new product replacement images (gallery images), and the rest are new product comparison images (query images). At the same time, the feature matrix of the new product replacement images is calculated. Feature matrix of the comparison image of the new product

[0062] Then, merge the feature matrix of all categories that have been supervised by the current reid model and the feature matrix of the new product replacement image Get the third feature matrix for reid classification

[0063] Calculate the third characteristic matrix characteristic matrix A feature matrix with each row comparing images of the new product Previous cosine distance Get The minimum coordinate of each column is (i, j), where i is the image index of the current new product replacement image, and j is the image index of the new product comparison image. Since each new product replacement image has a corresponding sku, the classification results of each image in the new product comparison image are obtained.

[0064] Calculate the accuracy of the classification result acc1 using the new product replacement image as the benchmark. The accuracy calculation formula is as follows: acc = accurate classification number / number to be identified, where the accurate classification number refers to the number of target sku to be verified for the classification result of the current new product reference image.

[0065] Using the new product sample pictures, calculate the feature matrix F of the new product sample pictures p×n , p is the number of new product sample images. Calculate the feature matrix F of new product sample images p×n A feature matrix with each row comparing images of the new product Previous cosine distance Get The minimum coordinate of each column is (i, j), where i is the image index of the current new product replacement image, and j is the image index of the new product comparison image. Since each new product replacement image has a corresponding sku, the classification results of each image in the new product comparison image are obtained.

[0066] Calculate the classification accuracy acc2 using the new product sample image as the benchmark.

[0067] When the accuracy acc1 is greater than the accuracy acc2, the verification is deemed to have passed. If the accuracy acc1 is less than the accuracy acc2, the verification fails.

[0068] In step S500 and step S600, when the verification is passed, the above-screened new product replacement pictures are replaced into the original new product sample picture library to replace the original product sample pictures.

[0069] Step S700: If the verification fails, the verification information is returned, and the verification is performed again after the product partial sub-graph in the new product sub-graph to be processed database is updated.

[0070] Example 2

[0071] This embodiment also discloses an electronic device, Figure 4 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Figure 4 As shown, the electronic device 100 includes a memory 101 and a processor 102; wherein,

[0072] The memory 101 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 102 to implement the above method steps.

[0073] Figure 5 It is a structural diagram of a computer system suitable for implementing an artificial intelligence model data updating method according to an embodiment of the present disclosure.

[0074] like Figure 5 As shown, the computer system 200 includes a processing unit 201, which can perform various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 202 or a program loaded from a storage part 208 into a random access memory (RAM) 203. In the RAM 203, various programs and data required for the operation of the system 200 are also stored. The processing unit 201, the ROM 202, and the RAM 203 are connected to each other through a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204.

[0075] The following components are connected to the I / O interface 205: an input section 206 including a keyboard, a mouse, etc.; an output section 207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN card, a modem, etc. The communication section 209 performs communication processing via a network such as the Internet. A drive 210 is also connected to the I / O interface 205 as needed. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 210 as needed, so that a computer program read therefrom is installed into the storage section 208 as needed. Among them, the processing unit 201 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0076] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a readable medium thereof, and the computer program includes a program code for executing the data inspection method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 209, and / or installed from the removable medium 211.

[0077] The embodiment of the present disclosure also discloses a computer program product, which includes a computer program / instruction, and when the computer program / instruction is executed by a processor, any of the above method steps is implemented.

[0078] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the road map or block diagram can represent a part of a module, program segment or code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a different order from the order marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0079] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0080] As another aspect, the embodiments of the present disclosure further provide a computer-readable storage medium, which may be a computer-readable storage medium included in the device described in the above-mentioned embodiment; or a computer-readable storage medium that exists independently and is not assembled into the device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the methods described in the embodiments of the present disclosure.

[0081] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A method for updating artificial intelligence model data for a smart vending machine, characterized in that: The method comprises the following steps: An actual purchase video containing untrained new products determined by a product recognition algorithm or manual review is obtained; the product recognition algorithm recognizes the product SKU in the actual purchase video based on standard pictures of products that have undergone supervised learning and sample pictures of new products that have not undergone supervised learning; the actual purchase video is annotated with target product SKU information. Capturing new product sub-images from the actual purchase video; and storing the product partial sub-images in a new product sub-image to-be-processed database; When the number of commodity local subgraphs in the new commodity subgraph to-be-processed database exceeds a threshold, the verification of all commodity local subgraphs in the new commodity subgraph to-be-processed database is initiated; Screening the local sub-images of the products in the new product sub-image to be processed database to obtain new product replacement images, and verifying the recognition accuracy of the screened new product replacement images; After the verification is passed, the above-screened new product replacement images are replaced into the original new product sample image library to replace the product sample images of the untrained products in the product recognition algorithm.

2. The artificial intelligence model data updating method according to claim 1, characterized in that: The actual purchase video containing untrained new products determined by the product recognition algorithm or manual review means: the products in the actual purchase video identified by the product recognition algorithm are untrained new products; or, the recognition results output by the product recognition algorithm have low confidence and need to be manually reviewed, and the recognition results are determined to be untrained new products through manual review.

3. The artificial intelligence model data updating method according to claim 1, characterized in that: The extracting of new product sub-images from the actual purchase video includes: using a target detection model to detect a local area of ​​the product on each frame of the actual purchase video; Perform binary classification on all product local areas to remove non-product local areas; The reserved local area of ​​the product is intercepted to generate a local sub-graph of the product; Generate a feature matrix of the local sub-graph of the product according to the local sub-graph of the product; and generate a feature matrix of all product pictures according to all product pictures; the all product pictures are the standard pictures of the product and the sample pictures of the new product; Perform REID classification based on the feature matrix of the local sub-graph of the product and the feature matrix of all product images to obtain the product SKU classification result of each image in the local sub-graph of the product; The new product sub-graph to be finally intercepted is determined based on the classification results.

4. The artificial intelligence model data updating method according to claim 1, characterized in that: The method of determining the new product sub-graph to be finally intercepted based on the classification result includes: according to the SKU information corresponding to each product local sub-graph given in the classification result, the SKU information that is the same as the target SKU information is screened out, and at the same time, according to the REID classification confidence value of each product local sub-graph, the product local sub-graph that exceeds the threshold is retained.

5. The artificial intelligence model data updating method according to claim 1, characterized in that: The obtaining of the new product replacement image includes: calculating the feature matrix of all product local sub-images in the new product sub-image to be processed database; Calculate the cosine distance between each row of the feature matrix of all product local sub-graphs and all other rows; delete the images whose cosine distance is greater than the first cosine distance threshold τ1; The remaining pictures whose cosine distance in the local sub-pictures of the product is greater than the second cosine distance threshold τ2 are selected as replacement pictures for the new product, and the pictures are used as comparison pictures for the new product.

6. The artificial intelligence model data updating method according to claim 1, characterized in that: The verification of the recognition accuracy of the screened new product replacement pictures includes: performing REID classification on the new product reference pictures based on the new product replacement pictures to obtain a first accuracy; performing REID classification on the new product reference pictures based on the new product sample pictures to obtain a second accuracy; when the first accuracy is greater than the second accuracy, the verification is passed.

7. An electronic device comprising a memory and a processor; wherein: The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having computer instructions stored thereon, wherein: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer program product, comprising a computer program / instruction, which implements the steps of the method according to any one of claims 1 to 6 when executed by a processor.

Images