A method and device for identifying product information in a cold and warm cabinet

Abstract

The present invention relates to a method and device for identifying product information in a cold and warm cabinet, belonging to the technical field of artificial intelligence data processing. It includes collecting through photo information, gravity sensor information, and offline business supplementary calibration information in the cold and warm cabinet; screening, filtering, summarizing, and fusing the collected information, extracting a feature data set of relevant information elements, processing it through feature engineering technology, and inputting it into each sub-module of the trainer for training to obtain an engine with information recognition ability; using the result output by the engine as a decision data set, classifying and inputting it into the decision classification tree and decision prediction tree models to obtain a model decision result that can be directly applied in practice; then integrating business budgets, market changes, personnel assessments, order records, etc. into the information learning and recommendation device, and combining with the model decision result, a multi-functional practical commercial value result such as data warning, performance prediction, data statistics, product recommendation score, and recommendation reason can be obtained.

Description

Technical Field

[0001] The present invention relates to a method and device for identifying product information in a cold and warm cabinet, belonging to the technical field of artificial intelligence data processing. Background Art

[0002] In traditional offline retail, cold and warm cabinets are often used to display products and keep the displayed products refrigerated or warm. Cold and warm cabinets are also one of the heavy assets of traditional merchants or enterprises. The equipment placement, equipment inventory, equipment maintenance, and equipment depreciation account for a large part of asset management. The placement, inventory, maintenance, and depreciation of cold and warm cabinets vary in different channels (such as chain supermarkets, hypermarkets, grocery stores, schools, stadiums, gas stations), and the supervision difficulty in different channels is different. Especially in winter and summer, more manpower and material resources may be required to maintain these equipment and products. To ensure that indicators such as the display saturation of products in the equipment, the product missing rate, the freshness of product age, and the number of product SKUs are within the normal supervision value range.

[0003] The traditional offline resource investment and supervision are time-consuming and laborious, which may cause first-level agents, distributors, wholesalers, or business teams to need to put in more effort for their work, and the expenditure will also increase the cost. For the identification of product information in cold and warm cabinets, image recognition and data processing technologies have not been widely and effectively applied. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and device for identifying product information in a cold and warm cabinet, which can overcome or at least partially solve the above problems of product identification and processing in a cold and warm cabinet based on picture recognition.

[0005] The technical solution of the present invention is: A method for identifying product information in a cold and warm cabinet, the specific steps are as follows:

[0006] Step1: Obtain the picture information, gravity sensor information, and business supplementary calibration information in the cold and warm cabinet, and these data information are summarized by the information collector;

[0007] Step1.1: Regularly collect photos inside the cabinet through a camera, including a regular picture set and an occlusion picture set;

[0008] Step1.2: Measure the data of the weight change of each layer according to the plan through the gravity sensor, extract the weight change data of the products on each layer of the cabinet body, and convert it into data information with characteristic quantities;

[0009] Step1.3: Through business supplementary calibration information, including cabinet body appearance inspection, cabinet body equipment identification, product occlusion supplement inside the cabinet, product age information entry, etc.;

[0010] Step 1.4: Collect the information of the three parties through the collector. If the data set is incomplete, a reminder will be triggered to request the completion of the relevant information. If it is not complete, it will be determined as an NA value.

[0011] Step 2: Filter, summarize, and integrate the collected data, extract the feature data set of relevant information elements, process it through feature engineering technology, and input it into each module of the trainer for training to obtain an engine with information recognition ability.

[0012] Step 2.1: Filter and screen the collected information, remove information such as poor lighting, blurred pictures, and obvious missing abnormalities, and summarize and integrate the information described in Step 1.4.

[0013] Step 2.2: Extract relevant information elements as the feature data set through technologies such as OCR technology and jieba word segmentation.

[0014] Step 2.3: Process it through feature engineering technologies such as timestamp processing, category decomposition, binning operation, cross-validation, feature scaling, and feature extraction, and input it into each module of the trainer for training. The training includes: cabinet space training, rule atlas training, OCR text set training, sku training, layer weight training, occlusion atlas training, etc.

[0015] Step 2.4: Train to obtain an engine with information recognition ability. The output capabilities of the engine include: identifying the product category, the number of products, the product age, the purity of each layer of products, etc. in the cold and warm cabinets.

[0016] Optionally, the sub-modules include: M = {m 1 , m 2 , m 3 , m 4 , m 5 , m 6 , m 7 ...} where m 1 is the product shape; m 2 is the product color; m 3 is the number of products; m 4 is the number of shelves; m 5 is the product name; m 6 is the product merchant; m 7 is the product age.

[0017] Step 3: Input the decision data set of the engine into the decision classification tree and decision prediction number model by category to obtain a directly applicable decision result.

[0018] Step 3.1: Regard the result output by the engine as the decision data set and screen it for the classification tree or prediction tree.

[0019] Step3.2: Input them into the decision classification tree and decision prediction tree models by category;

[0020] Step3.3: Obtain the model decision results that can be directly applied in practice;

[0021] Optionally, if there are too many or too few model decision branch nodes, or the AUC (Area Under Curve) value is low and does not meet the business requirements, pre-pruning, post-pruning techniques can be adopted, or algorithms such as ID3, C4.5, and C5.0 can be adjusted. It is also possible to go back to Step2, apply advanced feature engineering processing methods and default value processing, or optimize the model parameters using K-fold Cross-Validation until the model has good performance.

[0022] Step4: Integrate business budgets, market changes, personnel assessments, order records, etc. into the information learning recommender. Combining with the model decision results, actual business value results with multiple functions such as data warning, performance prediction, data statistics, product recommendation scores, and recommendation reasons can be obtained.

[0023] Step4.1: Integrate information such as business budgets, market changes, personnel assessments, order records, etc. into the information learning recommender in modules;

[0024] Step4.2: Combine with the manual judgment of experts within the business, filter out the recommended results with low value or uselessness, and give a more reasonable explanation for the recommended results. The explanatory language extracts a piece of information in one sentence through natural language processing technology as the recommendation reason;

[0025] Step4.3: The learning recommender will finally obtain actual business value results with multiple functions composed of data warning, performance prediction, data statistics, product recommendation scores, recommendation reasons, etc.;

[0026] Optionally, if there are differences in model interpretations among multiple people, a voting method or average value and weight value method can be optionally adopted for determination. For model results such as tied votes, no votes, and votes not exceeding 2 / 3 of the total votes, they can be discarded.

[0027] Optionally, according to the set of business value results, it can be specifically divided into different units, including: the detailed list unit of the AI warning module, the historical performance achievement unit, the future performance prediction unit, the gross profit prediction unit, the product out-of-stock replenishment recommendation unit, the cabinet miscellaneous improvement reminder unit, the product recommendation score unit, the data visualization chart unit, and the AI cabinet total score unit.

[0028] A product identification device inside a cold and warm cabinet, comprising:

[0029] A gravity sensor for collecting the weight value of each layer inside the cabinet;

[0030] A camera inside the cabinet is used to take pictures of the product information inside the cabinet and the overall situation of the cabinet.

[0031] A memory is used to store the required programs.

[0032] A processor is used to execute the required programs, and the programs cause the processor to execute the product information recognition method based on the cold and warm cabinet.

[0033] The beneficial effects of the present invention are as follows:

[0034] The present invention combines data processing technology with the business of cold and warm cabinet products, and has the following advantages:

[0035] 1. A relatively complete data chain collector that fully considers consumer privacy and avoids unnecessary privacy exposure.

[0036] 2. A multi-module ordered trainer. Through the examples, multiple trainers and sub-modules can be selected and trained in an orderly manner.

[0037] 3. A multi-task decision maker, a decision tree model with direct output of classification, which has direct reference value for business classification management.

[0038] 4. A recommender with commercial value and operation management. By combining multiple factors, the commercial value results recommended by the system can be obtained better. The commercial purpose of reducing operation management costs and improving efficiency is achieved. Description of the Drawings

[0039] Figure 1 is a flowchart of the steps of the recognition method of the present invention;

[0040] Figure 2 is a swimlane activity diagram in an embodiment of the present invention;

[0041] Figure 3 is a structural diagram of the recognition device of the present invention. Detailed Embodiments

[0042] The present invention will be further described below in conjunction with the drawings and specific embodiments. Although the embodiments of the present disclosure are shown in the drawings, it should be noted that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, the provided embodiments are only for better understanding the present invention and for fully communicating the scope of the disclosure to those skilled in the art.

[0043] In order to enable those skilled in the art to better understand the present invention, the following further explains the concepts involved in the application:

[0044] The data information includes information such as picture LOGO, advertisement text, product table, one-dimensional code (or two-dimensional code), date, etc. The data information can be one or more of the above information forms, but at least one information should be ensured to avoid the data information in the cold and warm cabinet being NA value, which may cause system anomalies. The embodiments of the present invention application do not limit this.

[0045] The feature engineering data information includes a data set representing commodity information elements, or a data set representing the overall space inside the cabinet. The description or connotative information volume of the information elements can be regarded as a kind of feature data. For example, the product taste and capacity information described by the text information elements are a kind of descriptive information. The text keywords can be obtained by using the textCNN (Text Convolutional Neural Network) model. The extraction of date text can be automatically realized by using the OCR (Optical Character Recognition) technology. The expected effect can be achieved by borrowing the paddleocr library. Among them, the keywords can better display the information described by the text information. The present application does not limit the technologies for extracting information such as text.

[0046] Between the same kind of information elements, certain interference or information asymmetry differences are often caused by factors such as the shooting time, shooting angle, shooting light, and shooting personnel. To screen out or overcome this difference, manual preliminary screening can be adopted. For example, re-upload the information with poor light or blurred shooting. Or a systematic method can be used for comparison and screening. For example, for the differences between the captured picture information, the picture information is often represented by the vector representation method. The cosine similarity can be calculated to obtain the difference. The cosine value of the two picture information vectors can be calculated to evaluate the difference between the two.

[0047] The cosine similarity formula is:

[0048]

[0049] Where A and B are two n-dimensional vectors. A is [A 1 , A 2 , …, A n , and B is [B 1 , B 2 , …, B n , and θ is the included angle between A and B.

[0050] Of course, another method can be to calculate it using the Pearson correlation coefficient. Calculating the covariance or standard deviation of the two vectors can evaluate their correlation. The present application does not limit the algorithm technologies used for information comparison.

[0051] The Pearson correlation coefficient formula is:

[0052]

[0053] The Pearson correlation coefficient ρ is used to measure the degree of linear correlation between two variables (X and Y), and its value ranges from -1 to 1. This linear correlation is intuitively expressed as whether Y increases or decreases simultaneously as X increases; when the two are distributed on a straight line, the Pearson correlation coefficient is equal to 1 or -1; when there is no linear relationship between the two variables, the Pearson correlation coefficient is 0.

[0054] In an alternative embodiment of the present application, the differences between similar information elements can be normalized to the range of 0 - 1. The closer to 1, the more similar the two information elements are, and the closer to 0, the greater the difference and the lack of correlation between the two information elements. Of course, it should be noted that words or symbols can also be used to represent this difference. For example, product images taken from completely different sides are represented by level 10, while pictures taken from the same side are represented by level 0. The present invention does not limit the method for selecting differences.

[0055] The information content represented by content information and picture information often overlaps. To understand the relationship between picture information elements and content text information, keyword extraction operations can be performed on the text information. An optional algorithm can be One - Hot encoding, and then obtained through jieba word segmentation. Picture information elements can be processed for classification information. For example, for a picture of a product logo, to determine which product type it belongs to, the method can be to query in an application system or a database system, such as classifying it as water, beverage, coffee, food, etc. The specific classification method and how to query are not limited in the present invention.

[0056] In the extraction of product age information, in the fast - moving consumer goods industry or industries with high liquidity and short shelf - lives, the principle of "first - in, first - out" is generally adopted for product age control. Due to its high liquidity, the cold storage cabinet is often one of the main forms of product display. Therefore, the market performance of product age can be roughly predicted based on this. The method and strategy that can be selected are: encode the age information of each sku (Stock Keeping Unit) product into a one - dimensional code (or two - dimensional code), extract the product id value from the one - dimensional code (or two - dimensional code), and then query the corresponding production date and shelf - life information in the logistics system or database system. The extraction of this date text can be automatically achieved using OCR (Optical Character Recognition) technology. Using the paddleocr library can achieve the expected effect, which is convenient for studying the market manifestation form of product age. The present invention is applicable but not limited to the method of extracting product age information.

[0057] As shown Figure 1 in the figure, a method for identifying product information in a cold and warm cabinet body, the specific steps are as follows:

[0058] Step1: Obtain the picture information, gravity sensor information, and business supplementary calibration information in the cold and warm cabinet body, and these data information are summarized by the information collector;

[0059] Step2: Filter and summarize the collected data, extract the feature data set of relevant information elements, process them through feature engineering technology, and input them into each module of the trainer for training to obtain an engine with information recognition ability;

[0060] Step3: Use the result output by the engine as the decision data set, and classify and input it into the decision classification tree and decision prediction tree models to obtain the model decision result that can be directly applied on the ground;

[0061] Step4: Integrate business budgets, market changes, personnel assessments, order records, etc. into the information learning and recommendation device, and combine with the model decision result to obtain the actual commercial value results with multiple functions such as data warning, performance prediction, data statistics, product recommendation scores, and recommendation reasons.

[0062] As shown Figure 2 in the figure, a product information recognition system in a cold and warm cabinet body. The departments corresponding to the swimlanes can of course also be one or several organizations, one or more roles, and those skilled in the relevant technology in this technical field can modify it accordingly. The present invention does not fix its role. It should be noted that the above example is only one form.

[0063] The first swimlane is the data collection department. It mainly collects pictures. One is that the data collector in the cabinet body regularly collects data information sources, and the other is that the relevant offline business personnel manually collect and manually upload picture information on the APP.

[0064] The second swimlane is the data training department. In this process, it mainly fuses, cleans, classifies, transforms through feature engineering, and trains sub-module models on the data obtained from the first swimlane.

[0065] The third swimlane is the data decision department. In this activity, it mainly models the decision prediction tree and decision classification tree on the training result set of the second swimlane. Then, combined with the actual business situation, it conducts model interpretation and application.

[0066] The fourth swimlane is the commercial value recommendation department. In this activity, it mainly conducts more detailed interpretation and application on the decision result of the third swimlane. Before application recommendation, it comprehensively assesses the company's budget target, strategic plan, personnel assessment, and market change calibration factors, and directly recommends it to the actual combat-effective activity execution.

[0067] like Figure 3 As shown, a device for identifying product information in a cooling and heating cabinet includes:

[0068] Camera information collector inside the cabinet and gravity sensor on each floor.

[0069] The present invention uses a camera inside the cabinet, which is located at the door handle. The camera is aimed at the cabinet. When the door opening angle is greater than 20°, the camera collector is triggered to work; if the cabinet door is closed for a long time, and exceeds 4 hours, the camera collector will be triggered to work. When the cabinet door is greater than 30°, the power of the door opening and closing is applied, and the camera cover will be pulled down, automatically blocking and disconnecting the camera to avoid the collection of personal privacy information.

[0070] Considering the protection of consumers' personal privacy, the invention also proposes a device application based on gravity sensor. To prevent consumers' facial and fingerprint information from being unintentionally collected when they open and close the cabinet door, the invention adds a pressure sensor under each shelf. This sensor is based on the principle of weight change of products placed on each shelf. Assuming that a certain shelf is required to place a certain type or multiple types of product sku weights, the gravity of each type of sku is w i To express it, the relationship between the weight displayed by the pressure sensor of the shelf and the items is linear. It can be expressed by the following data expression:

[0071] W=w 0 +x 1 w 1 +x 2 w 2 +x 3 w 3 +...

[0072] w 0 is the weight of the layer support itself. i The weight of a product placed on this layer. x i is the number of products (or bottles). Then the pure weight of the layer w p =n(Ww 0 ), n=0,1,2,3... indicates that the tare weight of the freezer on this layer is a linear combination of the specified products. When n=0, it indicates that the shelf products are empty. Pure means that the layer is filled with the specified product set without other miscellaneous items.

[0073] The number of miscellaneous bottles placed on this product layer can be set according to the budget. i The numerical value is combined with the results collected at multiple time periods within a certain period of time to solve the simultaneous equations for x i , whether it is pure can be determined by (Ww0 ) / w i The result is determined. Furthermore, the saturation of this layer, the number of product SKUs in this layer, and the purity of this layer are calculated.

[0074] For the overall indicators of the cold and warm cabinets, they can be obtained by summarizing each layer, using or W j one or more logical connective relationships such as conjunction, disjunction, and negation to obtain.

[0075] A camera inside the cabinet is used to take pictures of the product information inside the cabinet and the overall situation of the cabinet.

[0076] A memory is used to store the required programs.

[0077] A processor is used to execute the required programs, and the programs cause the processor to execute the method for identifying product information in the cold and warm cabinets described above.

[0078] The specific embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those of ordinary skill in the art, various changes can be made without departing from the gist of the present invention.

[0079] Although some embodiments are for the purpose of illustration and description, various alternatives, and / or equivalent implementation schemes or calculations are shown and described to achieve the same purpose as the embodiments described. Without departing from the scope of the embodiments of the present application. The present application aims to cover any modifications or variations of the embodiments discussed herein. Therefore, it is obvious that the embodiments described herein are only defined by the claims and their equivalents.

Claims

1. A method for identifying product information in a cooling and heating cabinet, Features: Step 1: Obtain the picture information inside the cold and warm cabinet, the gravity sensor information, and the business supplement calibration information, including at least the cabinet appearance inspection, cabinet equipment identification, cabinet product shielding supplement, and product age information entry, and summarize all information data through the information collector; Step 2: The collected data is filtered, aggregated and integrated to extract the feature data set of relevant information elements, processed by feature engineering technology, and input into the training modules of the trainer for training. The training includes: cabinet space training, rule atlas training, OCR text set training, sku training, layer weight training, and occlusion atlas training. The engine with information recognition capabilities is obtained. The engine output capabilities include: identifying the product category, product number, product age, and purity of each layer of products in the cold and warm cabinets; Step 3: The results output by the engine are used as decision data sets, and are classified and input into the decision classification tree and decision prediction tree models to obtain the model decision results for direct application; Step 4: Integrate business budgets, market changes, personnel assessments, and order records into the information learning recommender, and combine them with the model decision results to obtain a multifunctional actual business value result set with data warnings, performance forecasts, data statistics, product recommendation scores, and recommendation reasons.

2. The method for identifying product information in a cooling and heating cabinet according to claim 1, Features The Step 1 is specifically as follows: Step 1.1: Use the camera to regularly collect photos inside the cabinet, including regular photo sets and blocked photo sets; Step 1.2: Use the gravity sensor to measure the weight change data of each layer as planned, extract the weight change data of the products on each layer of the cabinet, and convert it into data information with characteristic quantities; Step 1.3: Supplement calibration information through business, including at least cabinet appearance inspection, cabinet equipment identification, cabinet product shielding supplement, and product age information entry; Step 1.4: Collect the three information through the collector. If the data set is incomplete, it will trigger a reminder and require the relevant information to be completed. If it is incomplete, it will be judged as NA value.

3. The method for identifying product information in a cooling and heating cabinet according to claim 2, Features The Step 2 is specifically as follows: Step 2.1: Filter the collected information, remove the information with poor lighting, blurred images, and obvious missing abnormalities, and summarize and integrate the information described in Step 1.4; Step 2.2: Extract relevant information elements as feature data sets through OCR technology and Jieba word segmentation technology; Step 2.3: Feature engineering such as timestamp processing, category decomposition, binning, cross-validation, feature scaling, feature extraction, etc. Technical processing, and input into the training module of the trainer for training, including: cabinet space training, rule atlas training, OCR text set training, sku training, layer weight training, occlusion atlas training; Step 2.4: Train an engine with information recognition capabilities. The engine output capabilities include: identifying the product categories in the cold and warm cabinets, the number of products, the age of the products, and the purity of the products on each layer.

4. The method for identifying product information in a cold and warm cabinet according to claim 1, characterized in that the specific content of Step 3 is as follows: Step 3.1: Use the result output by the engine as the decision data set and screen it for the classification tree or prediction tree; Step 3.2: Input it into the decision classification tree and decision prediction tree models according to the categories; Step 3.3: Obtain the model decision result that can be directly applied in practice.

5. The method for identifying product information in a cold and warm cabinet according to claim 1, characterized in that the specific content of Step 4 is as follows: Step 4.1: Integrate information such as business budget, market changes, personnel assessment, and order records into the information learning recommender in modules; Step 4.2: Combine the manual judgment of experts within the business to filter out the less valuable or useless recommendation results, and give a more reasonable explanation for the recommendation results. The explanatory language extracts a piece of information in one sentence through natural language processing technology as the recommendation reason; Step 4.3: The information learning recommender will finally obtain a multi-functional actual business value result set composed of data warning, performance prediction, data statistics, product recommendation score, recommendation reason, etc.

6. A device for executing the method for identifying product information in a cold and warm cabinet according to any one of claims 1-5, characterized in that it includes: a gravity sensor for collecting the weight value of each layer in the cabinet; a camera in the cabinet for taking pictures of the product information in the cabinet and the overall situation of the cabinet; a memory for storing the required programs; a processor for executing the required programs, and the programs cause the processor to execute the method for identifying product information based on the cold and warm cabinet.

Images