A method for automatically updating large-screen electronic tags in supermarkets with verification function

Abstract

The present invention discloses a method for automatically updating a large-screen electronic label in a supermarket with a verification function, comprising: constructing an electronic label data table; collecting shelf images; identifying product information; processing the placement area; splicing label images; and displaying label images. The present invention can automatically adjust the display image of the large-screen electronic label according to the real-time status of the goods on the shelf, and can realize automatic verification of the update result in combination with the repeated price tags of some goods. The present invention greatly reduces the time for manual adjustment and confirmation of product information, and the verification result has high reliability, so that the product type observable by the user is consistent with the price tag display, thereby improving the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of large-screen electronic labels for supermarkets, and in particular to an automatic updating method for large-screen electronic labels for supermarkets with a verification function. Background Art

[0002] With the popularization of electronic tags, in some supermarkets and physical stores, the original paper price tags have been completely replaced with more convenient electronic tags. When the price of a product needs to be adjusted, it is only necessary to send the adjusted product price image to the corresponding electronic tag unit to achieve the price update within the entire supermarket. The size and application range of electronic tags are also becoming more and more extensive. There are small and medium-sized price tags for a single product, and there are also large-sized price tags for multiple products. Large-sized price tags usually contain large-screen displays of more than 10 inches, which can display more and richer product information and enhance the user's visual experience. There is no shortage of large-screen display tags of more than 70 inches on the current market.

[0003] Currently, most electronic tags are updated through manual background processing, and a small number of research programs propose to automatically update electronic tags after identifying the goods through some reading devices.

[0004] The invention with application number 201810299344.5 published on October 18, 2019 discloses a method for automatic identification of commodity information, which can automatically adjust the commodity information on the electronic shelf label in combination with the technology of automatic commodity identification, greatly improving work efficiency and avoiding inconsistent information caused by wrong shipment of commodities. However, the application adopts RFID reader to realize the identification of commodity information, and can only update the electronic label corresponding to a single commodity. Once the electronic label corresponds to multiple commodities and the location area occupied by the commodity changes, the automatic update of the price label cannot be realized.

[0005] The invention with application number 202011132012.1 published on August 24, 2021 discloses an object information processing method and device, which scans and captures the initial image of the object storage device, then splices the initial image to generate a spliced ​​image, and finally recognizes the spliced ​​image to determine the object information stored in the object storage device. However, this invention is aimed at the problem of whether the various goods and labels on the shelves are placed correctly, and does not involve the price update of electronic tags, let alone the automatic update of price tags of multiple commodities.

[0006] Currently, there is no method for automatically updating the price tags of large-screen electronic tags that display multiple commodities at the same time. Summary of the invention

[0007] In view of the deficiencies in the prior art, the present invention provides a method for automatically updating large-screen electronic labels in supermarkets with a verification function, which can automatically adjust the display image of the large-screen electronic labels according to the real-time status of the goods on the shelves, and can automatically verify the update results in combination with the repeated price tags of some goods. The present invention greatly reduces the time for manual adjustment and confirmation of product information, and the verification results are highly reliable, so that the product type and price tag display that the user can observe are consistent, thereby improving the user experience.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A method for automatically updating a large-screen electronic label in a supermarket with a verification function, the method comprising the following steps:

[0010] S1, construct the electronic tag data table: combine the actual geographic information of the supermarket to build a 3D model map of the supermarket, which includes shelf A i , Unit Goods Area Aisles and electronic tags An electronic tag data table is constructed; the electronic tag data table is used to store the number, orientation, location information of each electronic tag, and the shelf number corresponding to each electronic tag; i=1, 2, ..., I, I is the total number of shelves; It is shelf A i The total number of unit commodity areas included in the above; It is shelf A i The total number of corresponding electronic tags;

[0011] S2, collect shelf images: combine the electronic tag orientation and aisle location to calculate the The shooting direction angle range is obtained by scanning and shooting each electronic tag regularly along the calculated shooting direction angle range. The corresponding shelf image;

[0012] S3, identifying product information: integrating the unit product areas on the shelf image to generate a number of product placement areas; identifying the product category placed on each integrated product placement area and marking the product placement area and product category, and constructing a data table of the relationship between the electronic tag and the product placement area;

[0013] S4, placement area processing: according to the frequency of occurrence of the product placement area, the product placement area corresponding to only one electronic tag is defined as the first placement area, and the remaining product placement areas are defined as the second placement areas, and the main label of each second placement area is determined according to the image area ratio;

[0014] S5, splicing label images: processing the marked shelf image again, retrieving the product label image corresponding to each product placement area, splicing the retrieved product label images according to the positional relationship of each product placement area on the shelf image, and generating an electronic label price tag; wherein, the label image of the second placement area other than the main label is scaled according to the area occupied by the second placement area on the shelf image;

[0015] S6, display label image: send the spliced ​​electronic label price tag to the corresponding electronic label for display.

[0016] To optimize the above technical solutions, the specific measures taken also include:

[0017] Furthermore, in step S1, the position of the electronic tag and the orientation of each electronic tag are calculated in combination with the shelf and aisle position information.

[0018] Furthermore, the process of calculating the position of the electronic tag and the orientation of each electronic tag by combining the shelf and aisle position information includes the following steps:

[0019] S11, calculate each shelf A by combining the shelf and aisle location information i Aisle perspective and aisle area Aisle viewing angle of each shelf and aisle area Calculate each shelf A i Total perspective and total viewing area

[0020] S12, calculate the position and orientation of the electronic tag on each shelf according to the following formula:

[0021]

[0022] st.P1:

[0023] P2:

[0024] P3:

[0025] Where P1 is the installation height restriction of the electronic tag. It is shelf A i The larger value between the maximum theoretical height after placing the product and the user's eye level height; ΔH max It is shelf A i The maximum height difference between the electronic tag and the product is related to the orientation of the electronic tag; P2 is the display viewing angle restriction condition of the electronic tag. It is an electronic tag Permitted covered shelves A i Maximum angle range, each shelf A i Aisle perspective One of the electronic tags P3 is the display coverage limit of the electronic tag. It is an electronic tag Shelves allowed to be covered A i Maximum viewing area, each shelf A i Aisle area One of the electronic tags The maximum viewing angle area covered.

[0026] Furthermore, in step S3, the process of identifying product information includes the following sub-steps:

[0027] S31, collecting a number of product image samples contained in the supermarket to construct a training set, where the product image samples include stacked image samples of multiple similar products, as well as a full set of image samples of packaged products and single product image samples;

[0028] S32, building a commodity recognition model based on a neural network, and training the commodity recognition model using a training set;

[0029] S33, preliminarily dividing the shelf image into unit product areas, performing similarity judgment on the images of adjacent unit product areas, and considering the adjacent unit product areas with similarity higher than a similarity threshold as the same product placement area; preliminarily integrating the unit product areas on the shelf image to generate a plurality of product placement areas, and marking each product placement area;

[0030] S34, importing the image of each product placement area into the product recognition model to identify the type of product in each product placement area.

[0031] Furthermore, in step S34, when one of the product placement areas contains more than two products, the product types contained therein are matched with the product types of the adjacent product placement areas. If all product types are matched successfully, the product placement area is cleared; otherwise, a corresponding product placement area is generated according to the product types that failed to match; the adjacent product placement areas refer to product placement areas that are within the coverage of the same electronic tag display.

[0032] Furthermore, in step S5, the process of stitching label images includes the following sub-steps:

[0033] S51, reading the product placement area number and the product type on each product placement area contained in the current collection cycle for each electronic tag, and retrieving the product label image data corresponding to each product placement area according to the product type;

[0034] S52, generating a label module according to the position information of the unit product area occupied by each product placement area, wherein the label template includes a plurality of label filling boxes, and each label filling box corresponds to a product placement area;

[0035] S53, determining whether the electronic label corresponds to a second placement area other than the main label, and if so, scaling the corresponding label filling frame within a preset image size range according to the area occupied by the second placement area on the shelf image;

[0036] S54, filling the retrieved product label image data into the corresponding label filling box to generate an electronic label price tag within the current collection cycle.

[0037] Furthermore, the automatic updating method comprises the following steps:

[0038] S7, collecting label images: collecting display images of multiple electronic labels in the same second placement area;

[0039] S8, verifying the label image: based on the correspondence between the second placement area and the electronic label, verifying the position information and proportion information of the second placement area on the plurality of electronic labels.

[0040] Furthermore, in step S8, the process of verifying the label image includes the following sub-steps:

[0041] S81, combining the electronic tags and the product placement area relationship data table, collecting the display image data set of the electronic tags in the same second placement area a And the original collection of shelf image data sets

[0042] S82, according to the position information of the unit product area occupied by the second display area a, from the display image data set of the electronic tag The corresponding image data set of the second placement area a is extracted from (m)};

[0043] S83, determine the image data set {a (m)} whether the product label image data in is complete, if complete, proceed to step S84, otherwise, determine that the update fails and end the process;

[0044] S84, calculating the second placement area a in the shelf image data set The scale ratio set of the label filling box of the second display area a is calculated by calculating the area occupied by each shelf image in

[0045] S84, according to the scale set For the data image data set {a (m)} to obtain the scaled validation set {b (m)}, Determine the validation set {b (m)}, if they are consistent, go to step S85, otherwise, determine that the update fails and end the process;

[0046] S85, for the validation set {b (m)} and the similarity between the product price tag of the corresponding product on the second display area a. If the similarity is higher than a preset similarity threshold, the update is judged to be successful; otherwise, the update is judged to have failed.

[0047] The beneficial effects of the present invention are:

[0048] First, the automatic updating method of the large-screen electronic label in supermarkets with a verification function of the present invention can automatically adjust the display image of the large-screen electronic label according to the real-time status of the goods on the shelf; and based on the user's visual range, the user can obtain the price information of the type of goods (including misplaced goods) within the visual range at any angle, thereby improving the user experience. The present invention is particularly suitable for updating the price signs in the promotion area where there are only a few kinds of goods on the shelf and the price signs change frequently.

[0049] Second, the automatic updating method of the large-screen electronic label in supermarkets with a verification function of the present invention can realize automatic verification of the update results in combination with the repeated price labels of some commodities, and the verification results are highly reliable.

[0050] Third, the automatic updating method of large-screen electronic labels in supermarkets with verification function of the present invention greatly reduces the time for manual adjustment and confirmation of product information; in addition, since the content displayed on the price label can include the price information of all misplaced products, the probability of manual rearrangement can be reduced by attracting users to buy. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of the automatic updating method of the supermarket large-screen electronic label with verification function of the present invention.

[0052] Figure 2 It is a flow chart of identifying product information of the present invention.

[0053] Figure 3 It is a flow chart of label image splicing of the present invention.

[0054] Figure 4 It is a flow chart of label image verification of the present invention. DETAILED DESCRIPTION

[0055] The present invention will now be described in further detail with reference to the accompanying drawings.

[0056] It should be noted that the terms such as "upper", "lower", "left", "right", "front", "back", etc. cited in the invention are only for the convenience of description and are not used to limit the scope of implementation of the present invention. Changes or adjustments in their relative relationships should be regarded as the scope of implementation of the present invention without substantially changing the technical content.

[0057] Figure 1 This is a flow chart of the automatic updating method of the supermarket large screen electronic label with verification function of the present invention. Figure 1 , the automatic updating method comprises the following steps:

[0058] S1, construct the electronic tag data table: combine the actual geographic information of the supermarket to build a 3D model map of the supermarket, which includes shelf A i , Unit Goods Area Aisles and electronic tags An electronic tag data table is constructed; the electronic tag data table is used to store the number, orientation, location information of each electronic tag, and the shelf number corresponding to each electronic tag; i=1, 2, ..., I, I is the total number of shelves; It is shelf A i The total number of unit commodity areas included in the above; It is shelf A i Total number of corresponding electronic tags.

[0059] S2, collect shelf images: combine the electronic tag orientation and aisle location to calculate the The shooting direction angle range is obtained by scanning and shooting each electronic tag regularly along the calculated shooting direction angle range. The corresponding shelf image.

[0060] S3, identify product information: integrate the unit product areas on the shelf image to generate several product placement areas; identify the product category placed on each integrated product placement area and mark the product placement area and product category, and construct a data table of the relationship between electronic tags and product placement areas.

[0061] S4, placement area processing: according to the frequency of occurrence of the product placement area, the product placement area corresponding to only one electronic tag is defined as the first placement area, and the remaining product placement areas are defined as the second placement areas. The main label of each second placement area is determined according to the image area ratio.

[0062] S5, stitching label images: process the marked shelf image again, retrieve the product label image corresponding to each product placement area, stitch the retrieved product label images according to the positional relationship of each product placement area on the shelf image, and generate an electronic label price tag; wherein, the label image of the second placement area other than the main label is scaled according to the area occupied by the second placement area on the shelf image.

[0063] S6, display label image: send the spliced ​​electronic label price tag to the corresponding electronic label for display.

[0064] The optimization technical solution of this embodiment is described in detail below with reference to the accompanying drawings.

[0065] S1, construct the electronic tag data table

[0066] Since the price information of multiple commodities is usually displayed on the large-screen electronic tags, the position and orientation setting of the large-screen electronic tags is crucial. In this embodiment, the display content of the electronic tags is affected by the user's visual range, that is, on the one hand, all electronic tags on a shelf must be able to display the prices of all commodities on the shelf, and on the other hand, the display content of each electronic tag includes the types of commodities that the user can observe from any angle within the field of vision. Compared with setting a number of electronic tags corresponding to the number of commodities according to the placement of commodities, the effort required by users to find prices is greatly reduced. In summary, this embodiment proposes a method for setting the position and orientation of electronic tags: in step S1, the position of the electronic tag and the orientation of each electronic tag are calculated in combination with the shelf and aisle position information. Specifically, the following steps are included:

[0067] S11, calculate each shelf A by combining the shelf and aisle location information i Aisle perspective and aisle area Aisle viewing angle of each shelf and aisle area Calculate each shelf A i Total perspective and total viewing area

[0068] S12, calculate the position and orientation of the electronic tag on each shelf according to the following formula:

[0069]

[0070] st.P1:

[0071] P2:

[0072] P3:

[0073] Where P1 is the installation height restriction of the electronic tag. ΔH is the larger value of the maximum theoretical height of shelf Ai after placing goods and the user's eye-level height; max It is shelf A i The maximum height difference between the electronic tag and the product is related to the orientation of the electronic tag; P2 is the display viewing angle restriction condition of the electronic tag. It is an electronic tag Shelves allowed to be covered A i Maximum angle range, each shelf A i Aisle perspective One of the electronic tags P3 is the display coverage limit of the electronic tag. It is an electronic tag Shelves allowed to be covered A i Maximum viewing area, each shelf A i Aisle area One of the electronic tags The maximum viewing angle area covered.

[0074] This embodiment proposes that P1 is the installation height restriction condition of the electronic tag. Considering that the maximum theoretical height of goods on shelves in supermarkets is usually not lower than the eye level of a user, the electronic tag is suspended or placed above the maximum theoretical height of the shelf to ensure that the user can observe the display screen of at least one tag on the shelf from any angle. If the maximum theoretical height of goods on the shelf is lower than the user's eye level, the user's eye level can be set as the minimum installation height of the electronic tag. The user here refers to the type of user group that the supermarket mainly targets. ΔH max It is shelf A i The maximum height difference between the electronic tag and the product is related to the orientation of the electronic tag. For example, if the electronic tag is parallel to the ground, the maximum height difference can be small or even 0. If the electronic tag is at an acute angle to the ground, a margin is required to prevent the image of the electronic tag near the bottom edge from being blocked by the product in certain viewing directions when the electronic tag is tilted.

[0075] P2 and P3 are the display viewing angle and display coverage constraints of the electronic tags, respectively. Constraint P2 is used to ensure that all electronic tags on a shelf can display the prices of all the goods on the shelf, i.e. i All electronic tags on Shelves allowed to be covered Ai The sum of the maximum angle ranges (after removing the overlapping angles) of each shelf A i Total perspective Take the four-sided table shelf as an example, its total viewing angle The total viewing angle is 360°; while the single-sided cabinet shelf is 180°. P3 is the display coverage restriction condition of the electronic tag. It is an electronic tag Permitted covered shelves A i Maximum viewing area, each shelf A i Aisle area One of the electronic tags The maximum viewing angle area covered by P2. i Aisle perspective One of the electronic tags The maximum angle range is covered, ensuring that users can achieve the technical effect of obtaining commodity prices in situ from any angle.

[0076] S2, collect shelf images: combine the electronic tag orientation and aisle location to calculate the The shooting direction angle range is obtained by scanning and shooting each electronic tag regularly along the calculated shooting direction angle range. The corresponding shelf image.

[0077] Each electronic tag The shooting direction angle range can be and the shelf A that it allows to cover i Maximum angular range The collection of shelf images can be achieved through cameras installed at preset positions, or can be combined with manual patrol shooting. If it is necessary to combine manual patrol shooting, the images taken by manual patrol can be calibrated again, or the correlation between them and the images taken at fixed points can be indirectly calculated, and then used for price update applications. In this embodiment, in addition to being executed when the goods are put on the shelves, price updates can also be reflected in the daily operation and maintenance of the supermarket. For example, if one of the goods on a shelf is sold out, the price tag can be adjusted in time through regular updates and inspections to avoid misunderstandings by users.

[0078] S3, identify product information: integrate the unit product areas on the shelf image to generate several product placement areas; identify the product category placed on each integrated product placement area and mark the product placement area and product category, and construct a data table of the relationship between electronic tags and product placement areas.

[0079] See also Figure 2 In step S3, the process of identifying product information includes the following sub-steps:

[0080] S31, collect a number of product image samples contained in the supermarket to construct a training set, the product image samples include stacked image samples of multiple similar products, and a full set of image samples of package products and single product image samples. The latter is to take into account the price display method of the package products.

[0081] S32, constructing a product recognition model based on a neural network, and training the product recognition model using a training set. The product recognition model can use image recognition technology in the prior art. For example, the invention with application number 202010874612.9 published on October 2, 2020 discloses a product recognition management method, which obtains the category and model information of the product by training and recognizing the image of the target product.

[0082] S33, preliminarily dividing the shelf image into unit product areas, performing similarity judgment on the images of adjacent unit product areas, and regarding the adjacent unit product areas with similarities higher than a similarity threshold as the same product placement area; preliminarily integrating the unit product areas on the shelf image to generate a plurality of product placement areas, and marking each product placement area.

[0083] S34, importing the image of each product placement area into the product recognition model to identify the type of product in each product placement area.

[0084] When one of the product placement areas contains more than two kinds of goods, the types of goods contained therein are matched with the types of goods in the adjacent product placement area. If all the types of goods are matched successfully, the product placement area is cleared, otherwise, the corresponding product placement area is generated according to the types of goods that are not matched successfully; the adjacent product placement area refers to the product placement area that belongs to the same electronic tag display coverage. This is because in a supermarket environment, due to the open shopping environment, it is often the case that users take the goods and put them in another area by mistake. In the prior art, for this situation, it is usually discovered through manual inspections or image recognition combined with database proofreading, and then the position of the goods is corrected by notifying the staff to re-place the goods. In this embodiment, it is not regarded as a simple misplaced product, but as a secondary product. Its price information is displayed simultaneously in the large-screen electronic tag, so as to increase the probability of users purchasing the product and reduce the workload of the staff to pick and place.

[0085] S4, placement area processing: according to the frequency of occurrence of the product placement area, the product placement area corresponding to only one electronic tag is defined as the first placement area, and the remaining product placement areas are defined as the second placement areas. The main label of each second placement area is determined according to the image area ratio.

[0086] In this embodiment, in addition to the misplaced secondary selected products, the products on the second display area also include products in the same viewing direction that are less likely to attract the user's attention because they occupy a smaller area. Due to the angle problem, these products occupy a smaller area in viewing direction A and can only be partially displayed, but they are likely to be fully displayed in viewing direction B, which is more in line with the user's visual law when shopping. The electronic tag corresponding to visual direction B is the main tag of the second display area where the product is located, and the remaining electronic tags showing the product are secondary tags.

[0087] S5, stitching label images: process the marked shelf image again, retrieve the product label image corresponding to each product placement area, stitch the retrieved product label images according to the positional relationship of each product placement area on the shelf image, and generate an electronic label price tag; wherein, the label image of the second placement area other than the main label is scaled according to the area occupied by the second placement area on the shelf image.

[0088] See also Figure 3 In step S5, the process of stitching label images includes the following sub-steps:

[0089] S51, for each electronic tag, read the product placement area number and the product type on each product placement area contained in the current collection cycle, and retrieve the product label image data corresponding to each product placement area according to the product type.

[0090] S52, generating a label module according to the position information of the unit product area occupied by each product placement area, wherein the label template includes a plurality of label filling boxes, and each label filling box corresponds to a product placement area.

[0091] S53, determining whether the electronic label corresponds to a second placement area other than the main label, and if so, scaling the corresponding label filling frame within a preset image size range according to the area occupied by the second placement area on the shelf image.

[0092] S54, filling the retrieved product label image data into the corresponding label filling box to generate an electronic label price tag within the current collection cycle.

[0093] After obtaining the correspondence between the electronic tag and the product placement area, the splicing process of the product price tag can be realized according to the positional relationship of the product placement area. The electronic tag price board of this embodiment is spliced ​​according to the positional relationship of the product, which can correctly guide customers to understand the product information; at the same time, the product label image in the electronic tag price board is related to the user's attention under the current viewing angle, which can reduce the effort required by the user to obtain price information and reduce the misjudgment rate.

[0094] In this embodiment, although the product label image is scaled, it is still within the reasonable range of human vision, and the size and number of the electronic labels corresponding to the shelves can be determined in combination with the type and quantity of the products on the shelf.

[0095] S6, display label image: send the spliced ​​electronic label price tag to the corresponding electronic label for display.

[0096] In this embodiment, the purpose of setting the second display area is not only to enable the user to fully and quickly understand the prices of all commodities within the field of vision, but also has another important significance, that is, to serve as a verification standard for the update results.

[0097] Currently, the verification of price tag display results is still done manually. There are many types of goods in supermarkets, and even experienced staff find it difficult to remember the real-time prices of most goods. If the update is frequent, the workload of the staff will increase exponentially. This is especially true for large-screen price tags that display multiple product prices.

[0098] This embodiment proposes a new automatic verification method:

[0099] S7, collecting label images: collecting display images of multiple electronic labels in the same second placement area.

[0100] S8, verifying the label image: based on the correspondence between the second placement area and the electronic label, verifying the position information and proportion information of the second placement area on the plurality of electronic labels.

[0101] Specifically, in step S8, the process of verifying the label image includes the following sub-steps:

[0102] S81, combining the electronic tags and the product placement area relationship data table, collecting the display image data set of the electronic tags in the same second placement area a And the original collection of shelf image data sets

[0103] S82, according to the position information of the unit product area occupied by the second display area a, from the display image data set of the electronic tag The corresponding image data set of the second placement area a is extracted from (m)}.

[0104] S83, determine the image data set {a (m)} is complete. If so, proceed to step S84. Otherwise, determine that the update has failed and end the process.

[0105] S84, calculating the second placement area a in the shelf image data set The scale ratio set of the label filling box of the second display area a is calculated by calculating the area occupied by each shelf image in

[0106] S84, according to the scale set For the data image data set {a (m)} to obtain the scaled validation set {b (m)}, Determine the validation set {b (m)} are consistent, if they are consistent, go to step S85, otherwise, determine that the update has failed and end the process.

[0107] S85, for the validation set {b (m)} and the similarity between the product price tag of the corresponding product on the second display area a. If the similarity is higher than a preset similarity threshold, the update is judged to be successful; otherwise, the update is judged to have failed.

[0108] Take the four-sided cargo platform as an example. Assume that there are several commodities 1 on the cargo platform on the east side. Some commodities 1 can also be observed from the south and north aisles. The four-sided cargo platform includes three electronic tags, one of which faces the east, and the other two face the southwest and northwest respectively. After analysis, the electronic tag 1 facing the east is the main tag, and the electronic tags 2 and 3 facing the southwest and northwest are the secondary tags. Electronic tags 1, 2, and 3 process and display the commodity price image of commodity 1 according to the area occupied and position of commodity 1 on its corresponding shelf image. In electronic tag 2, the price image of commodity 1 is small and displayed to the right, and in electronic tag 3, the price image of commodity 1 is small and displayed to the left.

[0109] During the verification process, the complete display images of electronic tags 1, 2, and 3 are collected respectively, and the corresponding display area price images are intercepted from the set positions in the complete display images according to the electronic tag template. If the display area price image is incomplete, it means that the current electronic tag update has failed, and it is very likely that the updated image data has not been received; otherwise, the original shelf image is processed again, the scaling ratio is calculated, and the intercepted display area price image is scaled and then compared for consistency, and the calculation results, scaling ratio, and image data display of the original shelf image are verified. The original shelf image here refers to the data basis when the electronic tag is updated, not just the image when the product is put on the shelf. This is for the verification work when the electronic tag is updated during the product sales process.

[0110] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for automatically updating a large-screen electronic label in a supermarket with a verification function, characterized in that: The automatic updating method comprises the following steps: S1, construct the electronic tag data table: combine the actual geographic information of the supermarket to build a 3D model map of the supermarket, which includes shelf A i , Unit Goods Area Aisles and electronic tags An electronic tag data table is constructed; the electronic tag data table is used to store the number, orientation, location information of each electronic tag, and the shelf number corresponding to each electronic tag; i = 1, 2, ..., I, I is the total number of shelves; It is shelf A i The total number of unit commodity areas included in the above; It is shelf A i The total number of corresponding electronic tags; S2, collect shelf images: combine the electronic tag orientation and aisle location to calculate the The shooting direction angle range is obtained by scanning and shooting each electronic tag regularly along the calculated shooting direction angle range. The corresponding shelf image; S3, identifying product information: integrating the unit product areas on the shelf image to generate a number of product placement areas; identifying the product category placed on each integrated product placement area and marking the product placement area and product category, and constructing a data table of the relationship between the electronic tag and the product placement area; S4, placement area processing: according to the frequency of occurrence of the product placement area, the product placement area corresponding to only one electronic tag is defined as the first placement area, and the remaining product placement areas are defined as the second placement areas, and the main label of each second placement area is determined according to the image area ratio; S5, stitching product label images: process the marked shelf image again, retrieve the product label image corresponding to each product placement area, stitch the retrieved product label images according to the positional relationship of each product placement area on the shelf image, and generate an electronic label price tag; wherein, the product label image of the second placement area other than the main label is scaled according to the area occupied by the second placement area on the shelf image; S6, displaying the electronic label price tag: sending the spliced ​​electronic label price tag to the corresponding electronic label for display; S7, collecting product label images: collecting display images of multiple electronic labels in the same second placement area; S8, verifying the product label image: combining the correspondence between the second placement area and the electronic label, verifying the position information and proportion information of the second placement area on the multiple electronic labels; In step S8, the process of verifying the product label image includes the following sub-steps: S81, combining the electronic tags and the product placement area relationship data table, collecting the display image data set of the electronic tags in the same second placement area a And the original collection of shelf image data sets S82, according to the position information of the unit product area occupied by the second display area a, from the display image data set of the electronic tag The corresponding image data set of the second placement area a is extracted from (m) }; S83, determine the image data set {a (m) } whether the product label image data in is complete, if complete, proceed to step S84, otherwise, determine that the update fails and end the process; S84, calculating the second placement area a in the shelf image data set The scale ratio set of the label filling box of the second display area a is calculated by calculating the area occupied by each shelf image in S84, according to the scale set For the data image data set {a (m) } to obtain the scaled validation set {b (m) }, Determine the validation set {b (m) }, if they are consistent, go to step S85, otherwise, determine that the update fails and end the process; S85, for the validation set {b (m) } and the similarity between the product price tag of the corresponding product on the second display area a. If the similarity is higher than a preset similarity threshold, the update is judged to be successful; otherwise, the update is judged to have failed.

2. The method for automatically updating a large-screen electronic label in a supermarket with a verification function according to claim 1 is characterized in that: In step S1, the position of the electronic tag and the orientation of each electronic tag are calculated based on the shelf and aisle position information.

3. The method for automatically updating a large-screen electronic label in a supermarket with a verification function according to claim 2 is characterized in that: The process of calculating the position of the electronic tag and the orientation of each electronic tag by combining the shelf and aisle location information includes the following steps: S11, calculate each shelf A by combining the shelf and aisle location information i Aisle perspective and aisle area Aisle viewing angle of each shelf and aisle area Calculate each shelf A i The total viewing angle θ Ai and total viewing area S12, calculate the position and orientation of the electronic tag on each shelf according to the following formula: Where P1 is the installation height restriction of the electronic tag. It is shelf A i The larger value between the maximum theoretical height after placing the product and the user's eye level height; ΔH max It is shelf A i The maximum height difference between the electronic tag and the product is related to the orientation of the electronic tag; P2 is the display viewing angle restriction condition of the electronic tag. It is an electronic tag Permitted covered shelves A i Maximum angle range, each shelf A i Aisle perspective One of the electronic tags P3 is the display coverage limit of the electronic tag. It is an electronic tag Shelves allowed to be covered A i Maximum viewing area, each shelf A i Aisle area One of the electronic tags The maximum viewing angle area covered.

4. The method for automatically updating a large-screen electronic label in a supermarket with a verification function according to claim 1, characterized in that: In step S3, the process of identifying product information includes the following sub-steps: S31, collecting a number of product image samples contained in the supermarket to construct a training set, where the product image samples include stacked image samples of multiple similar products, as well as a full set of image samples of packaged products and single product image samples; S32, building a commodity recognition model based on a neural network, and training the commodity recognition model using a training set; S33, preliminarily dividing the shelf image into unit product areas, performing similarity judgment on the images of adjacent unit product areas, and considering the adjacent unit product areas with similarity higher than a similarity threshold as the same product placement area; preliminarily integrating the unit product areas on the shelf image to generate a plurality of product placement areas, and marking each product placement area; S34, importing the image of each product placement area into the product recognition model to identify the type of product in each product placement area.

5. The method for automatically updating a large-screen electronic label in a supermarket with a verification function according to claim 4 is characterized in that: In step S34, when one of the product placement areas contains more than two products, the product types it contains are matched with the product types of the adjacent product placement areas. If all product types are matched successfully, the product placement area is cleared, otherwise, a corresponding product placement area is generated according to the product types that failed to match; the adjacent product placement area refers to the product placement area that belongs to the same electronic tag display coverage area.

6. The method for automatically updating a large-screen electronic label in a supermarket with a verification function according to claim 1, characterized in that: In step S5, the process of stitching the product label images includes the following sub-steps: S51, reading the product placement area number and the product type on each product placement area contained in the current collection cycle for each electronic tag, and retrieving the product label image corresponding to each product placement area according to the product type; S52, generating a label template according to the position information of the unit product area occupied by each product placement area, wherein the label template includes a plurality of label filling boxes, each label filling box corresponding to a product placement area; S53, determining whether the electronic label corresponds to a second placement area other than the main label, and if so, scaling the corresponding label filling frame within a preset image size range according to the area occupied by the second placement area on the shelf image; S54, filling the retrieved product label image into the corresponding label filling box to generate an electronic label price tag within the current collection cycle.

Images