Liquor label identification method and liquor label identification system

Abstract

The present invention provides a method for identifying alcoholic beverage labels, a method for managing alcoholic beverage information, and an apparatus, computer device, and computer-readable storage medium for the same. An image of an alcoholic beverage product is acquired, and characters contained in the image of the alcoholic beverage product are obtained by performing OCR identification on the image of the alcoholic beverage product using a predetermined OCR identification method (S21). Image features contained in the image of the alcoholic beverage product are obtained by performing deep learning identification on the image of the alcoholic beverage product using a predetermined deep learning identification method (S22). Based on the characters and image features, a target alcoholic beverage label that matches the characters and image features is selected from a predetermined alcoholic beverage label database, and the target alcoholic beverage label is designated as the alcoholic beverage label corresponding to the image of the alcoholic beverage product (S33). By using deep learning to overcome the shortcomings of OCR character identification in the complex environment where alcoholic beverage products are located, and by fully utilizing OCR character identification to overcome the shortcomings of deep learning-based alcoholic beverage label image identification, the accuracy and efficiency of alcoholic beverage label identification are improved, and the automation efficiency of alcoholic beverage information management is enhanced.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority from a Chinese patent application having application number 202011040731.0 and filing date September 28, 2020, all of which are hereby incorporated by reference in their entirety.

[0002] (Technical field) The present application relates to the technical field of alcohol product information management, and in particular to a method for identifying an alcohol label, a method for managing alcohol product information and an apparatus, a computer device and a computer-readable storage medium thereof. [Background technology]

[0003] Alcoholic beverages have played an important role in social interactions from ancient times to the present. For example, wine plays an increasingly important role in modern social interactions. Alcoholic beverages are generally stored in a wine cellar. The storage of alcoholic beverages requires specific environmental conditions for the wine cellar, and the wine products in the wine cellar must be managed. For example, the storage of wine has extremely strict requirements, particularly with regard to temperature and humidity. In addition, due to the limitations of the functionality of traditional wine cellars, the management of wine products stored in the wine cellar is mainly carried out artificially, resulting in low efficiency in wine management. Summary of the Invention [Problem to be solved by the invention]

[0004] The present application provides a liquor label identification method, a liquor product information management method and its apparatus, computer device and computer-readable storage medium, which can solve the problem of low accuracy of liquor label identification in the prior art, and thereby solve the problem of low efficiency of automated management of liquor products in the prior art. [Means for solving the problem]

[0005] In a first aspect, the present application provides a liquor label identification method, which includes the steps of: acquiring a liquor product image, performing OCR identification of the liquor product image with a predetermined OCR identification method to obtain characters contained in the liquor product image, performing deep learning identification of the liquor product image with a predetermined deep learning identification method to obtain image features contained in the liquor product image, and selecting a target liquor label that matches the characters and the image features from a predetermined liquor label database based on the characters and the image features, and setting the target liquor label as a liquor label corresponding to the liquor product image.

[0006] In a second aspect, the present application provides a method for managing alcoholic product information applied to a server side, the method including the steps of: acquiring alcoholic product information including an alcoholic product image; identifying the alcoholic product image using the alcoholic label identification method to obtain an alcoholic label included in the alcoholic product image; and managing the alcoholic product information based on the alcoholic label.

[0007] In a third aspect, the present application provides a method for managing alcoholic product information applied to a terminal, the method including the steps of: sending an alcoholic product information acquisition request to a predetermined server in response to a user's operation, so that the predetermined server acquires target alcoholic product information corresponding to the alcoholic product information acquisition request, including alcoholic product information associated based on a predetermined alcoholic label obtained by identifying an alcoholic product image by the alcoholic label identification method, based on the alcoholic product information acquisition request, and returning the target alcoholic product information to the terminal; receiving the target alcoholic product information sent from the predetermined server; and displaying the target alcoholic product information so that the user can acquire the target alcoholic product information.

[0008] In a fourth aspect, the present application further provides a liquor label identification device, including a first identification unit for obtaining a liquor product image and performing OCR identification of the liquor product image with a predetermined OCR identification method to obtain characters contained in the liquor product image, a second identification unit for obtaining image features contained in the liquor product image by performing deep learning identification of the liquor product image with a predetermined deep learning identification method, and a first sorting unit for selecting a target liquor label that matches the characters and the image features from a predetermined liquor label database based on the characters and the image features, and making the target liquor label a liquor label corresponding to the liquor product image.

[0009] In a fifth aspect, the present application further provides a liquor product information management device applied on a server side, comprising: an acquisition unit for acquiring liquor product information including a liquor product image; a third identification unit for obtaining a liquor label contained in the liquor product image by identifying the liquor product image using the liquor label identification method; and a management unit for managing the liquor product information based on the liquor label.

[0010] In a sixth aspect, the present application further provides a liquor product information management device applicable to a terminal, comprising: a transmitting unit for transmitting a liquor product information acquisition request to a predetermined server in response to a user's operation, the predetermined server including liquor product information associated based on a predetermined liquor label obtained by identifying a liquor product image using the liquor label identification method, acquiring target liquor product information corresponding to the liquor product information acquisition request based on the liquor product information acquisition request, and returning the target liquor product information to the terminal; a receiving unit for receiving the target liquor product information transmitted from the predetermined server; and a display unit for displaying the target liquor product information so that the user can acquire the target liquor product information.

[0011] In a seventh aspect, the present application further provides a computing device comprising a memory for storing a computer program, and a processor connected to the memory, the processor performing, when executing the computer program, steps of the liquor label identification method, or performing steps of the liquor product information management method applied to a server side, or performing steps of the liquor product information management method applied to a terminal.

[0012] In an eighth aspect, the present application further provides a computer-readable storage medium having a computer program stored therein, the computer program causing the processor to implement steps of the liquor label identification method, or steps of the liquor product information management method applied to a server side, or steps of the liquor product information management method applied to a terminal, when executed by a processor. Effect of the Invention

[0013] This application provides a liquor label identification method, a liquor product information management method, and an apparatus, computer device, and computer-readable storage medium thereof. In this application, a liquor product image is acquired, and characters contained in the liquor product image are obtained by performing OCR identification of the liquor product image using a predetermined OCR identification method. Image features contained in the liquor product image are obtained by performing deep learning identification of the liquor product image using a predetermined deep learning identification method. Based on the characters and the image features, a target liquor label that matches the characters and the image features is selected from a predetermined liquor label database, and the target liquor label is the liquor label corresponding to the liquor product image. This application combines OCR character identification and deep learning image identification to realize automatic identification of liquor labels based on machine vision. By fully combining the advantages of OCR optical identification and deep learning identification, the advantages of OCR character identification, that is, the identification of reference characters is fast and accurate, can be fully utilized to quickly identify the reference font characters contained in the image of the liquor product, and the advantages of image identification by deep learning can be used to overcome the disadvantages of the liquor label character identification by OCR character identification in the complex environment where the liquor product is located, and can also fully utilize the advantages of the image identification by deep learning, that is, the relative accuracy. Deep learning identification based on machine vision achieves good compatibility with multilingual characters, and fully utilizes the advantages of the reference character identification by OCR character identification to reduce the search range of deep learning, thereby overcoming the disadvantages of the high training data requirement and large calculation amount of the liquor label image identification by deep learning. At the same time, the liquor label image identification based on deep learning only needs to search within the small target liquor label range through OCR character identification, so that the demand for system computing resources is reduced, and the liquor label can be accurately identified efficiently in real time, thereby improving the accuracy and efficiency of liquor label identification. Based on the accurate identification of the wine label, the automated management of the wine products in the wine cellar can be realized, and the efficiency of the automated management of the wine products can be improved. For example, in a wine cellar storing wine, the wine can be automatically and highly efficiently managed in real time, thereby improving the efficiency of the automated management of the wine. [Brief description of the drawings]

[0014] In order to more clearly describe the technical aspects of the embodiments of the present application, the following briefly introduces the drawings that need to be used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without paying creative labor.

[0015] [Figure 1] FIG. 2 is a diagram illustrating an application environment of the liquor label identification method according to the embodiment of the present application. [Diagram 2] 1 is a schematic diagram of the flow of a liquor label identification method according to an embodiment of the present application. [Diagram 3] 1 is a schematic diagram of a first sub-flow in a liquor label identification method according to an embodiment of the present application. [Figure 4] 1 is a schematic diagram of a second sub-flow in a liquor label identification method according to an embodiment of the present application. [Diagram 5] 1 is a schematic diagram of the flow of a liquor product information management method applied to a server side according to an embodiment of the present application. [Figure 6] 1 is a schematic diagram of the flow of a liquor product information management method applied to a terminal according to an embodiment of the present application; [Figure 7] 1 is a block schematic diagram of a liquor label identification device according to an embodiment of the present application. [Figure 8] 1 is a block diagram of an alcoholic product information management device applied to a server side according to an embodiment of the present invention. [Figure 9] 1 is a block diagram illustrating a liquor product information management device applied to a terminal according to an embodiment of the present application. [Figure 10] 1 is a block schematic diagram of a computing device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, the technical aspects of the embodiments of the present application will be described clearly and completely in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person skilled in the art without performing creative labors belong to the protection scope of the present application.

[0017] It should be noted that, as used in this specification and the appended claims, the terms "comprising" and "consisting" indicate the presence of stated features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof.

[0018] Please refer to Fig. 1. Fig. 1 is an application environment diagram of the liquor label identification method according to the embodiment of the present application. As shown in Fig. 1, the application environment includes: (1) Sake Brewery A plurality of cameras are installed at different positions in the sake brewery. As shown in FIG. 1, the sake brewery in this example is installed with a camera 1, a camera 2, a camera 3, and a camera 4 for photographing the sake products in the sake brewery. Please continue to refer to the figure. As shown in FIG. 1, this example includes wine 1, wine 2, and wine 3. Camera 1 and camera 2 are used to photograph wine 1, camera 3 is used to photograph wine 2, and camera 4 is used to photograph wine 3. The sake products may be sake products stored in a sake cabinet. The sake products may also be sake products that enter and exit the entrance and exit of the sake brewery. In that case, a camera needs to be installed at the entrance and exit of the sake brewery, and the sake products that enter and exit the sake brewery are photographed by the camera at the entrance and exit of the sake brewery, thereby performing automated management of the sake products. (2) Server The system is used to receive alcoholic product images corresponding to alcoholic products photographed and uploaded by a camera, perform alcoholic label identification on the alcoholic product images, and manage alcoholic product information for the alcoholic products corresponding to the alcoholic product images based on the alcoholic label identification results corresponding to the alcoholic label identification. (3) Terminal It is used to receive user's operations such as inquiry, editing, and deletion on the alcoholic product information corresponding to the alcoholic product, and to obtain the alcoholic product information corresponding to the user's operation from the server in response to the user's operation. Operations on the alcoholic product information can be performed through the alcoholic product information management APP installed on the terminal.

[0019] The operational process of each of the above subjects is as follows: 1) The cameras in the brewery take pictures of wine products corresponding to the wine products in the brewery, and then upload the wine product pictures to the server. For example, in Fig. 1, camera 1 and camera 2 take wine product picture 1 corresponding to wine 1, camera 3 takes wine product picture 2 corresponding to wine 2, and camera 4 takes wine product picture 3 corresponding to wine 3, and then upload wine product picture 1, wine product picture 2, and wine product picture 3 to the server. 2) The server receives alcoholic product information including an alcoholic product image, for example, wine product image 1, wine product image 2, or wine product image 3. After receiving the alcoholic product image, the server performs OCR identification of the alcoholic product image using a predetermined OCR identification method to obtain characters contained in the alcoholic product image. The server performs deep learning identification of the alcoholic product image using a predetermined deep learning identification method to obtain image features contained in the alcoholic product image. Based on the characters and the image features, a target alcoholic label that matches the characters and the image features is selected from a predetermined alcoholic label database, and the target alcoholic label is set as the alcoholic label corresponding to the alcoholic product image. Based on the identified alcoholic label, the server manages alcoholic product information of the alcoholic product corresponding to the alcoholic product image, such as recording, deleting, and editing. 3) In response to a user's operation, a request for acquiring alcoholic product information is sent to a specified server. The specified server then acquires target alcoholic product information corresponding to the request for acquiring alcoholic product information based on the request for acquiring alcoholic product information, and returns the target alcoholic product information to the terminal. The target alcoholic product information includes alcoholic product information associated with a specified alcoholic label obtained by identifying an alcoholic product image using the alcoholic label identification method. The target alcoholic product information sent from the specified server is received. The target alcoholic product information is displayed so that the user can acquire the target alcoholic product information.

[0020] Please refer to Fig. 2. Fig. 2 is a schematic diagram of the flow of a liquor label identification method according to an embodiment of the present application. As shown in Fig. 2, the method includes the following steps S21 to S23.

[0021] In S21, an image of a liquor product is acquired, and characters contained in the image of the liquor product are obtained by performing OCR classification on the image of the liquor product using a predetermined OCR classification method.

[0022] Here, optical character recognition (OCR) optically converts characters in a document into a black and white grid image file, and then converts the characters in the image into text format through recognition software.

[0023] Specifically, when performing liquor label identification, a user may collect liquor product images corresponding to liquor products through an image collecting device with a camera such as a smartphone, or may collect liquor product images in a liquor brewery through a camera installed in the liquor brewery. The liquor product images are uploaded to a computer device that performs liquor label identification, and the computer device obtains the liquor product image to be identified. Then, the computer device performs OCR identification of the liquor product image using a predetermined OCR identification method to extract characters included in the liquor product image, and the characters are set as target characters. Since OCR technology is good at identifying character-type content such as normative characters, it can accurately identify characters included in the liquor product image. For example, when performing automated identification of liquor labels for wines in a wine cellar to manage wine product information, in the wine cellar, liquor product images of wine products are taken through multiple cameras arranged in the wine cellar. The liquor product image may be a single image (i.e., a picture-type screen) taken through a camera and uploaded to a background wine product information server, or may be a video collected by a camera and uploaded to a background wine product information server. Then, by extracting images obtained from video frames in the video, the background wine product information server performs OCR identification of the wine product image with a predetermined OCR identification method to obtain characters contained in the wine product image, and the characters are used as target characters corresponding to the wine label. In another application scenario, when managing wine product information, a wine label video of a wine product is shot with a mobile phone camera, and an image contained in the wine label video is extracted to obtain a wine label image, or an image of the wine product's wine label is directly shot, and the wine label image is OCR identified with a predetermined OCR identification method to obtain characters contained in the wine label image, and the characters are used as target characters.

[0024] Furthermore, before the step of acquiring the alcoholic product image, the method further includes a step of acquiring an initial alcoholic product image corresponding to the alcoholic product image, and a step of dividing the initial alcoholic product image to obtain an alcoholic product image corresponding to a partial image including a predetermined alcoholic label in the initial alcoholic product image.

[0025] Specifically, there is a high probability that different objects or different parts of the same object, for example, between a sake product and the surrounding environment, between different sake products, or between different parts of the same sake product, will show different image features such as different brightness, texture, or color. In view of this, since the sake products that need to be managed are within a predetermined sake product range, and the sake labels that need to be identified are also within a predetermined sake label range, before the step of automatically identifying the sake labels, after acquiring an initial sake product image corresponding to the sake product image and before performing automatic identification of the sake labels, the acquired initial sake product image is divided to smartly divide the environmental image other than the sake product contained in the initial sake product image, and other partial images of the sake product such as the sake bottle other than the sake label, and the partial image including the predetermined sake label after division is set to the approximate position of the target sake label image, and the target sake label image is set to the sake product image for identifying the sake label. By narrowing down the image to be identified to the partial image including the sake label, it is possible to accurately identify the sake product image corresponding to the sake bottle, especially in a complex environment including many environmental elements such as a sake brewery, and obtain the corresponding accurate image features. This can increase the accuracy of the subsequent liquor label identification and reduce the amount of data processing compared to identifying the entire initial liquor product image, thereby improving the efficiency of liquor label identification.

[0026] Furthermore, after the step of dividing the initial alcoholic product image, the method further includes a step of determining whether or not the initial alcoholic product image includes a specified alcoholic label, and if the initial alcoholic product image includes the specified alcoholic label, a step of obtaining an alcoholic product image corresponding to the partial image including the specified alcoholic label in the initial alcoholic product image, and if the initial alcoholic product image does not include the specified alcoholic label, a step of displaying an error message about the initial alcoholic product image.

[0027] Specifically, there is a high probability that different objects or different parts of the same object, for example, between a liquor product and the surrounding environment, between different liquor products, or between different parts of the same liquor product, will show different image features such as different brightness, texture, or color. The image features corresponding to the partial images containing the liquor label for each type of liquor product are unique, and in terms of image features, the image features corresponding to the partial images in the liquor bottle containing the liquor label are unique and prominent. After dividing the initial liquor product image, the image features obtained by dividing the initial liquor product image are combined with prominent liquor label image features corresponding to a predetermined partial image containing the liquor label to determine whether the initial liquor product image contains a predetermined liquor label in a preliminary manner. If the initial liquor product image contains a predetermined liquor label, a step of obtaining a liquor product image corresponding to the partial image containing the predetermined liquor label in the initial liquor product image is performed to perform more detailed image identification on the initial liquor product image and identify the liquor label contained in the initial liquor product image. If the initial alcoholic product image does not include a specific alcoholic label, for example, if it is an image of another part of the alcoholic bottle other than the alcoholic label, such as an image of the main body of the alcoholic bottle or an image of the alcoholic bottle cap, or if the image is a non-alcoholic label image, such as a face image, an image of a alcoholic cabinet, or an image of the surrounding environment, the initial alcoholic product image does not need to be further processed, and an error notification of the initial alcoholic product image is issued to the effect that the initial alcoholic product image is not a necessary image and may be discarded. Since the amount of image data corresponding to the image features corresponding to the partial image in the alcoholic bottle including the alcoholic label is relatively small, if the initial alcoholic product image does not include a specific alcoholic label, there is no need to perform a large amount of image data processing on the initial alcoholic product image, and the classification result for the alcoholic product image can be obtained as quickly as possible, the flow of alcoholic label classification can be shortened, and the classification efficiency for the alcoholic label image can be improved. In addition, a recommendation to re-image the initial alcoholic product image can be issued, and the processing efficiency of alcoholic label classification can be improved.

[0028] In S22, image features contained in the alcoholic product image are extracted by performing deep learning classification of the alcoholic product image using a predetermined deep learning classification method.

[0029] Here, deep learning constructs and simulates a neural network that performs analytical learning in the human brain, and interprets data by imitating the mechanism of the human brain. Deep learning models include feed forward neural networks (FF or FFNN) and perceptrons (P), Boltzmann machines (BM), etc., and through test samples, deep learning models with relatively good alcohol label recognition effects are selected from many deep learning models.

[0030] Specifically, after acquiring a liquor product image, a deep learning identification method is used to perform deep learning identification of the liquor product image. Deep learning is feature learning (or expression learning), so it is feature engineering, and image features included in the liquor product image can be obtained by deep learning. Since deep learning is good at learning training sample images and tags corresponding to the training sample images, a deep learning model based on deep learning after pre-training is good at identifying image features, so that it can accurately identify image features included in the liquor product image and set the image features as target image features corresponding to the liquor product image. For example, when managing wine product information by performing automated identification of wine labels for wine, after acquiring a wine product image corresponding to a wine product, a deep learning identification method is used to perform deep learning identification of the wine product image, so that image features included in the wine product image are identified and set the image features as target image features corresponding to the wine product image.

[0031] Here, the process of training a predetermined deep learning model corresponding to the predetermined deep learning identification scheme includes the following steps: 1) A tag is attached to a liquor label image corresponding to a given liquor product to obtain a pre-training sample of liquor label images. The pre-training sample of liquor label images includes the training liquor label image and a training liquor label tag corresponding to the training liquor label image. Specifically, a liquor label image is photographed for a liquor label corresponding to a liquor product that needs to be managed, and in particular, the liquor label image on the liquor product is photographed from various angles, and a tag is attached to the photographed liquor label image. The so-called tagging means that the photographed liquor label image is associated with the liquor label name corresponding to the liquor label, and it is described that the liquor label image corresponds to the liquor label name. The liquor label name is the liquor label tag corresponding to the liquor label image. 2) The predetermined deep learning model is trained using pre-training samples of the liquor label images. Specifically, the pre-training samples of the liquor label images are input into the predetermined deep learning model, and the predetermined deep learning model automatically learns based on the training liquor label images and the training liquor label tags corresponding to the training liquor label images, and learns the association relationship between the training image features corresponding to the training liquor label images and the training liquor label tags to complete the training of the deep learning model. 3) Test the predetermined deep learning model using a test sample of a liquor label image. The test sample of the liquor label image includes the test liquor label image and a test liquor label tag corresponding to the test liquor label image. Specifically, the predetermined deep learning model after training is tested using the test sample of the liquor label image. The predetermined deep learning model identifies the test liquor label image to obtain a test target liquor label. The test target liquor label is compared with the test liquor label tag corresponding to the test liquor label image to determine whether the test target liquor label and the test liquor label tag corresponding to the test liquor label image match. If the test target liquor label matches the test liquor label tag corresponding to the test liquor label image, it indicates that the deep learning model has a relatively accurate discrimination effect for the predetermined liquor label discrimination. If the test target liquor label does not match the test liquor label tag corresponding to the test liquor label image, it indicates that the deep learning model does not have a relatively accurate discrimination effect for the predetermined liquor label discrimination. Thereby, it is determined whether the predetermined deep learning model has reached a desired goal. If the desired goal has been reached, the predetermined deep learning model is put into a production environment. If the desired goal is not reached, training is resumed with more training samples, adjustments to the training samples, and adjustments to the given deep learning model until the given deep learning model finally meets the desired goal.

[0032] In S23, based on the characters and the image features, a target sake label that matches the characters and the image features is selected from a predetermined sake label database, and the target sake label is set as a sake label that corresponds to the sake product image.

[0033] Specifically, when identifying a liquor label, the range of the liquor label can be determined in advance, for example, the type of liquor products present in a liquor brewery can be determined in advance, so a predetermined liquor label database is constructed based on the liquor labels corresponding to the liquor products that need to be managed, and after extracting characters and image features contained in the liquor product images from the liquor product images, the characters and image features are combined based on the characters and image features, and a liquor label that matches the characters and image features is selected from the predetermined liquor label database as a target liquor label, and the target liquor label is set as the liquor label corresponding to the liquor product image, thereby identifying the liquor label corresponding to the liquor product image, and automatically managing liquor product information based on the identified liquor label. For example, when managing liquor product information by a wine cellar, after acquiring a wine product image, the characters and image features in the wine product image are identified, and the wine label corresponding to the wine product image is identified based on the characters and image features, and the wine product information is managed based on the identified wine label.

[0034] In the embodiment of the present application, the liquor label is generally composed of characters and images, and there are large differences between different liquor labels. Especially for wine labels, the font types, character sizes, and languages ​​of the wine labels produced by different manufacturers all have large font differences, especially under complex wine brewery environmental conditions, and there are many differences in liquor labels, such as multilingual. In order to realize unified automated management of liquor products, when the liquor labels are automatically identified based only on optical character recognition in the prior art, in addition to font differences, there are also problems such as poor compatibility of multilingual characters, and the recognition accuracy of the liquor label characters is low, which causes instability of the recognition effect. Conventional deep learning technology has too high requirements for the amount of training sample data to train a relatively accurate deep learning recognition model. Image recognition based on deep learning technology requires a large amount of calculation, and when the training sample is not large enough, the recognition efficiency of the liquor label is also low. Generally, it is difficult to accumulate a sufficiently large liquor label training data sample for the application environment of liquor product information management. Therefore, it is very difficult for a computer device to accurately identify many types of liquor labels. The applicant of the embodiment of the present application faces the need for uniform and accurate identification of many kinds of liquor labels in a complex environment with large differences. Based on the characteristics of liquor labels consisting of characters and images, OCR character identification and deep learning image identification are combined to realize automatic identification of liquor labels based on machine vision. By fully combining the advantages of OCR optical identification and deep learning identification, the advantages of OCR character identification, that is, the fast and accurate identification of reference characters, can be fully utilized to not only quickly identify the characters of the reference font contained in the image of the liquor product, but also the advantages of the accuracy of image identification by deep learning can be fully utilized. Deep learning identification based on machine vision adopts the advantages of image identification by deep learning to overcome the disadvantages of OCR character identification, and at the same time achieves good compatibility for multilingual characters. Therefore, the advantages of OCR character identification are fully utilized to reduce the search scope of deep learning, and overcome the disadvantages of image identification by deep learning, that the training data volume requirement is too high and the calculation volume is large.At the same time, image recognition based on deep learning only needs to search within a small search range, and can optimize the network structure of the deep learning model to reduce the demand for system computing resources. Therefore, by combining optical character recognition and deep learning recognition, the recognition results of the two recognition methods are integrated to realize a rough to precise and accurate intelligent recognition process for many kinds of wine labels, especially for the situation where there are large differences in multiple languages ​​for wine labels and the amount of wine label training sample data is not large enough, the recognition speed and accuracy of wine labels can be improved, and the deep learning training wine label recognition can be met, and accurate recognition can be efficiently performed for wine product labels in real time, improving the accuracy and efficiency of wine product label recognition.

[0035] In one embodiment, the step of selecting a target liquor label that matches the character and the image feature from a predetermined liquor label database based on the character and the image feature, and setting the target liquor label as the liquor label corresponding to the liquor product image includes the steps of selecting a predetermined liquor label that matches the character from a predetermined liquor label database based on the character, and organizing the predetermined liquor labels into a set to obtain a target liquor label set, and selecting a liquor label that has the highest degree of matching with the image feature from the target liquor label set based on the image feature as the target liquor label, and setting the target liquor label as the liquor label corresponding to the liquor product image.

[0036] Specifically, after extracting characters contained in an image through OCR recognition, based on the characters, select predetermined wine labels that match the characters from a predetermined wine label database, and then organize the predetermined wine labels into a set to obtain a target wine label set, thereby making full use of the advantages of fast and accurate reference character recognition by OCR to reduce the wine label search scope by deep learning to the target wine label set.

[0037] Furthermore, in general, a liquor label consists of characters and images, or the identified characters are not accurate. For example, the identified characters are similar or identical to other characters, or are included in other character combinations. Therefore, the identified characters alone cannot accurately identify the liquor label corresponding to the liquor product image. After obtaining the target liquor label set, the liquor label having the highest degree of matching with the image features is selected as the target liquor label from the target liquor label set based on the image features, i.e., the liquor label having the highest degree of matching with the image features is selected as the target liquor label from the target liquor label set, and the target liquor label is set as the liquor label corresponding to the liquor product image, thereby identifying the liquor label corresponding to the liquor product image. Due to the characteristic that the pre-trained deep learning model has a relatively accurate image identification, when the liquor label inspection range of the deep learning model has already been reduced to the target liquor label set, the calculation amount of the deep learning model can be greatly reduced compared to directly searching a predetermined liquor label database to obtain the target liquor label, thereby overcoming the drawback of low identification accuracy when identifying characters using an OCR model in a complex environment such as a liquor brewery. This allows for selecting an accurate target liquor label from the target liquor label set, and obtaining an accurate liquor label corresponding to the liquor product image, thereby improving the accuracy and efficiency of identifying the liquor label corresponding to the liquor product image.

[0038] Furthermore, prior to the step of selecting, based on the character, a specific liquor label that matches the character from a specific liquor label database and organizing the specific liquor labels into a set to obtain a target liquor label set, the method further includes the steps of: calculating the character similarity between the character and a specific liquor label character corresponding to a specific liquor label in the specific liquor label database; determining whether the character similarity is equal to or greater than a specific character similarity threshold; and, if the character similarity is equal to or greater than the specific character similarity threshold, setting the specific liquor label character as a target character and setting the target character as a character corresponding to the liquor product image.

[0039] Furthermore, the step of setting the specified liquor label character as a target character if the character similarity is equal to or greater than a specified character similarity threshold, and setting the target character as a character corresponding to the liquor product image includes a step of setting the specified liquor label character with the highest character similarity as the target character if the character similarity is equal to or greater than a specified character similarity threshold.

[0040] Specifically, the character similarity between the character included in the identified alcohol product image and a predetermined alcohol label character is determined, the character similarity between the character and a predetermined alcohol label character corresponding to a predetermined alcohol label in the predetermined alcohol label database is calculated, and it is determined whether the character similarity is equal to or greater than a predetermined character similarity threshold. If the character similarity is equal to or greater than the predetermined character similarity threshold, it is determined that the character and the predetermined alcohol label character are identical or similar, and the predetermined alcohol label character is set as a target character, and the target character is set as a character corresponding to the alcohol product image. If the character similarity is less than the predetermined character similarity threshold, it is determined that the difference between the character and the predetermined alcohol label character is too large, and a notification of alcohol label identification error or alcohol label identification failure is issued, and a notification is issued to re-take the alcohol product image, and the target alcohol label is further filtered by the predetermined character similarity threshold, so that the accuracy of alcohol label identification and the efficiency of alcohol label identification can be further improved.

[0041] Furthermore, the predetermined liquor label character is set as a target character. Since the same or similar predetermined liquor label character is set as the target character, the target character may include a plurality of predetermined liquor label characters. In order to further narrow the target range of characters, narrow the identification range of liquor labels, and increase the identification efficiency of liquor labels, the predetermined liquor label character with the highest character similarity is set as the target character, thereby identifying a unique target character, narrowing the range of the target liquor labels, and increasing the efficiency of liquor label identification.

[0042] Furthermore, the step of selecting a liquor label that has the highest degree of matching with the image features as a target liquor label from the target liquor label set based on the image features and setting the target liquor label as the liquor label corresponding to the liquor product image includes the steps of: calculating image similarities between the image features and a specified liquor label included in the target liquor label set to obtain all the image similarities; selecting a target image similarity having the highest image similarity from all the image similarities; determining whether the target image similarity is greater than or equal to a specified image similarity threshold; and, if the target image similarity is greater than or equal to a specified image similarity threshold, setting the specified liquor label corresponding to the target image similarity as the target liquor label and setting the target liquor label as the liquor label corresponding to the liquor product image.

[0043] Specifically, the image similarity between the image features included in the identified alcohol product image and a predetermined alcohol label included in the target alcohol label set is determined, and the image similarities between the image features and the predetermined alcohol labels included in the target alcohol label set are statistically calculated to obtain all the image similarities. The target image similarity having the highest image similarity is selected from all the image similarities, and it is determined whether the target image similarity is equal to or greater than a predetermined image similarity threshold. If the target image similarity is equal to or greater than a predetermined image similarity threshold, it indicates that the predetermined alcohol label corresponding to the target image similarity is the same as or similar to the alcohol label included in the alcohol product image, and the predetermined alcohol label corresponding to the target image similarity is set as the target alcohol label, and the target alcohol label is set as the alcohol label corresponding to the alcohol product image. If the target image similarity is less than a predetermined image similarity threshold, it indicates that the difference between the predetermined liquor label corresponding to the target image similarity and the liquor label included in the liquor product image is too large and insufficient to accurately identify the liquor label included in the liquor product image as the target liquor label within the predetermined range, and a suggestion of misidentification of the liquor label or failure to identify the liquor label is issued, and a suggestion is issued to re-take the liquor product image, and the target liquor label can be further filtered by the predetermined image similarity threshold to further improve the accuracy of identifying the liquor label.

[0044] Please refer to Fig. 3. Fig. 3 is a schematic diagram of a first sub-flow in a liquor label identification method according to an embodiment of the present application. As shown in Fig. 3, in this embodiment, the target liquor label set includes a plurality of initial target liquor labels, and the step of selecting a liquor label having the highest consistency with the image features from the target liquor label set as a target liquor label based on the image features, and setting the target liquor label as a liquor label corresponding to the liquor product image includes: In S31, image similarities between the image features and each of the initial target liquor labels are calculated. In S32, all the image similarities are sorted in descending order to obtain a sort queue. In S33, the first image similarity from the sort queue is selected as the target image similarity, and the initial target liquor label corresponding to the target image similarity is set as the target liquor label. In S34, the target alcohol label is set to the alcohol label corresponding to the alcohol product image.

[0045] Specifically, the image is sampled and quantized, the image is quantized into an image vector, the image features and the initial target liquor label are converted into image vectors, the image features and the initial target liquor label are quantized, and the similarity between the two image vectors is calculated. For example, the cosine similarity between the image vectors is calculated to calculate the image similarity between the image features and each of the initial target liquor labels. After obtaining the image similarity between the image features and each of the initial target liquor labels, all the image similarities are sorted in descending order to obtain a sort queue. The similarity of each image corresponds to the image similarity between the image features and each of the initial target liquor labels, and the sort queue describes the sorting of the initial target liquor labels. The first target image similarity is selected from the sort queue, the initial target liquor label corresponding to the target image similarity is set as the target liquor label, and the target liquor label is set as the liquor label corresponding to the liquor product image, and the liquor label having the highest degree of consistency with the image feature is selected as the target liquor label from the target liquor label set. Since the target liquor label set is already a liquor label range reduced by OCR identification, and the liquor label that most closely matches the image features is selected from the target liquor label set as the target liquor label, the efficiency and accuracy of identifying the liquor label corresponding to the liquor product image is improved.

[0046] In one embodiment, before the step of setting the initial target liquor label corresponding to the target image similarity as the target liquor label, the method further includes the step of determining whether the target image similarity is greater than or equal to a predetermined image similarity threshold, and if the target image similarity is greater than or equal to the predetermined image similarity threshold, the step of setting the initial target liquor label corresponding to the target image similarity as the target liquor label.

[0047] Specifically, in order to further improve the accuracy of identifying the liquor label, an image similarity threshold is set for the target image similarity, and it is determined whether the target image similarity is equal to or greater than a predetermined image similarity threshold. If the target image similarity is equal to or greater than a predetermined image similarity threshold, the initial target liquor label corresponding to the target image similarity is taken as the target liquor label, and the target liquor label is taken as the liquor label corresponding to the liquor product image, thereby identifying the liquor label corresponding to the liquor product image. If the target image similarity is less than the predetermined image similarity threshold, it indicates that the image similarity between the image feature and the liquor label in the predetermined liquor label database does not meet the requirements, and there is a high risk of inaccurate identification. In order to avoid confusion of subsequent liquor product information, the initial target liquor label corresponding to the target image similarity is not taken as the target liquor label. In order to avoid confusion of liquor product information due to liquor label identification errors, it is necessary to re-acquire the liquor product image and re-acquire the liquor product image for automatic liquor label identification, and a suggestion is given to re-take the liquor product image corresponding to the liquor label to be identified. Furthermore, when a suggestion is given to capture an image of a liquor product corresponding to the liquor label to be identified, advice can be given for capturing the liquor product image, such as that the liquor product image should include a complete liquor label image or that the liquor label image needs to be captured from directly in front.

[0048] Please refer to Fig. 4. Fig. 4 is a schematic diagram of a second sub-flow in the method for identifying alcoholic beverage labels according to an embodiment of the present application. As shown in Fig. 4, in this embodiment, the alcoholic beverage product image includes a plurality of images of the same target alcoholic beverage product, and the step of setting the target alcoholic beverage label as the alcoholic beverage label corresponding to the alcoholic beverage product image includes: In S41, the target liquor label corresponding to each image is set as the liquor label corresponding to that image.

[0049] After the step of determining that the target liquor label corresponding to each image is the liquor label corresponding to the image, the method further includes: In S42, of the liquor labels corresponding to each of all of the images, liquor labels belonging to the same liquor label group are classified as liquor labels of the same type. In S43, the number of liquor labels included in each type of liquor label of the same type is counted. In S44, based on the number of liquor labels included in each type of the same type of liquor labels, the liquor label with the largest numerical value is selected from all the liquor labels of the same type, and is set as the target liquor product label corresponding to the target liquor product.

[0050] Specifically, in one application environment, a plurality of images of the same target alcoholic beverage product are obtained. The above alcoholic beverage label identification method is performed for each image to obtain a target alcoholic beverage label corresponding to each image. The target alcoholic beverage label corresponding to each image is taken as the alcoholic beverage label corresponding to the image. Then, a target alcoholic beverage product label corresponding to the target alcoholic beverage product is selected from the alcoholic beverage labels corresponding to each of all the images. A alcoholic beverage label is identified for each of the plurality of images of the same target alcoholic beverage product, and the most suitable alcoholic beverage label is selected from the plurality of images as the target alcoholic beverage label corresponding to the target alcoholic beverage product, thereby improving the accuracy of identifying the alcoholic beverage label corresponding to the target alcoholic beverage product. For example, in a wine management system, it is common to install cameras at different angles in a wine cellar and shoot the same target alcoholic beverage product with multiple cameras. Each camera shoots a video and uploads the video to the background of a server. The background of the server extracts a plurality of images of the same target alcoholic beverage product from the plurality of videos shot by the plurality of cameras, and identifies the alcoholic beverage label for each of the plurality of images. Since the imaging angles of the images are different, when identifying the target alcoholic beverage label corresponding to each image, although the target alcoholic beverage product is the same, there may be differences in the identification results, resulting in multiple alcoholic beverage level identification results for the same target alcoholic beverage product. It is necessary to select the most suitable alcoholic beverage label from among the alcoholic beverage labels corresponding to each of all the images, and make the most suitable alcoholic beverage label the corresponding target alcoholic beverage product label of the target alcoholic beverage product. For example, from among the alcoholic beverage labels corresponding to each of all the images, the alcoholic beverage label with the highest identification probability is selected as the most suitable alcoholic beverage label, or, from among the alcoholic beverage labels corresponding to each of all the images, the alcoholic beverage label with the highest similarity between the image and the alcoholic beverage label image in the predetermined alcoholic beverage label database is selected as the most suitable alcoholic beverage label from among the alcoholic beverage labels corresponding to each of all the images, based on the similarity between the image and the alcoholic beverage label image in the predetermined alcoholic beverage label database, thereby improving the accuracy of identifying the target alcoholic beverage product alcoholic beverage label corresponding to the target alcoholic beverage product.

[0051] Furthermore, when selecting the liquor label with the highest probability of being identified from among all the liquor labels corresponding to each of the images as the liquor label that best matches the liquor product image, the liquor labels corresponding to each of all the images are classified, and the liquor labels belonging to the same liquor label among all the liquor labels corresponding to each of the images are assigned to the same type of liquor label to obtain different categories of liquor labels of the same type. The quantity of liquor labels included in each type of liquor label of the same type is statistically calculated, and the liquor label with the largest number of liquor labels is determined as the target liquor product label corresponding to the target liquor product. The liquor label with the largest number of liquor labels corresponds to the liquor label with the highest probability of being identified, thereby improving the accuracy of identifying the target liquor product liquor label corresponding to the target liquor product. For example, in a wine product management system, when identifying liquor labels for the same target liquor product, 50 images are obtained for the same target liquor label, of which 30 are identified as liquor label A, 15 are identified as liquor label B, and 5 are identified as liquor label C. Based on the sorting method in which the liquor label containing the greatest number of liquor labels corresponds to the liquor label with the greatest identified probability, it can be determined that the probability that liquor label A is the liquor label of the target liquor product is the greatest, and thus liquor label A is determined to be the liquor label of the target liquor product.

[0052] In one embodiment, the step of acquiring the alcoholic product image includes a step of receiving an alcoholic product image uploaded from a terminal, the alcoholic product image being photographed and uploaded by the terminal, the terminal including a camera pre-installed in the sake brewery.

[0053] Specifically, the alcohol product images are uploaded from a camera-equipped terminal such as a smartphone to a computer device that performs alcohol level identification after the terminal takes pictures or videos (video frames in a video are alcohol product images). Alternatively, the alcohol product images are collected through a terminal collection device such as a camera that is installed in advance in the brewery. The alcohol product images are uploaded to a computer device that performs alcohol label identification, such as a server background that manages alcohol product information, after a camera in the brewery takes pictures or videos (video frames in a video are alcohol product images).

[0054] In one embodiment, the step of photographing and uploading the alcoholic product images by the terminal includes a plurality of the images being acquired by a plurality of cameras pre-installed in a specified brewery, the alcoholic product images including a plurality of images, the plurality of cameras being installed at different predetermined locations within the brewery.

[0055] Specifically, multiple images are acquired by multiple cameras installed in a sake brewery in advance, and the multiple cameras are installed at different predetermined positions in the sake brewery to photograph the same target sake product from different angles. In the sake product information automatic management system, a picture of the sake product is taken with a camera-equipped photographing device such as a mobile phone, a video of the sake product is taken, and the video is extracted to extract a video frame, thereby obtaining a sake product image. Then, the sake product image is identified to realize automatic identification of the sake label. Furthermore, based on the identified sake label, the sake product information about the sake product is acquired and managed. Meanwhile, in a sake brewery-compatible sake product information automatic management system, such as a wine cellar automatic management system, in order to realize automatic management of sake product information of wine products in a wine cellar, multiple cameras are generally installed in the wine cellar, and the multiple cameras are installed at different predetermined positions in the wine cellar to photograph the target wine product from different positions or different angles at the same position. By acquiring pictures or videos of the liquor products using multiple cameras pre-installed in the wine cellar, multiple images of the liquor products are acquired, the captured liquor product images are identified, the liquor label of the target wine product is identified, and liquor product information of the target wine is acquired based on the identified liquor label, and then the liquor product information is processed, thereby enabling automatic management of the target wine.

[0056] In addition, the liquor label identification methods described in the above-mentioned embodiments can be combined as necessary to obtain combined embodiments by recombining the technical features contained in different embodiments, all of which are within the scope of protection claimed by this application.

[0057] Please refer to Fig. 5. Fig. 5 is a schematic diagram of the flow of an alcoholic product information management method applied to a server side according to an embodiment of the present application. As shown in Fig. 5, the method includes the following steps S51 to S53. In S51, alcoholic product information including an image of the alcoholic product is obtained. In S52, the alcoholic label included in the alcoholic product image is obtained by identifying the alcoholic product image using the alcoholic label identification method. In S53, the alcoholic beverage product information is managed based on the alcoholic beverage label.

[0058] Specifically, when the server automatically manages the alcoholic product information corresponding to the alcoholic product, it not only obtains an image of the alcoholic product, identifies the alcoholic label of the image of the alcoholic product by the alcoholic label identification method described in each embodiment described above, and obtains the alcoholic product information of other aspects of the alcoholic product, such as the year of the alcoholic product, the place of origin of the alcoholic product, the date of production, and the time of entering and leaving the brewery, and then, after identifying the alcoholic label corresponding to the image of the alcoholic product, performs recording operations such as adding, modifying, and editing on the alcoholic product information based on the alcoholic label, thereby realizing the automated management of the alcoholic product. Since the accuracy and efficiency of the identification of the alcoholic label of the alcoholic product is improved, when the alcoholic products in the brewery are automatically managed based on the accurate and rapid identification of the alcoholic label, the efficiency and accuracy of the automated management of the alcoholic products can also be improved. For example, in a wine cellar storing wine, the accuracy and rapidity of the identification of the wine label of the wine allows the wine to be automatically managed in real time with high quality, and the efficiency of the automatic management of the wine can be improved.

[0059] Please refer to Fig. 6. Fig. 6 is a schematic diagram of a flow of an alcoholic product information management method applied to a terminal according to an embodiment of the present application. As shown in Fig. 6, the method includes the following steps S61 to S63. In S61, in response to a user's operation, a request for obtaining information on alcoholic products is sent to a predetermined server, whereby the predetermined server obtains target alcoholic product information corresponding to the request for obtaining information on alcoholic products, including alcoholic product information associated with the predetermined alcoholic label obtained by identifying an alcoholic product image using the alcoholic label identification method, based on the request for obtaining information on alcoholic products, and returns the target alcoholic product information to the terminal. In S62, the target alcoholic product information transmitted from the predetermined server is received. In S63, the target alcoholic product information is displayed for the user to obtain the target alcoholic product information.

[0060] Specifically, when a user performs editing operations such as inquiry, modification, addition, or deletion on the alcoholic product information through a terminal, the terminal responds to the user's operation by transmitting an alcoholic product information acquisition request to a predetermined server. As a result, the predetermined server acquires target alcoholic product information corresponding to the alcoholic product information acquisition request, including alcoholic product information associated based on a predetermined alcoholic label obtained by identifying an alcoholic product image using the alcoholic label identification method described in each of the above embodiments, based on the alcoholic product information acquisition request, and returns the target alcoholic product information to the terminal. The terminal receives the target alcoholic product information transmitted from the predetermined server, and displays the target alcoholic product information so that the user can acquire the target alcoholic product information. Since the accuracy and efficiency of alcoholic label identification of alcoholic products are improved, when automated management of alcoholic products in a sake brewery is realized based on accurate and rapid identification of alcoholic labels, the user can easily realize improved efficiency, accuracy, and convenience of automated management of alcoholic products. For example, in a wine cellar storing wine, due to the accuracy and speed of identifying wine labels, it is possible to automatically perform high-quality management of wine in real time, which improves the efficiency, convenience and quality of automated wine management compared with the prior art method of artificially managing wine products in a wine cellar.

[0061] Please refer to Fig. 7. Fig. 7 is a block schematic diagram of a liquor label identification device according to an embodiment of the present application. Corresponding to the above-mentioned liquor label identification method, an embodiment of the present application further provides a liquor label identification device. As shown in Fig. 7, the liquor label identification device includes a unit for executing the above-mentioned liquor label identification method, and the liquor label identification device is disposed in a computer device such as a server. Specifically, referring to Fig. 7, the liquor label identification device 70 includes a first identification unit 71, a second identification unit 72, and a first sorting unit 73.

[0062] Here, the first identification unit 71 is used to obtain a liquor product image, and perform OCR identification of the liquor product image with a predetermined OCR identification method, thereby obtaining characters contained in the liquor product image. The second identification unit 72 is used to obtain image features contained in the liquor product image by performing deep learning identification of the liquor product image with a predetermined deep learning identification method. The first sorting unit 73 is used to sort a target liquor label that matches the character and the image features from a predetermined liquor label database based on the character and the image features, and set the target liquor label as the liquor label corresponding to the liquor product image.

[0063] In one embodiment, the first sorting unit 73 includes a first sorting subunit for sorting, based on the character, a predetermined liquor label that matches the character from a predetermined liquor label database, and organizing the predetermined liquor labels into a set to obtain a target liquor label set; and a second sorting subunit for sorting, based on the image feature, a liquor label that has the highest matching degree with the image feature from the target liquor label set as a target liquor label, and making the target liquor label the liquor label corresponding to the liquor product image.

[0064] In one embodiment, the liquor product images include multiple images of the same target liquor product, and the first sorting unit 73 is used to sort a target liquor label that matches the characters and image features from a predetermined liquor label database based on the characters and image features corresponding to each of the images, and to determine the target liquor label corresponding to each of the images as the liquor label corresponding to the image.

[0065] The liquor label identification device 70 further includes an attribution unit for attributing liquor labels belonging to the same liquor label type among all the liquor labels corresponding to each of the images to liquor labels of the same type, a statistical unit for calculating the number of liquor labels contained in each type of liquor label of the same type, and a second sorting unit for selecting the liquor label with the largest numerical value among all the liquor labels of the same type based on the number of liquor labels contained in each type of liquor label of the same type, and selecting it as a target liquor product label corresponding to the target liquor product.

[0066] Please refer to Fig. 8. Fig. 8 is a block schematic diagram of a liquor product information management device applied to a server side according to an embodiment of the present application. Corresponding to the liquor product information management method applied to the server side, an embodiment of the present application further provides a liquor product information management device applied to a server side. As shown in Fig. 8, the liquor product information management device 80 applied to the server side includes a unit for executing the liquor product information management method applied to the server side, and the liquor product information management device 80 is disposed in a computer device such as a server. Specifically, referring to Fig. 8, the liquor product information management device 80 includes an acquisition unit 81, a third identification unit 82, and a management unit 83.

[0067] Here, the acquisition unit 81 is used to acquire alcoholic product information including an alcoholic product image. The third identification unit 82 is used to obtain an alcoholic label contained in the alcoholic product image by identifying the alcoholic product image with the alcoholic label identification method. The management unit 83 is used to manage the alcoholic product information based on the alcoholic label.

[0068] Please refer to Fig. 9. Fig. 9 is a block schematic diagram of a liquor product information management device applied to a terminal according to an embodiment of the present application. Corresponding to the liquor product information management method applied to the above-mentioned terminal, an embodiment of the present application further provides a liquor product information management device applied to a terminal. As shown in Fig. 9, the liquor product information management device 90 applied to the terminal includes a unit for executing the liquor product information management method applied to the above-mentioned terminal, and the liquor product information management device 90 is disposed in a terminal device such as a smartphone. Specifically, referring to Fig. 9, the liquor product information management device 90 includes a transmission unit 91, a receiving unit 92, and a display unit 93.

[0069] Here, the sending unit 91 is used to send a liquor product information acquisition request to a predetermined server in response to a user's operation, so that the predetermined server acquires target liquor product information corresponding to the liquor product information acquisition request, including liquor product information associated with the predetermined liquor label obtained by identifying the liquor product image by the liquor label identification method, based on the liquor product information acquisition request, and returns the target liquor product information to the terminal. The receiving unit 92 is used to receive the target liquor product information sent from the predetermined server. The display unit 93 is used to display the target liquor product information, so that the user can acquire the target liquor product information.

[0070] In one application example, both the alcoholic product information management method applied to the server side and the alcoholic product information management method applied to the terminal are used in an alcoholic product information management system including a sake brewery, a terminal, and a server. Here, a sake cabinet for placing alcoholic products is installed in the sake brewery. A camera is installed at a predetermined position in the sake brewery. An alcoholic product image corresponding to the alcoholic product is taken through the camera, and the alcoholic product image is uploaded to the server. The server is used to execute steps of the alcoholic product information management method applied to the server side described in each of the above embodiments. The terminal is used to execute steps of the alcoholic product information management method applied to the terminal described in each of the above embodiments.

[0071] In addition, the specific implementation processes of the above-mentioned liquor label identification device and each unit, the specific implementation processes of the liquor product information management device applied to the server side and each unit, and the specific implementation processes of the liquor product information management device applied to the terminal and each unit will be clear to those skilled in the art by referring to the corresponding descriptions in the embodiments of the corresponding methods in each of the above-mentioned embodiments, and will not be described in detail here for the sake of convenience and conciseness of explanation.

[0072] At the same time, the classification and connection form of each unit in the liquor label identification device, the liquor product information management device applied to the server side, or the liquor product information management device applied to the terminal is used for illustrative purposes only. In other embodiments, the liquor label identification device, the liquor product information management device applied to the server side, or the liquor product information management device applied to the terminal may be classified into different units as necessary. Each unit of the liquor label identification device, the liquor product information management device applied to the server side, or the liquor product information management device applied to the terminal may use different connection sequences and methods to complete all or part of the functions of the liquor label identification device, the liquor product information management device applied to the server side, or the liquor product information management device applied to the terminal.

[0073] The liquor label identification device, the liquor product information management device applied to the server side, or the liquor product information management device applied to the terminal can each be realized in the form of a computer program executable on a computer device such as that shown in Figure 10.

[0074] Please refer to Fig. 10. Fig. 10 is a block schematic diagram of a computing device according to an embodiment of the present application. The computing device 500 may be a computing device such as a desktop computer or a server, or may be a component or part of other equipment.

[0075] 10, the computer device 500 includes a processor 502, a memory, and a network interface 505, which are connected via a system bus 501. Here, the memory may include a non-volatile storage medium 503 and an internal storage device 504, and may be a volatile computer-readable storage medium.

[0076] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute the above-mentioned liquor label identification method, the above-mentioned liquor product information management method applied to the server side, or the above-mentioned liquor product information management method applied to the terminal. The processor 502 is used to provide calculation and control capabilities for supporting the operation of the entire computer device 500. The internal storage device 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the above-mentioned liquor label identification method, the above-mentioned liquor product information management method applied to the server side, or the above-mentioned liquor product information management method applied to the terminal. The network interface 505 is used for network communication with other devices. The configuration shown in FIG. 10 is merely a block diagram of a part of the configuration related to the embodiment of the present application, and does not constitute a limitation of the computer device 500 to which the embodiment of the present application is applied. A particular computing device 500 may include more or fewer components than shown, or may combine some components or have a different component arrangement, as will be apparent to one of ordinary skill in the art. For example, in some embodiments, the computing device includes only a memory and a processor. In such embodiments, the structure and functionality of the memory and processor will be consistent with the embodiment shown in FIG. 10 and will not be described herein.

[0077] In one embodiment, the processor 502 is used to execute a computer program 5032 stored in the memory, so that in realizing the above-mentioned liquor label identification method, the processor 502 executes the steps of: acquiring a liquor product image, and performing OCR identification of the liquor product image with a predetermined OCR identification method to obtain characters contained in the liquor product image, performing deep learning identification of the liquor product image with a predetermined deep learning identification method to obtain image features contained in the liquor product image, and selecting a target liquor label that matches the characters and the image features from a predetermined liquor label database based on the characters and the image features, and setting the target liquor label as a liquor label corresponding to the liquor product image.

[0078] In one embodiment, when the processor 502 realizes the step of selecting a target liquor label that matches the character and the image feature from a predetermined liquor label database based on the character and the image feature, and setting the target liquor label as the liquor label corresponding to the liquor product image, the processor 502 specifically realizes the steps of selecting a predetermined liquor label that matches the character from a predetermined liquor label database based on the character, and organizing the predetermined liquor labels into a set to obtain a target liquor label set, and selecting a liquor label that has the highest degree of matching with the image feature from the target liquor label set based on the image feature as the target liquor label, and setting the target liquor label as the liquor label corresponding to the liquor product image.

[0079] In one embodiment, the alcoholic product images include a plurality of images of the same target alcoholic product, and the processor 502, when realizing the step of setting the target alcoholic label to the alcoholic label corresponding to the alcoholic product image, specifically realizes the step of setting the target alcoholic label corresponding to each image as the alcoholic label corresponding to the image. After realizing the step of setting the target alcoholic label corresponding to each image as the alcoholic label corresponding to the image, the processor 502 further realizes the steps of attributing the alcoholic labels belonging to the same alcoholic label among all the alcoholic labels corresponding to each of the images to the same type of alcoholic label, the step of calculating the number of alcoholic labels included in each type of the same type of alcoholic label, and the step of selecting the alcoholic label with the largest numerical value among all the same type of alcoholic labels based on the number of alcoholic labels included in each type of the same type of alcoholic label, and setting it as the target alcoholic product label corresponding to the target alcoholic product.

[0080] In one embodiment, the processor 502 is used to execute a computer program 5032 stored in a memory. In order to realize the above-mentioned server-side alcohol product information management method, the processor 502 executes the steps of obtaining alcohol product information including an alcohol product image, identifying the alcohol product image by the alcohol label identification method described in the above embodiment to obtain an alcohol label included in the alcohol product image, and managing the alcohol product information based on the alcohol label.

[0081] In one embodiment, the processor 502 is used to execute a computer program 5032 stored in memory. In order to realize the alcoholic product information management method applied to the above terminal, the processor 502 executes the steps of: sending an alcoholic product information acquisition request to a predetermined server in response to a user's operation, so that the predetermined server acquires target alcoholic product information corresponding to the alcoholic product information acquisition request, including alcoholic product information associated based on a predetermined alcoholic label obtained by identifying an alcoholic product image by the alcoholic label identification method described in the above embodiment, based on the alcoholic product information acquisition request, and returning the target alcoholic product information to the terminal; receiving the target alcoholic product information sent from the predetermined server; and displaying the target alcoholic product information so that the user can acquire the target alcoholic product information.

[0082] It should be noted that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), other general-purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. Here, the general-purpose processor may be a microprocessor, or any conventional processor, etc.

[0083] The implementation of all or part of the flow of the method of the above embodiment can be completed by a computer program stored in a computer-readable storage medium. This is obvious to those skilled in the art. The computer program is executed by at least one processor of the computer system to implement the steps of the above embodiment of the method.

[0084] Therefore, the present application further provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the liquor label identification method described in each of the above embodiments, or the liquor product information management method applied to the server side described in each of the above embodiments, or the liquor product information management method applied to the terminal described in each of the above embodiments.

[0085] The computer-readable storage medium may be an internal storage unit of the device, such as a hard disk or memory of the device. The computer-readable storage medium may be an external storage device of the device, such as a socket hard disk mounted on the device, a smart memory card (SMC), a secure digital (SD) card, a flash memory card (Flash Card), etc. Furthermore, the computer-readable storage medium may include both an internal storage unit and an external storage device of the device.

[0086] The specific operation procedures of the above-mentioned devices, apparatuses and units are obvious to those skilled in the art with reference to the corresponding procedures in the above-mentioned method embodiments, and will not be described further herein for ease of explanation and brevity.

[0087] The storage medium is a tangible, non-instantaneous storage medium, and may be any type of tangible storage medium capable of storing a computer program, such as a U disk, a removable hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0088] The units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized by electronic hardware, computer software, or a combination of both, and in order to clearly describe the compatibility of hardware and software, the above description generally describes the configurations and steps of each example functionally. This is obvious to those skilled in the art. Whether these functions are performed by hardware or software is determined by the specific application and design constraints of the technical aspects. Those skilled in the art may use different methods to realize the described functions for each specific application, and such realization is not considered to be beyond the scope of the present application.

[0089] It should be understood that in some embodiments of the present application, the disclosed apparatus and method may be realized in other ways. For example, the above-mentioned apparatus embodiments are merely exemplary. For example, the division of each unit is merely a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined, integrated into another system, or some features may be ignored or not implemented.

[0090] The steps of the method in the embodiments of the present application may be rearranged, merged, or reduced according to actual needs. The units in the apparatus in the embodiments of the present application may be merged, divided, or reduced according to actual needs. In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit.

[0091] This integrated unit can be realized as a software functional unit and stored in one storage medium when sold or used as an independent product. Based on this understanding, the technical aspects of the present application can be embodied essentially or in part by contributing to existing technology, or all or part of the technical aspects can be embodied in the form of a software product. This computer software product stored in one storage medium includes some instructions for causing an electronic device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0092] The above are only specific embodiments of the present application, and the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be in accordance with the scope of protection of the claims.

[0093] (Additional Note) (Appendix 1) obtaining a sake product image and performing OCR classification of the sake product image using a predetermined OCR classification method to obtain characters contained in the sake product image; performing deep learning classification of the alcoholic beverage product image using a predetermined deep learning classification method to obtain image features contained in the alcoholic beverage product image; and selecting a target sake label that matches the character and the image feature from a predetermined sake label database based on the character and the image feature, and setting the target sake label as a sake label corresponding to the sake product image. A method for identifying liquor labels.

[0094] (Appendix 2) The step of selecting a target sake label that matches the character and the image feature from a predetermined sake label database based on the character and the image feature, and setting the target sake label as a sake label corresponding to the sake product image includes: based on the character, selecting predetermined sake labels that match the character from a predetermined sake label database, and organizing the predetermined sake labels into a set to obtain a target sake label set; and selecting a sake label having the highest degree of matching with the image features from the set of target sake labels as a target sake label based on the image features, and setting the target sake label as a sake label corresponding to the sake product image. The method for identifying a liquor label described in Appendix 1.

[0095] (Appendix 3) the alcoholic product images include a plurality of images of the same target alcoholic product; The step of setting the target liquor label as a liquor label corresponding to the liquor product image includes: A step of setting a target liquor label corresponding to each image as a liquor label corresponding to the image, After the step of setting the target liquor label corresponding to each of the images as the liquor label corresponding to the image, A step of attributing, among the sake labels corresponding to each of all the images, sake labels belonging to the same sake label type to the same sake label type; A step of collecting statistics on the number of sake labels included in each type of sake label of the same type; Further comprising a step of selecting the liquor label with the largest numerical value from among all the liquor labels of the same type based on the number of liquor labels included in each type of the same type of liquor labels, and setting it as the target liquor product label corresponding to the target liquor product. 3. The method for identifying a liquor label according to claim 1 or 2.

[0096] (Appendix 4) A method for managing alcoholic beverage information applied to a server side, comprising: obtaining alcoholic product information including an image of the alcoholic product; A step of obtaining a sake label included in the sake product image by identifying the sake product image using the sake label identification method described in any one of Supplementary Note 1 to 3; and managing the liquor product information based on the liquor label. A method for managing alcoholic beverage product information.

[0097] (Appendix 5) A method for managing alcoholic beverage information applied to a terminal, comprising: a step of transmitting a request for obtaining alcoholic product information to a predetermined server in response to a user's operation, whereby the predetermined server obtains target alcoholic product information corresponding to the request for obtaining alcoholic product information based on the request for obtaining alcoholic product information, the target alcoholic product information including associated alcoholic product information based on a predetermined alcoholic label obtained by identifying an alcoholic product image using the alcoholic label identification method described in any one of Supplementary Notes 1 to 3, and returns the target alcoholic product information to the terminal; receiving the target alcoholic product information transmitted from the predetermined server; Displaying the target alcoholic product information so that the user can obtain the target alcoholic product information; A method for managing alcoholic beverage product information.

[0098] (Appendix 6) A first identification unit for obtaining a liquor product image and performing OCR identification of the liquor product image through a predetermined OCR identification method to obtain characters contained in the liquor product image; A second classification unit for obtaining image features contained in the alcoholic beverage product image by performing deep learning classification of the alcoholic beverage product image in a predetermined deep learning classification manner; and a first sorting unit for sorting a target sake label that matches the character and the image feature from a predetermined sake label database based on the character and the image feature, and making the target sake label a sake label that corresponds to the sake product image. A liquor label identification device.

[0099] (Appendix 7) A liquor product information management device applied to a server side, an acquisition unit for acquiring alcoholic beverage product information including alcoholic beverage product images; A third identification unit for obtaining a sake label included in the sake product image by identifying the sake product image using the sake label identification method described in any one of Supplementary Notes 1 to 3; A management unit for managing the liquor product information based on the liquor label. A liquor product information management device.

[0100] (Appendix 8) A liquor product information management device applied to a terminal, a transmission unit for transmitting a request for obtaining alcoholic product information to a predetermined server in response to a user's operation, the predetermined server including associated alcoholic product information based on a predetermined alcoholic label obtained by identifying an alcoholic product image using the alcoholic label identification method described in any one of appendices 1 to 3, obtaining target alcoholic product information corresponding to the request for obtaining alcoholic product information based on the request for obtaining alcoholic product information, and returning the target alcoholic product information to the terminal; A receiving unit for receiving the target alcoholic product information transmitted from the predetermined server; A display unit for displaying the target alcoholic product information, so that the user can obtain the target alcoholic product information; A liquor product information management device.

[0101] (Appendix 9) a memory for storing a computer program; a processor connected to the memory for executing the computer program to perform the steps of the method according to any one of claims 1 to 3, or the steps of the method according to claim 4, or the steps of the method according to claim 5; A computing device comprising:

[0102] (Appendix 10) 1. A computer-readable storage medium, comprising: The storage medium stores a computer program, When the computer program is executed by a processor, the steps of the method according to any one of appendices 1 to 3 are realized, or the steps of the method according to appendices 4 and 5 are realized, A computer-readable storage medium comprising:

Claims

1. A liquor label identification method comprising a method applied to a server and a method applied to a terminal, The method applied to the server comprises: a first identification step of acquiring an image of a sake product and performing OCR identification of the image of the sake product using a predetermined OCR identification method to obtain characters contained in the image of the sake product; A second classification step of performing deep learning classification of the alcoholic beverage product image using a predetermined deep learning classification method to obtain image features contained in the alcoholic beverage product image; A first selection step of selecting a target sake label that matches the character and the image feature from a predetermined sake label database based on the character and the image feature, and setting the target sake label as a sake label corresponding to the sake product image; The method applied to the terminal comprises: a step of transmitting a request for obtaining information on alcoholic product information to the server in response to a user's operation, so that the server returns to the terminal target alcoholic product information corresponding to the request for obtaining information on alcoholic product information, the target alcoholic product information including information associated based on the predetermined alcoholic product label obtained in the first selection step, in response to the request for obtaining information on alcoholic product information; receiving the target alcoholic product information sent from the server; Displaying the target alcoholic product information so that the user can obtain the target alcoholic product information; The first selection step includes: a target liquor label set acquisition step of selecting predetermined liquor labels that match the characters from the predetermined liquor label database based on the characters, and forming the predetermined liquor labels into a set to obtain a target liquor label set; a liquor label identification step of selecting, based on the image features, a liquor label having the highest degree of matching with the image features as a target liquor label from the target liquor label set, and setting the target liquor label as a liquor label corresponding to the liquor product image; The first selection step is a step executed before the target liquor label set acquisition step, A step of calculating a character similarity between the character and a predetermined sake label character corresponding to a predetermined sake label in the predetermined sake label database; determining whether the character similarity is equal to or greater than a predetermined character similarity threshold; If the character similarity is equal to or greater than the predetermined character similarity threshold, the predetermined liquor label character is set as a target character, and the target character is set as a character corresponding to the liquor product image, The liquor label identification step includes: calculating image similarities between the image features and predetermined liquor labels included in the target liquor label set to obtain all the image similarities; selecting a target image similarity having the highest image similarity from among all the image similarities; determining whether the target image similarity is greater than or equal to a predetermined image similarity threshold; If the target image similarity is equal to or greater than the predetermined image similarity threshold, a predetermined sake label corresponding to the target image similarity is set as a target sake label, and the target sake label is set as a sake label corresponding to the sake product image, the alcoholic product images include a plurality of images of the same target alcoholic product; In the liquor label identification step, a target liquor label corresponding to each image included in the liquor product image is determined as a liquor label corresponding to the image, The first sorting step is a step executed after the liquor label identifying step, A step of attributing, among the sake labels corresponding to each of all the images, sake labels belonging to the same sake label type to the same sake label type; A step of collecting statistics on the number of sake labels included in each type of sake label of the same type; Further comprising a step of selecting the liquor label with the largest numerical value from among all the liquor labels of the same type based on the number of liquor labels included in each type of the liquor labels of the same type, and setting the selected liquor label as the target liquor product label corresponding to the target liquor product. A method for identifying liquor labels.

2. The method applied to the server comprises: obtaining alcoholic product information including an image of the alcoholic product; A step of obtaining a sake label included in the sake product image by identifying the sake product image in the first identification step, the second identification step, and the first sorting step; and managing the liquor product information based on the liquor label.

2. The method for identifying a liquor label according to claim 1.

3. A liquor label identification system comprising a server and a terminal, The server, a first recognition unit for obtaining a liquor product image and performing OCR recognition on the liquor product image using a predetermined OCR recognition method to obtain characters contained in the liquor product image; A second classification unit for obtaining image features contained in the alcoholic beverage product image by performing deep learning classification of the alcoholic beverage product image in a predetermined deep learning classification manner; a first sorting unit for sorting a target sake label that matches the character and the image feature from a predetermined sake label database based on the character and the image feature, and making the target sake label a sake label that corresponds to the sake product image; The terminal includes: a sending unit for sending a request for obtaining information on alcoholic product information to the server in response to a user's operation, so that the server returns to the terminal target alcoholic product information corresponding to the request for obtaining information on alcoholic product information, the target alcoholic product information including information associated with the predetermined alcoholic product label obtained by the first sorting unit, in response to the request for obtaining information on alcoholic product information; a receiving unit for receiving the target alcoholic product information sent from the server; A display unit for displaying the target alcoholic product information, so that the user can obtain the target alcoholic product information; The first sorting unit comprises: a target liquor label set acquisition process for selecting predetermined liquor labels that match the characters from a predetermined liquor label database based on the characters, and forming a set of the predetermined liquor labels to obtain a target liquor label set; a liquor label identification process for selecting a liquor label having the highest degree of matching with the image features from the set of target liquor labels based on the image features, and setting the target liquor label as a liquor label corresponding to the liquor product image; The first sorting unit performs a process before the target liquor label set acquisition process, A process of calculating a character similarity between the character and a predetermined sake label character corresponding to a predetermined sake label in the predetermined sake label database; determining whether the character similarity is equal to or greater than a predetermined character similarity threshold; If the character similarity is equal to or greater than the predetermined character similarity threshold, the predetermined liquor label character is set as a target character, and the target character is set as a character corresponding to the liquor product image. The liquor label identification process includes: A process of calculating image similarities between the image features and predetermined liquor labels included in the target liquor label set to obtain all the image similarities; A process of selecting a target image similarity having the highest image similarity from among all the image similarities; determining whether the target image similarity is equal to or greater than a predetermined image similarity threshold; If the target image similarity is equal to or greater than the predetermined image similarity threshold, a predetermined sake label corresponding to the target image similarity is set as a target sake label, and the target sake label is set as a sake label corresponding to the sake product image, the alcoholic product images include a plurality of images of the same target alcoholic product; In the liquor label identification process, a target liquor label corresponding to each image included in the liquor product image is set as a liquor label corresponding to the image, The first sorting unit performs the following process after the liquor label identification process: A process of classifying, among the sake labels corresponding to each of all of the images, sake labels belonging to the same sake label type as sake labels of the same type; A process of collecting statistics on the number of sake labels included in each type of sake label of the same type; and further performing a process of selecting the liquor label having the largest numerical value from among all the liquor labels of the same type based on the number of liquor labels included in each type of the liquor labels of the same type, and setting the selected liquor label as a target liquor product label corresponding to the target liquor product. A liquor label identification system.

4. The server, an acquisition unit for acquiring alcoholic beverage product information including alcoholic beverage product images; a third identification unit for obtaining a liquor label included in the liquor product image by identifying the liquor product image using the first identification unit, the second identification unit, and the first sorting unit; A management unit for managing the liquor product information based on the liquor label.

4. The liquor label identification system according to claim 3.

Images