LLM-based intelligent label identification system

Abstract

The invention relates to the technical field of intelligent label identification, and particularly discloses an LLM-based intelligent label identification system, which comprises a visual identification module which comprises a Chinese character image library which stores Chinese character images appearing in all historical labels; obtaining all label images, and extracting Chinese characters from the label images by using a computer vision technology to obtain a Chinese character result; the semantic recognition module is used for performing semantic analysis on the Chinese character result obtained by the visual recognition module through a semantic analysis model to obtain semantic features; the identification verification module is used for acquiring a sample image of a specific area and establishing a specific area identification model; identifying the label image by using a specific region identification model, obtaining a specific region of the label image, extracting coding information from the specific region, and performing feature extraction to obtain coding semantic features; and comparing the coded semantic features with the previously obtained semantic features, and judging whether the semantic features are abnormal or not according to a comparison result.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent label recognition, and in particular to an intelligent label recognition system based on LLM. Background Art

[0002] Traditional machine learning methods rely on manual feature engineering, which requires expert experience for feature selection and extraction, which is time-consuming and labor-intensive. When faced with high-dimensional data, the performance of traditional machine learning algorithms may decline because they cannot handle high-dimensional data well. In addition, traditional machine learning algorithms such as linear regression and logistic regression often cannot effectively handle nonlinear relationships. These factors all increase the complexity and cycle of building recognition systems.

[0003] The existing intelligent tag recognition technology field is widely used, but it also has significant defects. For different types of tags, multiple independent training models are usually required, which makes the recognition system complex and time-consuming to build, and difficult to adapt to diverse tag requirements, limiting the versatility and flexibility of the same application in different scenarios. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent label recognition system based on LLM to solve the above technical problems.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] LLM-based intelligent label recognition system, including:

[0007] Visual recognition module: including a Chinese character image library and a recognition unit; the Chinese character image library includes images of all label Chinese characters, and the label Chinese characters are all Chinese characters that have appeared in the historical labels; the recognition unit obtains label images of all labels, and extracts the Chinese characters in the label images according to the Chinese character image library by computer vision technology, and records them as Chinese character results;

[0008] Semantic recognition module: establish a semantic analysis model, perform semantic analysis on the Chinese character results through the semantic analysis model, and obtain semantic features;

[0009] Identification and verification module: obtain sample images of specific areas, where the specific areas are the QR code area and the bar code area; establish a recognition model, train the recognition model according to the sample images, and obtain a final recognition model for the specific area;

[0010] The label image is identified by the specific region identification model to obtain the specific region of the label image; the coding information in the specific region is obtained, and the feature of the coding information is extracted to obtain the coding semantic feature; the coding semantic feature is compared with the semantic feature, and whether the semantic feature is abnormal is determined according to the comparison result;

[0011] If the judgment result is that there is no abnormality, the user's prompt word is obtained, and the tag with the highest semantic similarity between the key information and the prompt is selected and recorded as the target tag, and the target tag is prompted to the user.

[0012] As a further solution of the present invention: the Chinese character image library also includes:

[0013] The Chinese character image is divided into rectangles to obtain a rectangular grid, in which each rectangular grid is recorded as a pixel to obtain a plurality of pixels; in the Chinese character image, the pixel value of the pixel where the Chinese character is located is recorded as 1, and the pixel value of the remaining pixels is recorded as 0; in the Chinese character image, all pixels with a pixel value of 1 are regarded as pixel points, and the pixel points are connected with a smooth curve to obtain a Chinese character curve of each Chinese character image, which is recorded as a Chinese character curve sample, and a Chinese character curve library is obtained.

[0014] As a further solution of the present invention: the process of extracting the Chinese character result of the label image includes:

[0015] Obtain the Chinese character curve of the label image; overlap all Chinese character curve samples in the Chinese character curve library with the starting point of the Chinese character curve; obtain the number of overlapped pixels between the Chinese character curve and the Chinese character curve samples, select the Chinese character curve sample with the largest number of pixels overlapped with the Chinese character curve, and record it as the Chinese character result.

[0016] As a further solution of the present invention: the process of determining whether the pixel points overlap includes:

[0017] A rectangular coordinate system is established with the starting point as the origin, and the coordinates of each pixel point on the Chinese character curve and the Chinese character curve sample are obtained, which are recorded as (x1, y1) and (x2, y2) respectively, and the difference C=|x1-x2|+|y1-y2| is obtained; if the difference C=0, the pixel points corresponding to the two coordinates coincide; otherwise, the pixel points corresponding to the two coordinates do not coincide.

[0018] As a further solution of the present invention: when there is more than one Chinese character in the Chinese character image or label image, the Chinese character curve sample or Chinese character curve is a segmented curve.

[0019] As a further solution of the present invention: the semantic analysis model is based on the LLM large language model, and the recognition model is based on the OpenCV library.

[0020] As a further solution of the present invention: the process of comparing the encoded semantic feature with the semantic feature includes obtaining the cosine value of the angle between the encoded semantic feature and the semantic feature, recording it as semantic similarity, comparing the semantic similarity with a preset similarity threshold, and determining whether the semantic feature is abnormal.

[0021] As a further solution of the present invention: the process of determining whether the semantic feature is abnormal includes:

[0022] If the semantic similarity is less than or equal to the similarity threshold, then the semantic feature is abnormal; if the semantic similarity is greater than the similarity threshold, then the semantic feature is not abnormal.

[0023] Beneficial effects of the present invention:

[0024] Compared with the prior art, the present invention can quickly and accurately extract Chinese character information from label images through a visual recognition module; thanks to the support of computer vision technology and a rich Chinese character image library, the recognition process is more efficient; the semantic recognition module further performs semantic analysis on the extracted Chinese character results to obtain accurate semantic features; this multi-level recognition method greatly improves the accuracy of overall recognition; the recognition verification module obtains sample images of specific areas (such as QR code and barcode areas), establishes and trains recognition models, and ensures that specific areas of the label image can be accurately recognized; by comparing the encoded semantic features with the semantic features, it can be determined whether the semantic features are abnormal, thereby further improving the reliability of the recognition results; when the judgment result is normal, the user's prompt word will be obtained, and the label with the highest semantic similarity to the prompt will be selected as the target label, and prompted to the user; this intelligent prompt method greatly improves the user experience, allowing users to find the required label information more conveniently. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention will be further described below in conjunction with the accompanying drawings.

[0026] Figure 1 It is a schematic diagram of the process of the intelligent label recognition system based on LLM of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] See also Figure 1As shown, the present invention is an intelligent label recognition system based on LLM, comprising:

[0029] Visual recognition module: including a Chinese character image library and a recognition unit; the Chinese character image library includes images of all label Chinese characters, and the label Chinese characters are all Chinese characters that have appeared in the historical labels; the recognition unit obtains label images of all labels, and extracts the Chinese characters in the label images according to the Chinese character image library by computer vision technology, and records them as Chinese character results;

[0030] It can be understood that the Chinese character image library is a database containing Chinese character images that have appeared in all historical tags; these images are collected and stored in advance for subsequent recognition processes; the Chinese character image library provides a reference standard for the recognition unit, so that the recognition unit can match and recognize based on these known Chinese character images; in the process of extracting Chinese characters, the recognition unit will use a series of computer vision technologies, such as image preprocessing (including denoising, enhancement, etc.), feature extraction (such as edge detection, corner detection, etc.) and pattern matching (such as template matching, feature matching, etc.);

[0031] As a preferred embodiment of the present invention, the Chinese character image library further includes:

[0032] The Chinese character image is divided into rectangles to obtain a rectangular grid, in which each rectangular grid is recorded as a pixel, and a plurality of pixels are obtained; in the Chinese character image, the pixel value of the pixel where the Chinese character is located is recorded as 1, and the pixel value of the remaining pixels is recorded as 0; in the Chinese character image, all pixels with a pixel value of 1 are regarded as pixel points, and the pixel points are connected with a smooth curve to obtain a Chinese character curve of each Chinese character image, which is recorded as a Chinese character curve sample, and a Chinese character curve library is obtained;

[0033] The extraction process of the Chinese character result of the label image includes:

[0034] Obtain the Chinese character curve of the label image; overlap all Chinese character curve samples in the Chinese character curve library with the starting point of the Chinese character curve; obtain the number of overlapped pixels between the Chinese character curve and the Chinese character curve samples, select the Chinese character curve sample with the largest number of overlapped pixels with the Chinese character curve, and record it as the Chinese character result;

[0035] The process of determining whether the pixels overlap includes:

[0036] A rectangular coordinate system is established with the starting point as the origin, and the coordinates of each pixel point on the Chinese character curve and the Chinese character curve sample are obtained, which are recorded as (x1, y1) and (x2, y2) respectively, and the difference C=|x1-x2|+|y1-y2| is obtained; if the difference C=0, the pixel points corresponding to the two coordinates coincide; otherwise, the pixel points corresponding to the two coordinates do not coincide;

[0037] When there is more than one Chinese character in the Chinese character image or label image, the Chinese character curve sample or Chinese character curve is a segmented curve;

[0038] It can be understood that by aligning the starting points of the Chinese character curves, it can be ensured that each Chinese character curve sample is consistent with the Chinese character curve in the label image in spatial position, ensuring that the comparison between different Chinese character curves is based on the same reference point, thereby improving the matching accuracy and effectively reducing the problem of errors in matching results caused by position deviation;

[0039] It is understandable that Chinese characters are composed of strokes, and the number and structure of strokes of each Chinese character are different. When the label image or Chinese character image contains multiple Chinese characters, the strokes of these Chinese characters may cross, overlap or connect with each other, resulting in the Chinese character curve showing segmented characteristics; the advantage of this method is that it can make full use of the shape information of Chinese characters for recognition, rather than relying solely on simple pixel value comparison; through the precise comparison of Chinese character curves, the accuracy and robustness of recognition can be greatly improved; in addition, due to the use of the starting point alignment strategy, it can also deal with the translation problem of Chinese characters in the image well;

[0040] Semantic recognition module: establish a semantic analysis model, perform semantic analysis on the Chinese character results through the semantic analysis model, and obtain semantic features;

[0041] Identification and verification module: obtain sample images of specific areas, where the specific areas are the QR code area and the bar code area; establish a recognition model, train the recognition model according to the sample images, and obtain a final recognition model for the specific area;

[0042] It is understandable that, through a camera or other image acquisition device, image data containing a specific area can be captured in real time, and the image data will serve as the basis for training the recognition model; the recognition model is trained using the collected sample images. During the training process, the model will learn how to accurately extract information from specific areas such as QR codes and barcodes from the image; through continuous iterative optimization, the performance of the model will gradually improve; the trained specific area recognition model can analyze and process newly captured images in real time, quickly and accurately locate and extract key information such as QR codes and barcodes;

[0043] As a preferred embodiment of the present invention, the semantic analysis model is based on the LLM large language model, and the recognition model is based on the OpenCV library;

[0044] It should be noted that the semantic analysis model is based on the Large Language Model (LLM). LLM is an advanced natural language processing technology that can understand and generate human language. It has mastered a wealth of vocabulary, grammar and semantic knowledge by learning a large amount of text data, so it can accurately analyze and understand the input text or image. The recognition model is based on the OpenCV library, which is an open source computer vision library that provides a wealth of image processing and machine learning algorithms, suitable for various image recognition tasks.

[0045] The label image is identified by the specific region identification model to obtain the specific region of the label image; the coding information in the specific region is obtained, and the feature of the coding information is extracted to obtain the coding semantic feature; the coding semantic feature is compared with the semantic feature, and whether the semantic feature is abnormal is determined according to the comparison result;

[0046] If the judgment result is that there is no abnormality, the user's prompt word is obtained, and the tag with the highest semantic similarity between the key information and the prompt is selected and recorded as the target tag, and the target tag is prompted to the user;

[0047] It is worth noting that when anomalies are detected in semantic features, corresponding prompt information can be given to inform the user that the data source, feature extraction process or model parameter settings need to be checked; for abnormal situations, a more in-depth analysis can be conducted to find out the specific causes and make corrections; for example, retraining the model, adjusting the feature extraction method or optimizing the data processing process;

[0048] As a preferred embodiment of the present invention, the process of comparing the encoded semantic feature with the semantic feature includes obtaining a cosine value of an angle between the encoded semantic feature and the semantic feature, recording the cosine value as a semantic similarity, comparing the semantic similarity with a preset similarity threshold, and determining whether the semantic feature is abnormal;

[0049] The process of determining whether the semantic feature is abnormal includes:

[0050] If the semantic similarity is less than or equal to the similarity threshold, then the semantic feature is abnormal; if the semantic similarity is greater than the similarity threshold, then the semantic feature is not abnormal;

[0051] It should be noted that the encoded semantic features and semantic features are both vectors, and the cosine value of the angle is the cosine value of the angle between the two vectors. The cosine value of the angle is a value between 0 and 1. The closer this value is to 1, the more consistent the directions of the two vectors, that is, the higher the similarity; if the cosine similarity is lower than or equal to the preset threshold, it indicates that the angle between the two feature vectors is large and the direction difference is obvious, thereby inferring that the semantic features may be abnormal; this may mean that the extracted features do not meet expectations and require further inspection and processing.

[0052] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. The intelligent label recognition system based on LLM is characterized by: include: Visual recognition module: including a Chinese character image library and a recognition unit; the Chinese character image library includes images of all label Chinese characters, and the label Chinese characters are all Chinese characters that have appeared in the historical labels; the recognition unit obtains label images of all labels, and extracts the Chinese characters in the label images according to the Chinese character image library by computer vision technology, and records them as Chinese character results; Semantic recognition module: establish a semantic analysis model, perform semantic analysis on the Chinese character results through the semantic analysis model, and obtain semantic features; Identification and verification module: obtains sample images of specific areas, where the specific areas are the QR code area and the barcode area; Establishing a recognition model, and training the recognition model according to the sample image to obtain a final specific area recognition model; The label image is identified by the specific region identification model to obtain the specific region of the label image; the encoding information in the specific region is obtained, and the feature extraction of the encoding information is performed to obtain the encoding semantic feature; Comparing the encoded semantic feature with the semantic feature, and judging whether the semantic feature is abnormal according to the comparison result; If the judgment result is that there is no abnormality, the user's prompt word is obtained, and the tag with the highest semantic similarity between the key information and the prompt is selected and recorded as the target tag, and the target tag is prompted to the user.

2. The LLM-based smart label recognition system according to claim 1, characterized in that: The Chinese character image library also includes: The Chinese character image is divided into rectangles to obtain a rectangular grid, in which each rectangular grid is recorded as a pixel to obtain a plurality of pixels; in the Chinese character image, the pixel value of the pixel where the Chinese character is located is recorded as 1, and the pixel value of the remaining pixels is recorded as 0; in the Chinese character image, all pixels with a pixel value of 1 are regarded as pixel points, and the pixel points are connected with a smooth curve to obtain a Chinese character curve of each Chinese character image, which is recorded as a Chinese character curve sample, and a Chinese character curve library is obtained.

3. The LLM-based smart label recognition system according to claim 2 is characterized in that: The extraction process of the Chinese character result of the label image includes: Obtain the Chinese character curve of the label image; overlap all Chinese character curve samples in the Chinese character curve library with the starting point of the Chinese character curve; obtain the number of overlapped pixels between the Chinese character curve and the Chinese character curve samples, select the Chinese character curve sample with the largest number of pixels overlapped with the Chinese character curve, and record it as the Chinese character result.

4. The LLM-based smart label recognition system according to claim 3 is characterized in that: The process of determining whether the pixels overlap includes: A rectangular coordinate system is established with the starting point as the origin, and the coordinates of each pixel point on the Chinese character curve and the Chinese character curve sample are obtained, which are recorded as (x1, y1) and (x2, y2) respectively, and the difference C=|x1-x2|+|y1-y2| is obtained; if the difference C=0, the pixel points corresponding to the two coordinates coincide; otherwise, the pixel points corresponding to the two coordinates do not coincide.

5. The LLM-based smart label recognition system according to claim 3 is characterized in that: When there is more than one Chinese character in the Chinese character image or label image, the Chinese character curve sample or Chinese character curve is a segmented curve.

6. The LLM-based smart label recognition system according to claim 1, characterized in that: The semantic analysis model is based on the LLM large language model, and the recognition model is based on the OpenCV library.

7. The LLM-based smart label recognition system according to claim 7, characterized in that: The process of comparing the encoded semantic feature with the semantic feature includes obtaining a cosine value of an angle between the encoded semantic feature and the semantic feature, recording the cosine value as semantic similarity, comparing the semantic similarity with a preset similarity threshold, and determining whether the semantic feature is abnormal.

8. The LLM-based smart label recognition system according to claim 1, characterized in that: The process of determining whether the semantic feature is abnormal includes: If the semantic similarity is less than or equal to the similarity threshold, then the semantic feature is abnormal; if the semantic similarity is greater than the similarity threshold, then the semantic feature is not abnormal.

Images