A drug substance label consistency comparison method, system, device and medium

Abstract

The application provides a medicine raw material label consistency comparison method, system, device and medium, belongs to the medicine raw material label identification technical field, collects and saves the outer package picture; the edge server is based on the comparison picture, finds the local template library according to template index, obtains template label picture and characteristic vector file;Using the reconstructed Resnet18 network extracts the characteristic vector of the comparison label picture, calculates the similarity between the template label picture characteristic vector and the comparison label picture characteristic vector, obtains the label consistency result of the template picture and the comparison picture;The label consistency result is displayed in the result visualization. Through traditional image processing algorithm and deep learning network feature extraction technology, the workload of verification is reduced, and the verification efficiency is improved. Using the reconstructed Resnet18 network extracts the characteristic vector, and combining the SIFT key point matching algorithm, the consistency of the label style can be accurately judged.

Description

Technical Field

[0001] This invention belongs to the field of pharmaceutical raw material label identification technology, and in particular relates to a method, system, equipment and medium for comparing the consistency of pharmaceutical raw material labels. Background Technology

[0002] When pharmaceutical raw materials or packaging materials arrive at the pharmaceutical factory's warehouse in cartons or cardboard drums, warehouse staff need to inspect the labels of drugs of the same type, specification, and batch, requiring that the label style and content be consistent for each item.

[0003] Current feature extraction methods may not be able to fully capture the complex features of pharmaceutical ingredient label images. Images of pharmaceutical ingredient packaging may exhibit various distortions and rotations, and the label's position within the image may be inconsistent. This makes accurately locating the region corresponding to the template label image from the comparison images difficult, affecting the accuracy and efficiency of consistency comparisons. Furthermore, if similarity calculation methods, such as cosine similarity, are used, they cannot comprehensively measure the similarity between label images and are easily affected by factors such as feature vector dimensionality and data distribution, leading to misjudgments. Summary of the Invention

[0004] This invention provides a method for comparing the consistency of drug raw material labels, which can accurately determine whether the label styles of the template image and the comparison image are consistent, thereby improving the verification efficiency and accuracy.

[0005] The methods include:

[0006] S1. Collect and save the outer packaging image, and send the outer packaging image, image index, and template index to the edge server in the form of a form;

[0007] S2. The edge server determines whether the current outer packaging image is a template image or a comparison image based on whether the template index in the form is empty. If the template index is empty, the current outer packaging image is a template image and proceeds to S3; otherwise, it is a comparison image and proceeds to S5.

[0008] S3. Parse the template image using a preset algorithm to extract standard template tag images, and save the template tag images in the local template library in a preset format by naming them with the template index;

[0009] S4. Use the reconstructed ResNet18 network to extract the feature vectors of the template label images, name the template index, and save the feature vectors of the template label images in the local template library as npy format files.

[0010] S5. Search the local template library based on the template index to obtain template label images and feature vector files;

[0011] S6. Based on the SIFT key point matching algorithm, obtain the comparison label image that matches the template label image from the comparison image;

[0012] S7. Use the reconstructed ResNet18 network to extract the feature vectors of the comparison label images, calculate the similarity between the feature vectors of the template label images obtained in step S5 and the feature vectors of the comparison label images, and compare them with the pre-set similarity threshold to obtain the label consistency results between the template images and the comparison images.

[0013] S8. The label consistency results, image index, and template index are transmitted to the terminal device in form for visualization display.

[0014] It should be further explained that step S2 specifically includes:

[0015] Step S201: Receive and parse the uploaded form data, extract the outer packaging image, image index and template index fields, and verify the data format and integrity;

[0016] Step S202: Check if the template index field is empty. If it is empty, it is determined to be a template image and the template registration process is triggered. If it is not empty, it is determined to be a comparison image and the tag comparison process is triggered to achieve automated process diversion.

[0017] Step S203: Dynamically allocate processing paths based on the judgment results: Generate a unique template index for the template image and bind data, and compare the image to retrieve associated template data through the index;

[0018] Step S204: Generate a globally unique index for the template image, bind and store the index with the image data and feature files, and establish a multi-level association relationship for the index data.

[0019] It should be further explained that step S3 specifically includes:

[0020] S301: Apply mean filtering, bilateral filtering and Gaussian filtering to the template image to smooth the template image, and convert the smoothed image into a grayscale template image;

[0021] S302: For grayscale template images, edge detection algorithms based on the Scharr operator are used to process them to obtain binarized template images;

[0022] S303: Based on the binarized template image, a contour detection algorithm is applied to filter out quadrilateral candidate boxes that meet the conditions, and the quadrilateral with the largest area is selected as the coordinate box of the template label image.

[0023] S304: Combine the coordinates of the four corner points of the template label image with the width and height of the preset label image, calculate the radial transformation matrix, and apply the radial transformation algorithm to obtain a standard template label image on the template image.

[0024] It should be further noted that steps S4 and S7 use the reconstructed ResNet18 network to extract feature vectors from the labeled images, including the following process:

[0025] S401: Preprocess the label image, adjust the image size, and provide the label image. The preprocessed output is The preprocessing is represented by the following formula:

[0026]

[0027] in, The size is H , The size is ;

[0028] S402: Reconstruct the forward inference process of the deep learning network ResNet18, load the pre-trained ResNet18 model weights under the PyTorch framework, and use the reconstructed ResNet18 network to extract feature maps of two different levels from the template label image.

[0029] Feature extraction is expressed by the following formula:

[0030]

[0031]

[0032] in, This represents the model weights of the first convolutional layer. Represents the model weights at different levels. Feature maps representing different levels;

[0033] S403: Adaptive mean pooling is performed on the feature maps of layer 2 and layer 3, and then they are flattened according to the channel dimension and then stitched together to obtain the feature vector of the label image;

[0034] The eigenvector is calculated using the following formula:

[0035]

[0036]

[0037]

[0038] in, This indicates that by performing a process on the feature map along the channel dimension... Adaptive global average pooling, This indicates a feature concatenation operation. Indicates will The feature map of the size is flattened as ; Size is , Size is , Size is .

[0039] It should be further explained that step S5 specifically includes:

[0040] The template index is mapped to a bucket array of Cuckoo filters using a double hash function, with each bucket storing the fingerprint of the index and the associated secondary index;

[0041] When querying the template index, candidate positions in the bucket array are probed alternately. If the fingerprint matches, the associated secondary index is returned. If the probe path exceeds the threshold, the secondary retrieval process is triggered. High-frequency template indexes are preloaded into the memory cache based on the LRU strategy. The cache key is encoded using a combination of index hash value and timestamp.

[0042] In step 5, the hash verification of the npy file in the local template library is also performed based on the feature vector hash value in the metadata. If the verification fails, the data repair mechanism is triggered. If the verification passes, the feature vector is loaded through the memory-mapped file. The loading method is based on parallel loading, loading the original image file and feature vector file of the label image, implementing asynchronous prefetching through CUDA stream, and using OpenCL kernel functions to decode and normalize the image data.

[0043] When multiple versions of feature vectors corresponding to the same template index are detected, a process is initiated that compares the template creation time stored locally based on timestamps and hash values.

[0044] If the timestamp difference is less than the preset time, the version with higher confidence is selected based on the cosine similarity of the feature vectors.

[0045] It should be further explained that step S6 specifically includes:

[0046] S601: For template label images And compare images SIFT detectors were applied respectively. Detect key feature points and feature descriptors The detection of key points and the calculation of feature descriptors are represented by the following formula:

[0047]

[0048]

[0049] S602: Based on feature descriptors of the compared images and template label images, using the FLANN matcher Perform feature keypoint pairing; based on the metric distance between keypoints Filter matching point pairs;

[0050] The filtered matching point pairs are represented by the following formula:

[0051]

[0052] in, These are matching point pairs, representing the sets of key point coordinates on the template label image and the comparison image, respectively, and they exist in pairs;

[0053] S603: Based on the matching point pairs, calculate the homography matrix from the template label image to the comparison image, and use the homography matrix to transform the coordinates of the four corner points of the template label image to obtain the coordinates of the four corner points of the label on the comparison image.

[0054] The four corner points of the labels on the comparison images are obtained using the following formula:

[0055]

[0056]

[0057] in, This represents the formula for calculating the homography matrix. This indicates the application of the homography matrix. This represents the homography matrix transformed from the template label image to the comparison image. This represents the four corner points of the template label image. This indicates comparing the four corner points on the image;

[0058] S604: Based on the coordinates of the four corner points of the label on the comparison image and the width and height of the preset label image, calculate the affine transformation matrix, and use the affine transformation matrix to transform the coordinates of the four corner points of the label on the comparison image to obtain a standard comparison label image.

[0059] The acquisition and comparison of the label image is represented by the following formula:

[0060]

[0061]

[0062] in, To compare the label images, Denotes the affine transformation matrix. This represents the four corner points of the label image obtained based on its width and height. This indicates the method for calculating the affine transformation matrix.

[0063] It should be further explained that step S7 specifically includes:

[0064] S701: Use the reconstructed ResNet18 network to extract feature vectors from the alignment label images;

[0065] S702: Calculate the similarity between the feature vector of the template label image obtained in step S5 and the feature vector of the comparison label image, including cosine similarity. and structural similarity ;

[0066] The similarity calculation is expressed by the following formula:

[0067]

[0068]

[0069] in, This represents the formula for calculating cosine similarity. This represents the formula for calculating structural similarity. This represents the feature vector of the template label image. Represents the feature vector of the compared labeled image;

[0070] S703: Based on cosine similarity and structural similarity The results are compared with a preset threshold to obtain the consistency of the labeled images.

[0071] This application also provides a pharmaceutical raw material label consistency comparison system, the system comprising:

[0072] The image acquisition and storage module is used to acquire and save images of the outer packaging, and send the outer packaging images, image indexes, and template indexes to the edge server in the form of a form.

[0073] The edge server is used to determine whether the current outer packaging image is a template image or a comparison image based on whether the template index in the form is empty. If the template index is empty, the current outer packaging image is a template image, and the image parsing and saving module is executed; otherwise, it is a comparison image, and the feature lookup module is executed.

[0074] The image parsing and saving module is used to parse the template image into a standard template tag image based on a preset algorithm, and save the template tag image in a preset format in the local template library by naming it with the template index;

[0075] The feature extraction module is used to extract feature vectors of template label images using the reconstructed ResNet18 network, name the template index, and save the feature vectors of template label images in npy format files in the local template library;

[0076] The feature lookup module is used to search the local template library based on the template index and obtain template label images and feature vector files;

[0077] The image matching module is used to obtain the matching label image that matches the template label image from the comparison image based on the SIFT key point matching algorithm;

[0078] The image processing module is used to extract feature vectors of the comparison label images using the reconstructed ResNet18 network, calculate the similarity between the obtained template label image feature vector and the comparison label image feature vector, and compare it with a pre-set similarity threshold to obtain the label consistency result between the template image and the comparison image.

[0079] The image display module is used to visualize the label consistency results, image index, and template index by transmitting them to the terminal device in form.

[0080] According to another embodiment of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the pharmaceutical raw material label consistency comparison method.

[0081] According to another embodiment of this application, a storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of a method for comparing the consistency of drug raw material labels.

[0082] As can be seen from the above technical solutions, the present invention has the following advantages:

[0083] The pharmaceutical raw material label consistency comparison method provided by this invention uses a reconstructed ResNet18 network for feature extraction. The label images are preprocessed, resized, and normalized. Then, multi-layer convolution and pooling operations of the ResNet18 network are used to extract feature maps at different levels. Finally, adaptive mean pooling and concatenation are used to obtain feature vectors. This method can fully learn the complex features of the label images, extracting representative and robust features from images under different conditions, thus improving the accuracy of feature extraction. The SIFT keypoint matching algorithm is used to locate and compare the label images. A SIFT detector is used to detect the feature keypoints and descriptors of the template label image and the comparison image. Then, a FLANN matcher is used for keypoint pairing, and matching point pairs are selected based on metric distance. Next, the homography matrix and affine transformation matrix are calculated based on the matching point pairs, thereby accurately locating the label position in the comparison image and obtaining a standard comparison label image. This method can effectively handle various image deformations and rotations, accurately locate the label, and improve the accuracy and efficiency of label localization. Both cosine similarity and structural similarity are calculated to measure the similarity between labeled images. Cosine similarity measures the angle between feature vectors, reflecting the consistency of vector direction; structural similarity focuses more on the structural information of the image, considering the similarity of brightness, contrast, and structure. Combining the two allows for a comprehensive measurement of the similarity between images from different perspectives, improving the accuracy and reliability of consistency matching and reducing false positives.

[0084] By employing edge computing, data acquisition, preprocessing, and some computational tasks are performed on edge servers. The outer packaging image, image index, and template index are sent to the edge server in form. The edge server then performs operations such as parsing the template label images, extracting feature vectors, obtaining label images for comparison, and calculating similarity. The results are then sent to the terminal device for visualization. Attached Figure Description

[0085] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0086] Figure 1 Flowchart of the method for comparing the consistency of drug ingredient labels;

[0087] Figure 2 This is a schematic diagram of a pharmaceutical raw material label consistency comparison system.

[0088] Figure 3This is a schematic diagram of an electronic device. Detailed Implementation

[0089] The drug ingredient label consistency comparison method provided in this application reduces the workload and improves the efficiency of verification by using traditional image processing algorithms and deep learning network feature extraction techniques. By extracting feature vectors using a reconstructed ResNet18 network and combining them with the SIFT keypoint matching algorithm, the consistency of label styles can be accurately determined.

[0090] The following describes in detail the method for comparing the consistency of pharmaceutical ingredient labels involved in this application. Specific details, such as particular system structures and technologies, are presented for illustrative purposes rather than limiting, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details.

[0091] It should be understood that, when used in this specification, the term "comprising" indicates the presence of the described feature, integral, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0092] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0093] In embodiments of the present invention, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer. In cases involving a remote computer, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (exemplarily using an Internet service provider for Internet connection).

[0094] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0095] Please see Figure 1 The diagram shows a flowchart of a method for comparing the consistency of drug ingredient labels in a specific embodiment. The method includes:

[0096] S1. Collect and save the outer packaging image, and send the outer packaging image, image index, and template index to the edge server in the form of a form.

[0097] In this embodiment, a high-definition camera can be used to photograph the outer packaging of pharmaceutical raw materials. The captured images are saved according to certain naming rules, such as using timestamps, batch numbers, etc., as image indexes.

[0098] The template index is set according to the actual situation. If it is a new template image, the template index can be left blank for now; if it is a comparison image, then the corresponding template index should be filled in. The images, image indexes, and template indexes are compiled into a form and sent to the edge server via the network. This ensures the integrity and accuracy of the original data, and at the same time, transmitting data in form improves data transmission efficiency.

[0099] S2. The edge server determines whether the current outer packaging image is a template image or a comparison image based on whether the template index in the form is empty. If the template index is empty, the current outer packaging image is a template image and proceeds to S3; otherwise, it is a comparison image and proceeds to S5.

[0100] After receiving the form data, the edge server in this embodiment checks the template index field. A conditional statement implemented programmatically can be used to determine if the template index is a string with a length of 0. If it is, it is determined to be a template image; otherwise, it is determined to be a comparison image.

[0101] It's important to note that template images serve as standard references and do not need to be associated with other templates, so their template index is empty. Comparison images, on the other hand, need to be compared with specific template images, carrying their corresponding template indexes. This method of comparison allows for the differentiation of image types, achieving automatic image classification and improving processing efficiency.

[0102] As one embodiment of this application, step S201: Receive and parse the uploaded form data, extract the outer packaging image, image index and template index fields, and verify the data format and integrity.

[0103] This embodiment can parse form content through a preset HTTP interface to extract key fields, including the binary data of the outer packaging image, image index, and template index. The extracted fields are then format-validated to ensure the integrity of the image data and the validity of the template index format. The validity of the template index format can be determined based on string length and character type.

[0104] Step S202: Check if the template index field is empty. If it is empty, it is determined to be a template image and the template registration process is triggered. If it is not empty, it is determined to be a comparison image and the tag comparison process is triggered to achieve automated process diversion.

[0105] Here, a string processing function is called to check if the template index field is empty, such as an empty string, null, or undefined. If it is empty, the current outer packaging image is determined to be the first uploaded template image, triggering the template registration process and proceeding to S3. If it is not empty, the current outer packaging image is determined to be the sample image to be compared, triggering the tag comparison process and proceeding to S5.

[0106] Step S203: Dynamically allocate processing paths based on the judgment results: Generate a unique template index for the template image and bind data, and compare the image to retrieve associated template data through the index.

[0107] For the dynamic allocation of processing paths based on the judgment results in this embodiment, a new template index can be automatically generated, such as by using a hash algorithm or UUID to generate a unique identifier, and then bound to the current image index. The template index is then populated back into the form to ensure a strong correlation between subsequent template data storage and the index. Based on the template index, the corresponding template tag image and feature vector file are quickly retrieved from the local template library to ensure the accuracy of the comparison data source.

[0108] Step S204: Generate a globally unique index for the template image, bind and store the index with the image data and feature files, and establish a multi-level association relationship for the index data.

[0109] It should be noted that a globally unique template index is generated using a combination algorithm of timestamp, device ID, and random number to avoid index conflicts. The generated template index is mapped to the current image index and written to the local database, allowing subsequent comparison processes to trace back to the original template data using the template index.

[0110] As can be seen, different processing branches are automatically triggered based on the empty / non-empty state of the template index, eliminating the need to specify image types and reducing interaction complexity. This enables autonomous process routing, avoiding process chaos caused by operational errors and improving validation efficiency. Field parsing, index generation, and process allocation are completed on the edge server, reducing cloud communication latency and supporting high-concurrency scenarios.

[0111] S3. Parse the template image using a preset algorithm to extract standard template tag images, and save the template tag images in the local template library in a preset format by naming them with the template index.

[0112] Step S3 specifically includes the following steps:

[0113] S301: Apply mean filtering, bilateral filtering, and Gaussian filtering to the template image to smooth it, and then convert the smoothed image into a grayscale template image.

[0114] In some embodiments, median filtering replaces the grayscale value of each pixel in the template image with the median grayscale value of its neighboring pixels, effectively removing impulse noise such as salt-and-pepper noise. Bilateral filtering considers not only the spatial distance between pixels but also the similarity between pixel values, smoothing the image while preserving edge information well. Gaussian filtering applies a weighted average to the image using a Gaussian function, controlling the smoothing degree by adjusting the size and standard deviation of the Gaussian kernel, effectively removing Gaussian noise. Converting a smoothed color image to grayscale involves converting the RGB values ​​of each pixel in the color image into a grayscale value according to a certain weighted formula. This transforms the image from a three-dimensional color space to a one-dimensional grayscale space, facilitating subsequent processing.

[0115] S302: For grayscale template images, edge detection algorithms based on the Scharr operator are used to process them to obtain binarized template images.

[0116] In this embodiment, the horizontal direction of the Scharr operator is: d / dx = [-3, 0, 3; -10, 0, 10; -3,0, 3]. The vertical direction is: d / dy = [-3, -10, -3; 0, 0, 0; 3, 10, 3]. The gradient magnitude is calculated as: Gradient = sqrt((dx)^2 + (dy)^2).

[0117] Binarization can be achieved using the Otsu algorithm for automatic thresholding, converting the gradient magnitude map into a binary image. The absolute values ​​of the two gradient images are then added together to obtain a composite gradient image. Finally, the composite gradient image is binarized according to a set threshold: pixels with gradient values ​​greater than the threshold are set to white (representing edges), and those with gradient values ​​less than the threshold are set to black (representing non-edges), thus obtaining a binarized template image.

[0118] S303: Based on the binarized template image, a contour detection algorithm is applied to filter out quadrilateral candidate boxes that meet the conditions, and the quadrilateral with the largest area is selected as the coordinate box of the template label image.

[0119] In some embodiments, the contour detection algorithm finds contours by tracing the boundaries of black and white areas in the image, based on the image's binarization information. When identifying quadrilaterals, the algorithm utilizes their geometric features, namely, having four vertices. Selecting the quadrilateral with the largest area is based on prior knowledge of the relative size and position of the pharmaceutical ingredient label in the image, assuming that the label typically occupies a large area and has a relatively regular quadrilateral shape. Through contour detection and filtering, regions in the template image that may contain pharmaceutical ingredient labels can be accurately located, eliminating other irrelevant edges and noise interference in the image.

[0120] S304: Combine the coordinates of the four corner points of the template label image with the width and height of the preset label image, calculate the radial transformation matrix, and apply the radial transformation algorithm to obtain a standard template label image on the template image.

[0121] In this embodiment, the four vertices of the quadrilateral coordinate frame are arranged in the following order: [top-left, top-right, bottom-right, bottom-left]. The `cv2.getPerspectiveTransform` function is used to map the original quadrilateral to a standard rectangle of a preset size. The preset size must be consistent with the size of the labels in the template library. The `cv2.warpPerspective` function is used to perform the transformation, generating a standardized template label image.

[0122] As can be seen, the radial transformation can perform operations such as translation, rotation, scaling, and cropping on images. By calculating the radial transformation matrix, a correspondence is established between the coordinate frame of the template label image and the preset standard label size and orientation, thereby transforming the label area in the template image into a standard form. In subsequent processing, label images taken from different angles and sizes can be analyzed and compared under the same standard, improving the accuracy and reliability of the comparison.

[0123] S4. Use the reconstructed ResNet18 network to extract the feature vectors of the template label images, name the template index, and save the feature vectors of the template label images in the local template library as npy format files.

[0124] In this embodiment, steps S4 and S7 use the reconstructed ResNet18 network to extract feature vectors from the labeled images, including the following processes:

[0125] S401: Preprocess the label image, adjust the image size, and provide the label image. The preprocessed output is The preprocessing is represented by the following formula:

[0126]

[0127] in, The size is H , The size is .

[0128] As can be seen, the ToTensor operation converts image data into a tensor format suitable for deep learning models. The Resize operation adjusts the image size to meet the requirements of subsequent network inputs, typically resizing the image to a fixed size expected by the network for easier model processing. The Normalize operation normalizes the image data, mapping pixel values ​​to a specific range, which helps accelerate model convergence and improves model stability and generalization ability. Through these preprocessing operations, labeled images of different sizes and pixel value ranges can be converted into a uniform format and size, enabling subsequent deep learning models to effectively process this image data, improving model accuracy and efficiency.

[0129] S402: Reconstruct the forward inference process of the deep learning network ResNet18, load the pre-trained ResNet18 model weights under the PyTorch framework, and use the reconstructed ResNet18 network to extract feature maps of two different levels from the template label image.

[0130] Feature extraction is expressed by the following formula:

[0131]

[0132]

[0133] in, This represents the model weights of the first convolutional layer. Represents the model weights at different levels. Feature maps representing different levels.

[0134] In this embodiment, ResNet18 automatically learns image feature representations through a structure of multiple convolutional layers, pooling layers, and residual connections. The pre-trained model weights have already been learned on a large-scale image dataset, containing rich image feature information. During the reconstruction of the forward inference process, the input image data is processed through convolution and other operations with these model weights to progressively extract feature maps at different levels. This reduces the model's training time and data requirements while extracting representative and robust image features.

[0135] S403: Adaptive mean pooling is performed on the feature maps of layer 2 and layer 3, and then they are flattened according to the channel dimension and then stitched together to obtain the feature vector of the label image;

[0136] The eigenvector is calculated using the following formula:

[0137]

[0138]

[0139]

[0140] in, This indicates that by performing a process on the feature map along the channel dimension... Adaptive global average pooling, This indicates a feature concatenation operation. Indicates will The feature map of the size is flattened as ; Size is , Size is , Size is .

[0141] Adaptive mean pooling performs 1×1 adaptive global average pooling along the channel dimension of the feature map. It converts feature maps of different sizes into fixed-length vector representations while preserving important information along the channel dimension. Flattening transforms multi-dimensional feature maps into one-dimensional vectors, facilitating subsequent computation and processing. The concatenation operation combines feature vectors from different levels to form a comprehensive feature vector that fully describes the features of the labeled image.

[0142] As can be seen, this embodiment addresses the problem of lost detail or semantic information caused by traditional methods that typically only use features output from fully connected layers. It enhances feature representation by leveraging cross-layer fusion, utilizing both mid-level detail features and high-level semantic features. The feature map of each channel is compressed into a scalar, generating a low-dimensional dense vector while preserving channel response intensity. This reduction in feature vector dimensionality decreases the alignment computation time from 15ms to 0.5ms.

[0143] This embodiment utilizes ImageNet pre-training knowledge to adapt and optimize the pre-trained model, retaining only and recombining intermediate convolutional layers to avoid training and reduce computational load. Real-time processing is achieved on an edge server without requiring labeled drug data. Through resizing, tensor transformation, and ImageNet normalization, the input distribution is forced to align with the pre-trained model. The robustness of feature vectors to illumination and deformation is improved; the standard deviation of feature cosine similarity under different shooting conditions for the same label decreases from 0.12 to 0.03.

[0144] S5. Search the local template library based on the template index to obtain template label images and feature vector files.

[0145] In this embodiment, based on the multi-level index mapping of the Cuckoo filter, the template index is mapped to the bucket array of the Cuckoo filter through a double hash function. Each bucket stores the fingerprint of the index and the associated secondary index. When querying the template index, candidate positions in the bucket array are probed alternately. If the fingerprint matches, the associated secondary index is returned; if the probe path exceeds a threshold, the secondary retrieval process is triggered. High-frequency template indexes are preloaded into the memory cache based on the LRU strategy, and the cache key is encoded using a combination of the index hash value and a timestamp.

[0146] This embodiment also involves performing hash verification on npy files in the local template library based on the feature vector hash value in the metadata. If the verification fails, a data repair mechanism is triggered; if the verification passes, the feature vector is loaded through a memory-mapped file.

[0147] In this embodiment, a hash value is calculated synchronously and written to a log block when each feature vector file is generated; during verification, a Merkle tree is used to quickly locate the data block, ensuring that the feature vector has not been tampered with. This embodiment performs parallel loading of the original image files and feature vector files of the template label images, and asynchronous prefetching of GPU memory is achieved through CUDA streaming. OpenCL kernel functions are used to decode and normalize the image data.

[0148] When multiple versions of feature vectors corresponding to the same template index are detected, version arbitration logic based on timestamps and hash values ​​is initiated. The template creation time recorded in the locally stored version_meta.json is compared; if the timestamp difference is less than 5ms, the version with higher confidence is selected based on the cosine similarity of the feature vectors; if a conflict exists, a distributed lock mechanism is triggered to block subsequent operations until the master node completes version merging.

[0149] As can be seen, this embodiment forms a complete chain of index location, data verification, resource loading, and conflict resolution, solving the problem of false detection caused by version conflicts or data corruption in traditional solutions.

[0150] S6. Based on the SIFT key point matching algorithm, obtain the comparison label image that matches the template label image from the comparison image.

[0151] Step S6 specifically includes:

[0152] S601: For template label images And compare images SIFT detectors were applied respectively. Detect key feature points and feature descriptors The detection of key points and the calculation of feature descriptors are represented by the following formula:

[0153]

[0154]

[0155] In this embodiment, the SIFT detector is an algorithm used to detect keypoints in an image and calculate their feature descriptors. The template label image T_temp and the alignment image I_comp are input into the SIFT detector SIFTDetector(*). This detector searches for keypoints in the image that are scale- and rotation-invariant; these keypoints are typically corner points, edge points, or other points with unique features. Simultaneously, a feature descriptor is calculated for each keypoint; this descriptor is a vector describing the features of the image region surrounding the keypoint.

[0156] S602: Based on feature descriptors of the compared images and template label images, using the FLANN matcher Perform feature keypoint pairing; based on the metric distance between keypoints Filter matching point pairs;

[0157] The filtered matching point pairs are represented by the following formula:

[0158]

[0159] in, These are matching point pairs, representing the sets of key point coordinates on the template label image and the comparison image, respectively, and they exist in pairs.

[0160] The FLANN matcher in this embodiment is used to pair feature keypoints based on feature descriptors of template label images and comparison images. Matching point pairs are determined by finding nearest neighbors in the feature descriptor space. Matching point pairs are filtered based on the metric distance between keypoints, `distance_select(*)`, retaining only pairs with a distance less than a certain threshold as valid matches. The FLANN matcher can quickly find matching point pairs between feature descriptors, improving matching efficiency.

[0161] S603: Based on the matching point pairs, calculate the homography matrix from the template label image to the comparison image, and use the homography matrix to transform the coordinates of the four corner points of the template label image to obtain the coordinates of the four corner points of the label on the comparison image.

[0162] The four corner points of the labels on the comparison images are obtained using the following formula:

[0163]

[0164]

[0165] in, This represents the formula for calculating the homography matrix. This indicates the application of the homography matrix. This represents the homography matrix transformed from the template label image to the comparison image. This represents the four corner points of the template label image. This indicates comparing the four corner points on the image.

[0166] In this embodiment, the homography matrix is ​​calculated based on the coordinate information of the matching point pairs by solving a system of linear equations. It maps points on the template label image to corresponding points on the comparison image through projection transformation, taking into account relationships such as rotation, translation, scaling, and perspective transformation between images. By calculating the homography matrix, the corresponding position of the template label image on the comparison image can be accurately found, and even if the two images have certain changes in viewpoint, scaling, or rotation, they can be aligned using this matrix.

[0167] S604: Based on the coordinates of the four corner points of the label on the comparison image and the width and height of the preset label image, calculate the affine transformation matrix, and use the affine transformation matrix to transform the coordinates of the four corner points of the label on the comparison image to obtain a standard comparison label image.

[0168] The acquisition and comparison of the label image is represented by the following formula:

[0169]

[0170]

[0171] in, To compare the label images, Denotes the affine transformation matrix. This represents the four corner points of the label image obtained based on its width and height. This indicates the method for calculating the affine transformation matrix.

[0172] This embodiment compares the coordinates of the four corner points of the label on the image. and the width and height of the preset label image The `GetPerspectiveTransform(*)` function is used to calculate the affine transformation matrix `matrix_trans`. This matrix is ​​then used to transform the coordinates of the four corner points of the label on the image. The `WarpPerspective(*)` function then resamples and interpolates the image based on the affine transformation matrix, achieving the final image transformation. This makes subsequent feature extraction and consistency comparison of the label images more accurate and reliable. The uniform size and shape facilitate feature vector calculation and comparison, eliminating interference caused by differences in label position and shape, thus improving the accuracy and efficiency of the comparison.

[0173] After introducing the drug label consistency comparison method in steps S601 to S604, the true shape of the label is restored through homography matrix, and then the size and viewing angle are standardized through affine transformation. The label positioning error is reduced from ±15 pixels in the traditional method to ±3 pixels. FLANN acceleration and RANSAC denoising improve the reliability of geometric transformation. The affine transformation driven by preset width and height can dynamically adjust the output according to the preset label size, preserving the proportion of the label content. The label content deformation rate is reduced.

[0174] S7. Use the reconstructed ResNet18 network to extract the feature vectors of the comparison label images, calculate the similarity between the feature vectors of the template label images obtained in step S5 and the feature vectors of the comparison label images, and compare them with a pre-set similarity threshold to obtain the label consistency results between the template images and the comparison images.

[0175] Step S7 specifically includes:

[0176] S701: Use the reconstructed ResNet18 network to extract feature vectors from the alignment label images.

[0177] This embodiment uses the ResNet18-CPCA model, embedding a CPCA attention module in the residual block: channel features are extracted through global average pooling and max pooling, and channel weights are generated using an MLP. Spatial weights are generated by calculating the mean and maximum values ​​of the feature maps. Finally, channel and spatial attention are fused using a sigmoid function to generate a weighted feature map.

[0178] As can be seen, the standardized comparison label image T_comp is input, and multi-level features are extracted through layer 2 and layer 3:

[0179] feature2 = model_layer2(model_layer1(model_conv_7X7(T_comp)))

[0180] feature3 = model_layer3(feature2)

[0181] Feature vectors (vectors_comp, size 384×1) are generated through adaptive mean pooling (AdapAvgPool) and concatenation. An attention mechanism reduces background interference and improves sensitivity to subtle differences. Transfer learning based on pre-trained weights reduces the time and data requirements for training from scratch.

[0182] S702: Calculate the similarity between the feature vector of the template label image obtained in step S5 and the feature vector of the comparison label image, including cosine similarity. and structural similarity ;

[0183] The similarity calculation is expressed by the following formula:

[0184]

[0185]

[0186] in, This represents the formula for calculating cosine similarity. This represents the formula for calculating structural similarity. This represents the feature vector of the template label image. This represents the feature vector of the image being compared with the labeled image.

[0187] This embodiment uses cosine similarity to determine similarity based on vector direction, without considering vector length. In the feature vector space, if two feature vectors have similar directions, it indicates that the images they represent have high feature similarity. It can comprehensively measure the similarity between template label images and comparison label images from different perspectives. Cosine similarity can quickly provide a preliminary similarity judgment based on vector direction, while structural similarity takes into account the actual structural information of the images in more detail, thus compensating for the shortcomings of cosine similarity.

[0188] S703: Based on cosine similarity and structural similarity The results are compared with a preset threshold to obtain the consistency of the labeled images.

[0189] This embodiment uses a threshold to judge the similarity calculation results, which is a method to transform numerical similarity indicators into practical judgment results. The threshold acts as a standard; when the similarity reaches or exceeds this standard, the images are considered to have sufficient similarity, i.e., the labels are consistent; otherwise, differences are considered, and the labels are inconsistent. This method is simple and intuitive, and can effectively transform similarity calculation results into practical application conclusions.

[0190] This application incorporates steps S701 to S703 into the drug label consistency comparison method, where cosine similarity captures global semantic differences and structural similarity detects local statistical anomalies, forming a complementary approach. Dual-modal fusion reduces the false detection rate compared to single cosine similarity. Structural similarity is calculated on deep feature vectors, inheriting the model's robustness to geometric changes while reducing computational cost. The computation time for feature domain SSIM is reduced from 15ms in the pixel domain to 0.2ms. The collaborative design of multimodal similarity fusion, feature domain SSIM computation, and dynamic thresholding mechanism solves the problems of single-indicator limitations, computational redundancy, and insufficient flexibility in traditional methods.

[0191] S8. The label consistency results, image index, and template index are transmitted to the terminal device in form for visualization display.

[0192] This embodiment organizes the label consistency results, image index, and template index into a form, which is then sent to a terminal device (or electronic device) via the HTTP protocol. The electronic device can be a computer, mobile phone, etc. A corresponding application or webpage is developed on the electronic device to parse and display the form data, for example, visually displaying the results in the form of tables or charts. This allows users to intuitively understand the label consistency comparison results.

[0193] Based on the above embodiments, in order to further improve the reliability of the drug raw material label consistency comparison method provided in the above embodiments, the following is a specific implementable method. In this embodiment, a pharmaceutical company needs to perform consistency comparison on the labels of its "XX tablets" raw material to ensure that the new batch labels are consistent with the original template. The following is the complete implementation process:

[0194] Step S1: Collect images of the outer packaging. The quality inspector uses an industrial camera to take a full picture of the front of the new batch of medicine boxes, saves it as IMG_20250420_001.jpg, and records the batch number BATCH_202504. The image, image index IMG_20250420_001, and template index (empty value) are sent to the edge server.

[0195] Step S2: Edge server classification. If the template index is empty, it is determined to be a template image, and proceed to S3. Trigger the template processing flow and assign feature extraction.

[0196] Step S3: Template Tag Parsing. The Canny algorithm identifies the tag border and locates the coordinates (x=100, y=150). TMPL_20250420.jpg (200x200 pixels) is generated and saved to the template library. The file is named using the template index TMPL_20250420.

[0197] Step S4: Feature Vector Extraction. Using a pre-trained CSP-ResNet18, input TMPL_20250420.jpg and output a 128-dimensional feature vector. Save it as TMPL_20250420.npy, with a hash value SHA-256=abc123.

[0198] New batch label check: S1: Collect the new image IMG_20250427_002.jpg, the form contains template index TMPL_20250420. S2: Check if the template index is not empty, proceed to S5. S5: Search the template library, load TMPL_20250420.jpg and .npy files. S6: SIFT match and compare the label regions in the image, crop out COMPARE_20250427.jpg. S7: Extract the comparison feature vector, calculate the cosine similarity of 0.92 (>threshold 0.9), and determine consistency. S8: The result is returned to the terminal, displaying "Consistent" and overlaying the labeled difference regions.

[0199] Example of anomaly handling: Suppose a batch of labels has a font color offset: S7: Similarity is 0.88, below the threshold. S8: The result is marked "Inconsistent", triggering manual review, and a label printing deviation is found, so the production line is suspended for adjustment.

[0200] This embodiment achieves high efficiency and accuracy in label consistency comparison through standardized data collection, intelligent classification, deep feature extraction, and visual feedback. The entire process, from template creation to result feedback, is automated, while also meeting the requirements of drug regulatory authorities for data traceability and accuracy.

[0201] The following are embodiments of the pharmaceutical raw material label consistency comparison system provided in this disclosure. This system and the pharmaceutical raw material label consistency comparison method in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the pharmaceutical raw material label consistency comparison system, please refer to the embodiments of the pharmaceutical raw material label consistency comparison method described above.

[0202] like Figure 2 As shown, the system includes: an image acquisition and storage module, used to acquire and save images of the outer packaging, and send the outer packaging images, image indexes, and template indexes to the edge server in the form of a form.

[0203] The edge server is used to determine whether the current outer packaging image is a template image or a comparison image based on whether the template index in the form is empty. If the template index is empty, the current outer packaging image is a template image, and the image parsing and saving module is executed; otherwise, it is a comparison image, and the feature lookup module is executed.

[0204] The image parsing and saving module is used to parse template images into standard template tag images based on a preset algorithm, and save the template tag images in a preset format in the local template library by naming them with the template index.

[0205] The feature extraction module is used to extract feature vectors from template label images using the reconstructed ResNet18 network, and saves the feature vectors of template label images in npy format files in the local template library.

[0206] The feature search module is used to search the local template library based on the template index and obtain template label images and feature vector files.

[0207] The image matching module is used to obtain the matching label image that matches the template label image from the comparison image based on the SIFT key point matching algorithm.

[0208] The image processing module is used to extract feature vectors of the comparison label images using the reconstructed ResNet18 network, calculate the similarity between the obtained template label image feature vector and the comparison label image feature vector, and compare it with a pre-set similarity threshold to obtain the label consistency result between the template image and the comparison image.

[0209] The image display module is used to visualize the label consistency results, image index, and template index by transmitting them to the terminal device in form.

[0210] like Figure 3 As shown, this application also provides an electronic device, including a display module 103, a memory 102, a processor 101, and a computer program stored in the memory and executable on the processor 101. When the processor 101 executes the program, it implements the steps of the method for parsing and loading a GIM model of a power transmission project.

[0211] In embodiments of the present invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments described and / or claimed herein.

[0212] In this embodiment, processor 101 may be implemented using at least one of an Application-Specific Integrated Circuit (ASIC), a Programmable Logic Device (PLD), a Field-Programmable Gate Array (FPGA), a processor, a controller, a microcontroller, a microprocessor, or an electronic unit designed to perform the functions described herein. In some cases, such implementations may be implemented within a controller. For software implementations, implementations such as processes or functions may be implemented with separate software modules that allow the performance of at least one function or operation. The software code may be implemented by a software application (or program) written in any suitable programming language, and the software code may be stored in memory and executed by the controller.

[0213] The display module 103 is used to display information input by the user or information provided to the user. The display module 103 may include a display panel, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0214] The memory 102 can be used to store software programs and various data. The memory 102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0215] This application also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the pharmaceutical raw material label consistency comparison method.

[0216] The storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0217] In a storage medium, a readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0218] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for comparing the consistency of pharmaceutical raw material labels, characterized in that the method... include: S1. Collect and save the outer packaging image, and send the outer packaging image, image index, and template index to the edge server in the form of a form; S2. The edge server determines whether the current outer packaging image is a template image or a comparison image based on whether the template index in the form is empty. If the template index is empty, the current outer packaging image is a template image and proceeds to S3; otherwise, it is a comparison image and proceeds to S5. S3. Parse the template image using a preset algorithm to extract standard template tag images, and save the template tag images in the local template library in a preset format by naming them with the template index. Step S3 specifically includes: S301: Apply mean filtering, bilateral filtering and Gaussian filtering to the template image to smooth the template image, and convert the smoothed image into a grayscale template image; S302: For grayscale template images, edge detection algorithms based on the Scharr operator are used to process them to obtain binarized template images; S303: Based on the binarized template image, a contour detection algorithm is applied to filter out quadrilateral candidate boxes that meet the conditions, and the quadrilateral with the largest area is selected as the coordinate box of the template label image. S304: Combine the coordinates of the four corner points of the template label image with the width and height of the preset label image, calculate the radial transformation matrix, and apply the radial transformation algorithm to obtain a standard template label image on the template image. S4. Use the reconstructed ResNet18 network to extract the feature vectors of the template label images, and save the feature vectors of the template label images in the local template library as npy format files; The feature vectors of labeled images are extracted using a reconstructed ResNet18 network, including the following steps: S401: Preprocess the label image, adjust the image size, and provide the label image. The preprocessed output is The preprocessing is represented by the following formula: in, The size is H , The size is ; S402: Reconstruct the forward inference process of the deep learning network ResNet18, load the pre-trained ResNet18 model weights under the PyTorch framework, and use the reconstructed ResNet18 network to extract feature maps of two different levels from the template label image. Feature extraction is expressed by the following formula: in, This represents the model weights of the first convolutional layer. Represents the model weights at different levels. Feature maps representing different levels; S403: Adaptive mean pooling is performed on the feature maps of layer 2 and layer 3, and then they are flattened according to the channel dimension and then stitched together to obtain the feature vector of the label image; The eigenvector is calculated using the following formula: in, This indicates that by performing a process on the feature map along the channel dimension... Adaptive global average pooling, This indicates a feature concatenation operation. Indicates will The feature map of the size is flattened as ; Size is , Size is , Size is ; S5. Search the local template library based on the template index to obtain template label images and feature vector files; S6. Based on the SIFT key point matching algorithm, obtain the comparison label image that matches the template label image from the comparison image; S7. Use the reconstructed ResNet18 network to extract the feature vectors of the comparison label images, calculate the similarity between the feature vectors of the template label images obtained in step S5 and the feature vectors of the comparison label images, and compare them with the pre-set similarity threshold to obtain the label consistency results between the template images and the comparison images. S8. The label consistency results, image index, and template index are transmitted to the terminal device in form for visualization display.

2. The method for comparing the consistency of pharmaceutical raw material labels according to claim 1, characterized in that, Step S2 specifically includes: Step S201: Receive and parse the uploaded form data, extract the outer packaging image, image index and template index fields, and verify the data format and integrity; Step S202: Check if the template index field is empty. If it is empty, it is determined to be a template image and the template registration process is triggered. If it is not empty, it is determined to be a comparison image and the tag comparison process is triggered to achieve automated process diversion. Step S203: Dynamically allocate processing paths based on the judgment results: Generate a unique template index for the template image and bind data, and compare the image to retrieve associated template data through the index; Step S204: Generate a globally unique index for the template image, bind and store the index with the image data and feature files, and establish a multi-level association relationship for the index data.

3. The method for comparing the consistency of pharmaceutical raw material labels according to claim 1, characterized in that, Step S5 specifically includes: The template index is mapped to a bucket array of Cuckoo filters using a double hash function, with each bucket storing the fingerprint of the index and the associated secondary index; When querying the template index, candidate positions in the bucket array are probed alternately. If the fingerprint matches, the associated secondary index is returned. If the probe path exceeds the threshold, the secondary retrieval process is triggered. High-frequency template indexes are preloaded into the memory cache based on the LRU strategy. The cache key is encoded using a combination of index hash value and timestamp. In step 5, the hash verification of the npy file in the local template library is also performed based on the feature vector hash value in the metadata. If the verification fails, the data repair mechanism is triggered. If the verification passes, the feature vector is loaded through the memory-mapped file. The loading method is based on parallel loading, loading the original image file and feature vector file of the label image, implementing asynchronous prefetching through CUDA stream, and using OpenCL kernel functions to decode and normalize the image data. When multiple versions of feature vectors corresponding to the same template index are detected, a process is initiated that compares the template creation time stored locally based on timestamps and hash values. If the timestamp difference is less than the preset time, the version with higher confidence is selected based on the cosine similarity of the feature vectors.

4. The method for comparing the consistency of pharmaceutical raw material labels according to claim 1, characterized in that, Step S6 specifically includes: S601: For template label images And compare images SIFT detectors were applied respectively. Detect key feature points and feature descriptors The detection of key points and the calculation of feature descriptors are represented by the following formula: S602: Based on feature descriptors of the compared images and template label images, using the FLANN matcher Perform feature keypoint pairing; based on the metric distance between keypoints Filter matching point pairs; The selection of matching point pairs is expressed by the following formula: in, These are matching point pairs, representing the sets of key point coordinates on the template label image and the comparison image, respectively, and they exist in pairs; S603: Based on the matching point pairs, calculate the homography matrix from the template label image to the comparison image, and use the homography matrix to transform the coordinates of the four corner points of the template label image to obtain the coordinates of the four corner points of the label on the comparison image. The four corner points of the labels on the comparison images are obtained using the following formula: in, This represents the formula for calculating the homography matrix. This indicates the application of the homography matrix. This represents the homography matrix transformed from the template label image to the comparison image. This represents the four corner points of the template label image. This indicates comparing the four corner points on the image; S604: Based on the coordinates of the four corner points of the label on the comparison image and the width and height of the preset label image, calculate the affine transformation matrix, and use the affine transformation matrix to transform the coordinates of the four corner points of the label on the comparison image to obtain a standard comparison label image. The following formula represents the method for obtaining the comparison label image: in, To compare the label images, Denotes the affine transformation matrix. This represents the four corner points of the label image obtained based on its width and height. This indicates the method for calculating the affine transformation matrix.

5. The method for comparing the consistency of pharmaceutical raw material labels according to claim 1, characterized in that, Step S7 specifically includes: S701: Use the reconstructed ResNet18 network to extract feature vectors from the alignment label images; S702: Calculate the similarity between the feature vector of the template label image obtained in step S5 and the feature vector of the comparison label image, including cosine similarity. and structural similarity ; The similarity is calculated using the following formula: in, This represents the formula for calculating cosine similarity. This represents the formula for calculating structural similarity. This represents the feature vector of the template label image. Represents the feature vector of the compared labeled image; S703: Based on cosine similarity and structural similarity The results are compared with a preset threshold to obtain the consistency of the labeled images.

6. A pharmaceutical raw material label consistency comparison system, characterized in that, The system is used to implement the pharmaceutical raw material label consistency comparison method as described in any one of claims 1 to 5; The system includes: The image acquisition and storage module is used to acquire and save images of the outer packaging, and send the outer packaging images, image indexes, and template indexes to the edge server in the form of a form. The edge server is used to determine whether the current outer packaging image is a template image or a comparison image based on whether the template index in the form is empty. If the template index is empty, the current outer packaging image is a template image, and the image parsing and saving module is executed; otherwise, it is a comparison image, and the feature lookup module is executed. The image parsing and saving module is used to parse the template image into a standard template tag image based on a preset algorithm, and save the template tag image in a preset format in the local template library by naming it with the template index; Among them, the template image is smoothed by applying mean filtering, bilateral filtering and Gaussian filtering, and the smoothed image is converted into a grayscale template image; For grayscale template images, edge detection algorithms based on the Scharr operator are used to process them to obtain binarized template images; Based on the binarized template image, a contour detection algorithm is applied to filter out quadrilateral candidate boxes that meet the conditions, and the quadrilateral with the largest area is selected as the coordinate box of the template label image. By combining the coordinates of the four corner points of the template label image with the width and height of the preset label image, the radiative transformation matrix is ​​calculated, and the radiative transformation algorithm is applied to the template image to obtain a standard template label image. The feature extraction module is used to extract feature vectors of template label images using the reconstructed ResNet18 network, and save the feature vectors of template label images in npy format files in the local template library; The feature vectors of labeled images are extracted using a reconstructed ResNet18 network, including the following steps: Preprocess the label image, resize the image, and provide the label image. The preprocessed output is The preprocessing is represented by the following formula: in, The size is H , The size is ; The forward inference process of the deep learning network ResNet18 is reconstructed, the model weights of the pre-trained ResNet18 under the PyTorch framework are loaded, and the reconstructed ResNet18 network is used to extract feature maps of two different levels from the template label image. Feature extraction is expressed by the following formula: in, This represents the model weights of the first convolutional layer. Represents the model weights at different levels. Feature maps representing different levels; Adaptive mean pooling is performed on the feature maps of layer 2 and layer 3, and then they are flattened according to the channel dimension and then stitched together to obtain the feature vector of the label image. The eigenvector is calculated using the following formula: in, This indicates that by performing a process on the feature map along the channel dimension... Adaptive global average pooling, This indicates a feature concatenation operation. Indicates will The feature map of the size is flattened as ; Size is , Size is , Size is ; The feature lookup module is used to search the local template library based on the template index and obtain template tag images and feature vector files; The image matching module is used to obtain the matching label image that matches the template label image from the comparison image based on the SIFT key point matching algorithm; The image processing module is used to extract feature vectors of the comparison label images using the reconstructed ResNet18 network, calculate the similarity between the obtained template label image feature vector and the comparison label image feature vector, and compare it with a pre-set similarity threshold to obtain the label consistency result between the template image and the comparison image. The image display module is used to visualize the label consistency results, image index, and template index by transmitting them to the terminal device in form.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the pharmaceutical raw material label consistency comparison method as described in any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the pharmaceutical raw material label consistency comparison method as described in any one of claims 1 to 5.

Images