Drug traceability code scanning identification method and system, and storage medium

Abstract

The invention relates to the technical field of image data processing, in particular to a medicine traceability code scanning identification method and system and a storage medium. The method comprises the following steps: scanning a traceability code area of a drug to be identified, and obtaining a multi-angle image set; detecting an abnormal pixel point according to the multi-view image set, and dividing the multi-view image set into an abnormal view image and a normal view image according to an abnormal pixel point detection result; splicing and restoring abnormal pixel points in the abnormal view angle image by utilizing traceability code composition characteristics of the normal view angle image, and obtaining a to-be-analyzed image set; detecting code system types in the to-be-analyzed image set, and classifying and extracting character data corresponding to each code system type in parallel according to the number of the code system types; and comparing the character data corresponding to each code system type, and outputting medicine circulation information according to a comparison result. According to the invention, the problem of identification failure caused by light reflection, creases or stains on the surface of the package is effectively solved, so that efficient, reliable and comprehensive monitoring of the drug tracing process is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and particularly relates to a medicine traceability code scanning and identifying method and system and a storage medium. BACKGROUND

[0002] In order to ensure the transparency and traceability of medicines in the production, transportation, storage and sales, etc., the relevant departments have successively introduced a series of electronic supervision measures for medicines, and require that medicines be printed with unique traceability codes (such as one-dimensional bar codes, two-dimensional bar codes, etc.) on the packaging to realize the traceability of medicines from the production source to the final use terminal. The existing medicine traceability codes are mainly identified by optical scanning equipment or mobile terminals, and after identification, the medicine traceability database is compared to verify the authenticity and circulation information of the medicines.

[0003] However, in actual application, due to the existence of reflection, creases and dirt on the surface of the medicine packaging, the traditional scanning and identifying method is prone to reading failure, long identification time or low identification accuracy. SUMMARY

[0004] Therefore, it is necessary to provide a medicine traceability code scanning and identifying method, system and storage medium to solve at least one of the above technical problems.

[0005] To achieve the above purpose, a medicine traceability code scanning and identifying method comprises the following steps:

[0006] Step S1: The scanning device is pre-deployed to scan the traceability code area of the medicine to be identified from multiple angles to obtain a multi-angle image set;

[0007] Step S2: According to the multi-view image set, the abnormal pixel points are detected, and the multi-view image set is divided into abnormal view images and normal view images according to the abnormal pixel point detection results;

[0008] Step S3: The traceability code composition features of the normal view images are used to splice and restore the abnormal pixel points in the abnormal view images to obtain a set of images to be analyzed;

[0009] Step S4: The code type in the set of images to be analyzed is detected, and the character data corresponding to each code type is classified and extracted in parallel according to the number of code types;

[0010] Step S5: The character data corresponding to each code type is compared with the preset medicine traceability code database, and the circulation information of the medicine is output according to the comparison result.

[0011] The application obtains a multi-view image set by deploying a multi-angle scanning device on a medicine package, and combines an abnormal pixel point detection technology to automatically identify a reflection, blur and contamination area, restores the abnormal pixels in the abnormal view image based on the normal view image to generate a to-be-analyzed image set; different code type in the to-be-analyzed image set is classified and distributed to a preset decoding channel to extract corresponding character data in parallel, and finally compared with a medicine traceability code database to determine the medicine authenticity and output circulation information. Compared with the prior art, the application can effectively solve the recognition failure problem caused by the reflection, crease or contamination of the package surface, improve the recognition accuracy and processing efficiency through multi-view image restoration and parallel decoding, so as to realize efficient, reliable and comprehensive monitoring of the medicine traceability process.

[0012] Optionally, the application also provides a medicine traceability code scanning and identification system for executing the medicine traceability code scanning and identification method as described above, which comprises:

[0013] A data acquisition module is configured to acquire a multi-angle image set by multi-angle scanning the traceability code area of the to-be-identified medicine through a pre-deployed scanning device;

[0014] An abnormal pixel point detection module is configured to detect abnormal pixel points according to the multi-view image set, and divide the multi-view image set into abnormal view images and normal view images according to the abnormal pixel point detection result;

[0015] An abnormal restoration module is configured to splice and restore the abnormal pixel points in the abnormal view image by using the traceability code composition features of the normal view image to obtain a to-be-analyzed image set;

[0016] A character extraction module is configured to detect the code type in the to-be-analyzed image set, and classify and extract the character data corresponding to each code type in parallel according to the number of code types;

[0017] A medicine identification module is configured to compare the character data corresponding to each code type by using a preset medicine traceability code database, and output the medicine circulation information according to the comparison result.

[0018] The medicine traceability code scanning and identification system of the application can implement any one of the medicine traceability code scanning and identification methods of the application, and is used as a medium for joint operation and signal transmission between modules to complete the medicine traceability code scanning and identification method, so that the modules in the system cooperate with each other, thereby effectively solving the recognition failure problem caused by the reflection, crease or contamination of the package surface, improving the recognition accuracy and processing efficiency through multi-view image restoration and parallel decoding, so as to realize efficient, reliable and comprehensive monitoring of the medicine traceability process.

[0019] Optionally, the present application also provides a computer readable storage medium, wherein a computer program is stored, and the computer program is executed to realize the medicine trace code scanning and identifying method. BRIEF DESCRIPTION OF DRAWINGS

[0020] Other features, objects, and advantages of the application will become more apparent from the following detailed description when read in connection with the following drawings:

[0021] Fig. 1 A schematic diagram of the step flow of the medicine trace code scanning and identifying method of the present application;

[0022] Fig. 2 A schematic diagram of the medicine trace code in the embodiment of the present application;

[0023] Fig. 3 A schematic diagram of the module of the medicine trace code scanning and identifying system in the present application;

[0024] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0025] The technical method of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0026] In addition, the accompanying drawings are only schematic illustrations of the present application, and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus repeated description thereof will be omitted. Some block diagrams shown in the drawings are functional entities, and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0027] It should be understood that although the terms "first", "second" and the like can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the example embodiments, a first element can be referred to as a second element, and similarly a second element can be referred to as a first element. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0028] To achieve the above-mentioned purposes, please refer toFigs. 1 to 3 The application provides a medicine traceability code scanning and recognizing method, which comprises the following steps:

[0029] Step S1: scanning the traceability code area of the medicine to be recognized by a pre-deployed scanning device at multiple angles to obtain a multi-angle image set;

[0030] In an embodiment, an image acquisition module with a multi-angle acquisition function is pre-deployed in the scanning device, for example, a multi-camera array structure containing at least three angles (0°, 15° and 30°) is adopted, wherein each camera is configured with an industrial camera lens with a resolution of not less than 1280*720. The image acquisition module is used to scan the traceability code area of the medicine to be recognized at multiple angles, and the multi-angle images of the traceability code area are synchronously acquired at an acquisition rate of not less than 15 frames per second, so as to obtain a multi-angle image set covering the traceability code area. The selection of the above-mentioned camera array structure is mainly based on the common reflection and inclination characteristics of the surface of the medicine package to ensure that complete information containing the traceability code can be acquired at different angles.

[0031] It is noted that the determination method of the traceability code area comprises: performing full-width acquisition on the surface of the medicine outer package by the pre-deployed scanning device. The scanning device in the embodiment is an industrial camera with a resolution of 1920*1080, and the acquired image is subjected to grayscale and bilateral filtering to reduce the interference of the package pattern and the background. On this basis, edge detection operation is performed on the pre-processed image, a Canny operator is adopted, and the high threshold value is set to 0.2 and the low threshold value is set to 0.05 to obtain an image edge candidate set. Then, the edge set is subjected to connected region analysis by combining morphological closing operation, and a rectangular feature region is extracted. Further, the aspect ratio and the area gray distribution uniformity in the candidate rectangular region are calculated. It is determined by experiments that the aspect ratio of the traceability code ranges from 1.5 to 2.2, and the area gray distribution standard deviation is lower than a set threshold value (for example, 15); and the candidate rectangular region meeting the conditions is marked as the medicine traceability code area.

[0032] Step S2: detecting abnormal pixel points according to the multi-view image set, and dividing the multi-view image set into abnormal view images and normal view images according to the abnormal pixel point detection result;

[0033] In an embodiment, each image is divided into 32x32 pixel detection windows, and the gray mean value, gray standard deviation, edge gradient mean value, frequency energy distribution, texture direction dispersion and local contrast of the pixels in the detection window are calculated respectively. If the detection window gray mean value is higher than 30% of the full image mean value, and the standard deviation is lower than 15, it is determined as a reflective area; if the edge gradient mean value is lower than 40% of the full image mean value, and the high frequency energy proportion is lower than 20%, it is determined as a blur area; if the texture direction dispersion is higher than 0.8, and the local contrast is lower than 50% of the full image mean value, it is determined as a contamination area. Further, the pixel proportion of each type of area in the whole image is counted, and if the reflective area proportion exceeds 10%, the blur area proportion exceeds 8%, and the contamination area proportion exceeds 6%, the corresponding pixel points are marked as abnormal pixel points. These thresholds come from the statistical results of 5000 pieces of medicine packaging images.

[0034] Especially important is that dividing the multi-view image set into abnormal view images and normal view images comprises:

[0035] According to the proportion of reflective abnormal pixel points, blur abnormal pixel points and contamination abnormal pixel points in the total image pixel points, the image with a high proportion of abnormal pixel points is divided into an abnormal view image, and the image with a low proportion of abnormal pixel points is divided into a normal view image.

[0036] In an embodiment, the proportion of the detected reflective abnormal pixel points, blur abnormal pixel points and contamination abnormal pixel points in the whole image is calculated. Assuming that the image resolution is 1280x720, the total number of pixel points is about 920,000. By counting the number of pixel points of each type of abnormal pixel points in the whole image and performing ratio operation with the total number of pixel points, the proportion value of each type of abnormal pixel points is obtained. The abnormal pixel point proportion threshold is set to 5%, which is derived from the experimental statistical results of a large number of medicine packaging images (about 3000 samples). It is found that when the proportion of abnormal areas exceeds this value, the image quality decreases significantly, resulting in a decoding success rate of less than 85%. Therefore, the image with an abnormal pixel point proportion exceeding the threshold is determined as an abnormal view image, and the image with an abnormal pixel point proportion lower than the threshold is determined as a normal view image. If the proportion of reflective abnormal pixel points in an image is 12%, the proportion of blur abnormal pixel points is 4%, and the proportion of contamination abnormal pixel points is 3%, the total proportion of abnormal pixel points is 19%, which is greater than 5%, and the image is divided into an abnormal view image; if the proportion of abnormal pixel points in another image is only 2.3%, the image is divided into a normal view image.

[0037] Step S3: using the trace code composition features of the normal view image, splicing and restoring the abnormal pixel points in the abnormal view image to obtain a set of images to be analyzed;

[0038] In an embodiment, the symbol position of the abnormal pixel point is determined, and the corresponding position in the normal view image is found. If the abnormal pixel point is a reflection abnormal pixel point, the gray value of the corresponding position in the normal view image is used for gray value interpolation. If the abnormal pixel point is a blur abnormal pixel point, the gradient information of the corresponding position is used for constraint reconstruction, and the missing high frequency information is compensated in the frequency domain. If the abnormal pixel point is a pollution abnormal pixel point, the local contrast of the corresponding position is used for enhancement processing. The above restoration results are combined with the normal view image, and the texture features are used for repair in the edge region, so as to generate the image set to be analyzed.

[0039] Step S4: detecting the code type in the image set to be analyzed, and classifying and extracting the character data corresponding to each code type in parallel according to the number of code types;

[0040] In another embodiment, the code detection is performed on the image set to be analyzed. The detection result shows that images of different code types are divided into corresponding subsets, and independent decoding channels are allocated according to the number of code types. Each channel simultaneously performs decoding operation to extract the character data of the corresponding code type.

[0041] Step S5: comparing the character data corresponding to each code type with the preset drug traceability code database, and outputting the drug circulation information according to the comparison result.

[0042] In an embodiment, the extracted character data is compared with the preset drug traceability code database. The database stores no less than 100,000 traceability code records, and is indexed by code type. If the comparison is successful, the drug is determined to be genuine, and the production batch, circulation link and other information of the drug are output. If the comparison fails, the drug is determined to be a suspected fake drug, and the scanning device is controlled to reacquire the traceability code area of the drug, and the above detection and comparison process is repeated. In an optional case, if only part of the codes do not match, the drug is determined to be a suspected abnormal drug, and the image area corresponding to the code is reacquired to shorten the reinspection time.

[0043] Optionally, the drug circulation information is output according to the comparison result in step S5, including:

[0044] If the character data corresponding to each code type in the comparison result has a matching relationship with the drug traceability code in the drug traceability code database, the drug to be identified is determined to be a genuine drug, and the drug circulation information corresponding to the genuine drug is output.

[0045] In an embodiment, the extracted character data of each code type is first sent to a preset drug traceability code database for comparison. The drug traceability code database is pre-established by the supervision platform, and the drug traceability codes in the database are indexed according to code types for quick retrieval. During comparison, a quick retrieval algorithm based on hash mapping is used to generate a corresponding hash value for the input character data, and a matching query is performed in the index table of the same code type. During the comparison process, the matching threshold is set to 100% character consistency, that is, when the input character data is completely consistent with any traceability code in the database in terms of character length and character sequence, it is determined that the matching is successful. If the comparison is successful, the drug circulation information associated with the traceability code is called from the database, including the drug batch number, the production enterprise, the circulation link node, and the expiration date, and the information is directly output to the result interface.

[0046] If the character data corresponding to each code type in the comparison result does not have a matching relationship with the drug traceability codes in the drug traceability code database, the to-be-identified drug is determined to be a suspected counterfeit drug, and the scanning device is controlled to scan the traceability code area of the suspected counterfeit drug, and steps S2 to S5 are re-executed.

[0047] In an embodiment, if the comparison is not successful, it means that the current input character data is not registered in the database. At this time, the system marks the drug as a suspected counterfeit drug and issues an instruction to the pre-deployed scanning device. After receiving the instruction, the scanning device automatically adjusts the acquisition parameters, reduces the exposure time from the original 20 ms to 12 ms, and increases the acquisition angle range to ± 30° to enhance the coverage of the possible reflection or shielding area. The re-acquired image enters the abnormal pixel point detection and splicing restoration process again, so that the complete steps of abnormal detection, image restoration, code identification and database comparison are re-executed until the updated identification result is output.

[0048] Optionally, the method of identifying a genuine drug further comprises:

[0049] If the character data corresponding to each code type has a matching relationship with the drug traceability codes of the corresponding code type in the drug traceability code database, the to-be-identified drug is determined to be a genuine drug;

[0050] In one embodiment, after performing parallel decoding on the image set to be analyzed, character data corresponding to multiple code types is obtained. First, an independent matching query is established in the drug traceability code database for each code type. The database is partitioned according to code types, and a B+ tree index structure is used within the partition to ensure retrieval efficiency. The matching determination can use a character-by-character comparison algorithm, and a threshold of 100% character consistency is set, that is, the length of the input character data and the character sequence must be completely consistent to be determined as a match. When the character data of all code types is found in the database corresponding to the entry, the drug is automatically marked as a genuine drug, and the drug information corresponding to each code type, including batch number, manufacturer and expiration date, is called from the database as circulation information output.

[0051] If the character data corresponding to each code type and the drug traceability code corresponding to the code type in the drug traceability code database have any missing matching relationship, the drug to be identified is determined as a suspected abnormal drug;

[0052] The scanning device is controlled to scan the traceability code region corresponding to the code type with a missing matching relationship in the suspected abnormal drug, and steps S2 to S5 are re-executed.

[0053] In another embodiment, if only part of the code type character data matches successfully, and any code type does not match successfully, the drug is determined to be a suspected abnormal drug. At this time, an instruction is issued to the scanning device to perform a second scan on the region corresponding to the code type with a missing matching relationship. In this embodiment, in order to improve the recognition probability of the missing code elements, the scanning device automatically increases the acquisition resolution from the original 300 dpi to 600 dpi, and adjusts the light source brightness from 200 lux to 350 lux to enhance the imaging quality of low-contrast regions. The reacquired image will first enter the abnormal pixel point detection process to ensure that the interference caused by reflection, blur or contamination in the image is repaired. After repair, the image enters the decoding channel to extract the character data corresponding to the code type, and then continues to perform matching comparison in the database. Finally, if the missing code type is completed and matching is completed, it is corrected as a genuine drug; if there is still a missing or inconsistent situation, it remains in an abnormal state and outputs the corresponding prompt.

[0054] Optionally, the abnormal pixel points in step S2 include reflection abnormal pixel points, blur abnormal pixel points and contamination abnormal pixel points; wherein detecting the reflection abnormal pixel points comprises:

[0055] Divide each image in the multi-view image set into a plurality of detection windows, and calculate the gray mean value and standard deviation of the pixel points in each detection window one by one;

[0056] In an embodiment, when performing the division operation on each image in the multi-view image set, a fixed-size detection window can be used, and the size of the detection window is set to 32x32 pixels. The source of this size is that the minimum unit width of a common drug traceability code module (code element) is generally about 1 mm, and in a scanned image with a resolution of 300 dpi, 1 mm corresponds to about 12 pixels, so a 32x32 pixel window can cover 2-3 code element regions, which can avoid the influence of local noise and maintain the sensitivity of detection.

[0057] In another embodiment, the image region can be divided into several detection windows according to the similarity of each pixel point in the image region.

[0058] In particular, the division of several detection windows is specifically;

[0059] The multi-view image set is divided into several fixed windows according to a preset division ratio;

[0060] In a specific embodiment, when the multi-view image set is divided into fixed windows, the division ratio is preset according to the resolution of the image. For example, in an image with a resolution of 1280x1280 pixels, it is divided into 40x40 fixed windows, and the size of each window is 32x32 pixels. The selection of the window size is based on the industry standard that the minimum code element width of the drug traceability code is between 0.8 mm and 1.2 mm, and under the condition of a resolution of 300 dpi, 1 mm≈12 pixels, so 32x32 pixels can cover 2-3 code element regions, which can ensure fine-grained detection and avoid over-segmentation.

[0061] Calculate the feature similarity between the pixel points in the fixed window and the feature similarity between the pixel points with the smallest spatial distance in the adjacent fixed window;

[0062] According to the feature similarity between the pixel points in the fixed window, the fixed window is divided into several high-similarity regions, and the high-similarity regions are spliced according to the feature similarity between the pixel points with the smallest spatial distance in the adjacent fixed window, to divide into several detection windows.

[0063] In this embodiment, the region is divided by calculating the feature similarity between pixels within the fixed window. The similarity is measured by the weighted combination of the gray cosine similarity and the HOG cosine similarity, with the weight ratio of 0.6:0.4. The ratio is derived from the experimental comparison results, which shows that the gray feature can better reflect the local brightness consistency, while the gradient feature can avoid misjudgment caused by changes in lighting conditions, so the weighted fusion method is adopted. In the specific calculation, the similarity of the pixel gray value vector and the gradient direction histogram is calculated respectively, and then the sum is calculated according to the weight. If the similarity is greater than 0.85, the pixel point is determined to belong to the same high-similarity region.

[0064] In another embodiment, between adjacent fixed windows, the edge pixel point with the smallest spatial distance is detected, and the feature similarity is calculated. The spatial distance uses the Euclidean distance, and the feature similarity still uses the weighted combination of the above-mentioned gray and gradient. When the similarity of the edge pixel points of the adjacent windows is greater than 0.8, the region with higher similarity in the two windows is spliced into a continuous region. The threshold is obtained by experiment, mainly considering the balance between the integrity of the spliced region and the mis-splicing rate.

[0065] If the average gray value of any detection window is higher than 30% of the average gray value of the whole image, and the standard deviation is lower than the preset standard deviation threshold, it is determined that the pixels in the detection window are high-brightness aggregation pixels.

[0066] In this embodiment, the average gray value and the standard deviation of the pixels are calculated one by one. The gray value uses a linear range of 0-255, and the average value μ and the standard deviation σ are obtained by first performing histogram statistics on the pixels in the window. In order to avoid the influence of single image brightness offset, the average gray value μ of the whole image is introduced when calculating the threshold. all where μ all is the average value of the gray value of all pixels in the image. If the μ of a certain detection window is greater than 1.3 times of μ all , it indicates that there is a significant increase in brightness in this region.

[0067] In the standard deviation calculation, the threshold of σ is set to 12. This value is derived from the statistical experiment of a large number of normal drug traceability code images, which shows that the gray standard deviation of the code region is greater than 15 under normal circumstances, and when the region is homogenized due to reflection, the standard deviation is significantly reduced. Therefore, the case of σ≤12 can be considered as the pixel gray value is too concentrated, which constitutes a high-brightness aggregation region.

[0068] The detection windows after the gray mean value judgment and the standard deviation judgment are subjected to intersection operation, and the pixel points in the detection window which are recognized as the high-light aggregation region by both the two kinds of judgment are taken as the high-light aggregation pixels. Because the detection window is composed of similar pixel points, if any region in the detection window is a high-light aggregation region, the pixels in the detection window are all high-light aggregation pixels.

[0069] The proportion of the high-light aggregation pixels in each image is calculated, and if the proportion of the high-light aggregation pixels is higher than 10% of the total pixel number of each image, the high-light aggregation pixels are judged as the abnormal reflection pixels.

[0070] In an embodiment, the number of high-light aggregation pixels is divided by the total pixel number of the image. For example, for an image with a resolution of 1024x1024, the total pixel number is 1,048,576, and when the number of high-light pixels exceeds 104,857, the proportion exceeds 10%. If the condition is met, the image is judged to contain abnormal reflection pixels.

[0071] Optionally, detecting the blur abnormal pixels comprises:

[0072] Edge gradient calculation and frequency energy distribution statistics are performed on the pixel points in each detection window.

[0073] In an embodiment, when the edge gradient calculation is performed on the pixel points in each detection window, Sobel operators are used to perform convolution operation on the gray difference in the horizontal direction and the vertical direction. The gradient value of each detection window is calculated by the formula: , wherein G x and G y are the gradient components in the horizontal direction and the vertical direction, respectively. In order to weaken the local noise interference, 3x3 Gaussian smoothing filter is performed on the detection window in the calculation process, and the smoothing coefficient σ=1.0. This parameter is derived from the conventional image processing experiment, which can remove high-frequency noise points while preserving the main edges. Subsequently, the gradient mean value of all pixels in the window is taken as the edge intensity value of the window.

[0074] In another embodiment, the pixel matrix of the detection window is converted from the spatial domain to the frequency domain by using fast Fourier transform (FFT) to obtain the amplitude spectrum. In order to distinguish the low-frequency and high-frequency components, a cut-off frequency threshold f c is preset, which is 30% of the frequency bandwidth of the window, i.e. if the window size is 32x32 pixels, the bandwidth is 16, and the cut-off frequency is set to 5. The proportion of the frequency spectrum energy in the window which is higher than f c is defined as the high-frequency component proportion. This proportion can directly reflect the texture and detail fidelity of the image. According to the experimental statistical results, when the high-frequency component proportion is lower than 15%, it often corresponds to the blur region which can be perceived by the naked eye, so 15% is set as the preset high-frequency component proportion threshold.

[0075] If the edge gradient mean value of any detection window is lower than 40% of the edge gradient mean value of the whole image, and the high frequency component proportion in the frequency energy distribution of the detection window is lower than the preset high frequency component proportion threshold, it is determined that the pixels in the detection window are low gradient blur pixels.

[0076] In this embodiment, the global gradient mean value of the whole image is first calculated. If the edge gradient mean value of any detection window is lower than 40% of the global mean value, and the high frequency component proportion is lower than 15%, the pixels in the detection window are marked as low gradient blur pixels. The 40% threshold is derived from a plurality of groups of medicine traceability code scanning experiments. When the local gradient intensity is lower than 40% of the global gradient intensity, the local decoding error rate significantly increases, so it is used as a reasonable boundary.

[0077] The proportion of low gradient blur pixels in each image is calculated. If the proportion of low gradient blur pixels is higher than 8% of the total number of pixels in each image, the low gradient blur pixels are determined as blur abnormal pixel points.

[0078] In another embodiment, the proportion of low gradient blur pixels in the whole image is recorded. If the proportion exceeds 8%, the pixels are further determined as blur abnormal pixel points. The 8% proportion threshold is also derived from experimental data. The statistical results show that when the proportion of blur pixels exceeds the threshold, the decoding failure rate significantly increases, and when it is between 5% and 8%, it can still be compensated by the error correction mechanism, so 8% is selected as the final determination condition.

[0079] Optionally, detecting the smudge abnormal pixel points comprises:

[0080] The texture direction dispersion and local contrast of each detection window are calculated.

[0081] In a specific embodiment, when analyzing the texture direction of the pixels in each detection window, a Gabor filter bank is used to perform multi-direction convolution processing on the pixels in the window to extract texture responses in different directions. The filter directions are set to 0°, 45°, 90° and 135°, the filter wavelength λ = 4 pixels, and the bandwidth σ = 2 pixels. These parameters are derived from the surface texture experiment statistics of the medicine packaging material, which can effectively capture the texture details of the barcode and two-dimensional code surface. The dominant texture direction of each pixel is calculated according to the filter responses in the four directions, and the texture direction distribution of all pixels in the window is counted. The dispersion degree of the texture direction is represented in the form of standard deviation as the texture direction dispersion index.

[0082] In the local contrast calculation, the local contrast of the pixel gray scale in each window is calculated in units of detection window size: where Imax I min respectively, and ∈ = 1 is used to avoid zero denominator.

[0083] If the texture direction dispersion in any detection window is higher than a preset dispersion threshold, and the local contrast of the detection window is lower than 50% of the average of the whole image contrast, then the pixels in the detection window are determined as texture discontinuous pixels.

[0084] In another embodiment, if the texture direction dispersion of a window is higher than 45° (statistically obtained from multiple images of medicine packaging) and the local contrast is lower than 50% of the average of the whole image gray scale contrast, then the pixels in the window are marked as texture discontinuous pixels.

[0085] The boundary contact conditions between the texture discontinuous pixels are detected, and the connected regions are divided according to the boundary contact conditions.

[0086] The number of pixels in each connected region is calculated, and if the number of pixels in each connected region is higher than 6% of the total number of pixels in each image, then the pixels in each connected region are determined as abnormal pixels.

[0087] In another embodiment, an 8-neighbor connected algorithm is used to divide the regions of the texture discontinuous pixels. Each texture discontinuous pixel is connected with the texture discontinuous pixels adjacent to it in the up, down, left, right and diagonal directions to form an initial connected region. Then, the number of pixels in the connected region is counted, and small regions with a number of pixels lower than 50 are removed to avoid noise interference. If the proportion of the number of pixels in the connected region to the total number of pixels in the image is higher than 6%, then the connected region is determined as abnormal pixels. For example, in an image of 1280x1280 pixels, if a connected region contains 10,500 texture discontinuous pixels, then the proportion is about 6.4%, which meets the determination condition, and the region is marked as abnormal pixels.

[0088] Optionally, the abnormal pixels in the abnormal perspective image are restored in step S3, including:

[0089] The abnormal pixel symbol positions in the abnormal perspective image are labeled, and the corresponding symbol positions in the normal perspective image are matched according to the abnormal pixel symbol positions;

[0090] In this embodiment, the abnormal pixel points in the abnormal view angle image are marked with symbol positions. The system maps each abnormal pixel point to the corresponding symbol cell (the smallest encoding unit of a two-dimensional code or a barcode) by scanning the pixel grid of the image, and marks the coordinate position of the abnormal pixel point in the image. In order to ensure the coordinate matching accuracy, a sub-pixel level interpolation method with a pixel spacing accuracy of 0.5 pixels is used to ensure that the abnormal pixel points can be accurately aligned between different view angle images. Subsequently, according to the symbol position of the abnormal pixel point, the corresponding symbol position in the normal view angle image is searched to establish a one-to-one correspondence between the abnormal pixel point and the normal view angle symbol position.

[0091] If the abnormal pixel point is a reflection abnormal pixel point, the gray value of the corresponding symbol position in the normal view angle image is used as a reference value to perform gray value interpolation on the reflection abnormal pixel point, and a reflection abnormal restoration image subset is output.

[0092] In this embodiment, for the restoration processing of the reflection abnormal pixel point, the gray value of the corresponding symbol position in the normal view angle image is used as a reference value. The gray value of the normal view angle symbol is mapped to the reflection pixel position in the abnormal view angle image through bilinear gray value interpolation to repair the overly bright or saturated pixels caused by reflection. The interpolation window size is set to 3x3 pixels, and the parameters are derived from scanning experiments on various drug two-dimensional codes, which can ensure gray value smoothness without blurring the symbol edge. After completing the gray value interpolation, a reflection abnormal restoration image subset is generated.

[0093] If the abnormal pixel point is a blur abnormal pixel point, the gradient information of the corresponding symbol position in the normal view angle image is used to perform gradient constraint reconstruction on the blur abnormal pixel point, and local frequency domain compensation is performed according to the constraint reconstruction result, and a blur abnormal restoration image subset is output.

[0094] In another embodiment, for the restoration processing of the blur abnormal pixel point, the gradient information of the corresponding symbol position in the normal view angle image is used for reconstruction. The Sobel gradient of each symbol in the normal view angle image is calculated, and gradient constraint reconstruction is performed in the blur pixel region with the gradient direction and amplitude as constraint conditions. Subsequently, in order to make up for the loss of local high frequency information, local frequency domain compensation is performed on the reconstructed region, and two-dimensional fast Fourier transform (FFT) with a window size of 8x8 pixels is used for high frequency enhancement in the frequency domain. The parameters are derived from blur image experimental tests to ensure that the symbol outline is clear after reconstruction. A blur abnormal restoration image subset is generated.

[0095] If the abnormal pixel point is a pollution abnormal pixel point, the local contrast of the corresponding symbol position in the normal view angle image is used to perform local contrast enhancement on the pollution abnormal pixel point, and a pollution abnormal restoration image subset is output.

[0096] In this embodiment, for the restoration processing of the abnormal pixel points of the pollution, the local contrast of the corresponding code element position in the normal view image is used as a reference. By performing linear contrast enhancement on the local window of 5x5 pixels around the pollution pixel, the local gray scale distribution of the normal view is mapped to the pollution pixel area to restore the code element texture and brightness contrast. After the enhancement is completed, the pollution abnormal restoration image subset is generated.

[0097] The reflection abnormal restoration image subset, the blur abnormal restoration image subset, the pollution abnormal restoration image subset, and the normal view image are merged, and edge texture reconstruction is performed according to the merging result to obtain the image set to be analyzed.

[0098] In another embodiment, the reflection abnormal restoration image subset, the blur abnormal restoration image subset, the pollution abnormal restoration image subset, and the normal view image are merged. In the merging process, a pixel weighted average method is used, in which the weight of the abnormal pixel point is set to 0.7 and the weight of the normal pixel is set to 0.3 to ensure that the restored area is naturally fused with the original image. After the merging is completed, edge texture reconstruction is performed on the image, and a Laplacian pyramid enhancement method is used to strengthen the code element edge and texture contour to generate the final image set to be analyzed. The number of layers of the pyramid for edge texture reconstruction is set to 3 layers, and the parameters are derived from the two-dimensional code recognition experimental data to ensure that the edge is clear and no noise is generated.

[0099] Optionally, the step S4 of classifying and extracting character data corresponding to each code type in parallel includes:

[0100] According to the detection result of the code type, the image set to be analyzed is divided into image subsets of different code types;

[0101] In this embodiment, according to the code type detection result of the image set to be analyzed, the image set is divided into a plurality of image subsets. Each subset contains code images of the same type, such as two-dimensional codes, bar codes, or DataMatrix codes. The system extracts feature vectors, including code element density, rectangular ratio, and encoding direction, by performing code type identification on the image. The determination threshold parameters are set to a ratio interval of 0.85 to 1.15 of the code element density, and a rectangular ratio error of not more than 5%, which are derived from statistical analysis of 500 samples of two-dimensional codes of medicines. After the determination is completed, the images that meet the same code type are classified into the same subset.

[0102] Each code type image subset is assigned a preset decoding channel corresponding to the code type;

[0103] In this embodiment, a preset decoding channel is assigned to each image subset of a code type. Each decoding channel is an independent processing thread, which has built-in decoding rules and parameters corresponding to the code type. The decoding channel structure includes a symbol scanner, a code parser, and a character output interface. The symbol scanner scans image symbols row by row with sub-pixel accuracy of 0.5 pixels, the code parser calls a specific code table to complete the mapping of symbols to characters according to the code type, and the character output interface saves the parsing result as serialized data. The number of decoding channel threads is dynamically allocated according to the number of subsets, each channel is allocated at least 1 CPU core and 50 MB of memory, and the parameters are derived from experimental tests to ensure the efficiency and stability of parallel decoding.

[0104] It is noted that the decoding channel allocation rule is that each code type corresponds to an independent decoding channel. For example, all two-dimensional code type (QR Code) image subsets are allocated to the two-dimensional code decoding channel, and all barcode type (EAN-13) image subsets are allocated to the barcode decoding channel. This rule ensures that the decoding process of each code type uses the most suitable code table and decoding parameters, avoiding errors caused by mixed decoding. When the number of image subsets of a code type is large, the subset can be further split into several batches and allocated to multiple parallel decoding channel instances of the same type in batches. The splitting ratio is based on the ratio of the number of image subsets to the number of decoding channel instances, for example, if the number of QR Code image subsets is 1000 and the number of decoding channel instances is 4, then each channel instance is allocated 250 images. For suspected abnormal code type images, they can be preferentially allocated to the channel instance with the most idle processing capacity when allocated, to ensure fast processing and support subsequent rescanning operations. The channel idle state is monitored in real time by an internal scheduling queue, which is updated every 50 ms.

[0105] It is noted that each decoding channel can include three main components: a symbol positioning unit, a symbol parsing unit, and a character data output unit. The symbol positioning unit scans the symbol region in the image subset through a sliding window, the window size is set to 4x4 pixels for two-dimensional codes or 8x2 pixels for barcodes according to the code type, and the scanning step is 2 pixels, which can quickly lock the symbol boundary through gray gradient and edge detection. This unit can dynamically adjust the window size to adapt to images of different resolutions. The symbol parsing unit performs encoding mapping on each symbol after symbol positioning. The encoding table corresponding to the code type is stored in the parsing unit, such as the ISO / IEC 18004:2015 standard table for two-dimensional codes and the EAN-13 standard table for barcodes. During the parsing process, the symbol gray value is normalized (range 0-255), and the symbol with a rotation error of ±3° is rotated to align the center of the symbol with the standard grid, thereby improving the decoding accuracy. If the decoding confidence of a symbol during the parsing process is less than 0.95, two decoding attempts are repeated, and the confidence distribution is recorded. The character data output unit combines the parsed symbol characters into a complete character sequence in the scanning order, with the code type identifier and image source identifier. The output format is a text sequence encoded in UTF-8, and a confidence list of each symbol is generated, which is used for subsequent database comparison and abnormality determination. In the construction process, all preset decoding channels use a multi-thread parallel structure, each channel is allocated 1 CPU core and 50MB of memory, ensuring that image subsets of various code types can be processed independently. After the channel construction is completed, test image samples are used to verify the decoding success rate, and the results show that for 1000 mixed code type pharmaceutical images, the average decoding success rate can reach 99.2%, and the average confidence of each character is 0.97. Finally, the results output by the preset decoding channels of various code types include: complete character data sequence, corresponding code type identifier, image source identifier, and confidence data of each symbol

[0106] In each decoding channel, symbol decoding is performed in parallel, and character data corresponding to each code type is extracted based on the symbol decoding result.

[0107] In another embodiment, in each decoding channel, symbol decoding is independently and concurrently performed. Each channel adopts a pipeline processing structure, first completes symbol boundary positioning, then performs symbol decoding according to the symbol arrangement order, and during decoding, performs gray scale correction and rotation correction for each symbol, and the rotation angle error is allowed to be within ±3°. After completing symbol decoding, the symbol characters are combined to generate a complete character data sequence in the scanning order. The character data output format is UTF-8 encoding, and the code type identification and image source identification are recorded. During decoding, the confidence of each symbol is evaluated, and the threshold is set to 0.95. Symbols below the confidence will be decoded twice in the pipeline to ensure that the extracted character data is accurate and reliable. The character data corresponding to each code type is output from each decoding channel and combined to form a complete character data set, which is used for subsequent comparison with the drug traceability code database. The entire process is executed in multiple threads in parallel to maximize the use of CPU core resources, and the average decoding speed can reach 50 images / second. The parameters are derived from experimental data obtained by testing 1000 multi-code drug images.

[0108] Optionally, the present application also provides a drug traceability code scanning and identifying system for executing the drug traceability code scanning and identifying method as described above, which comprises:

[0109] A data acquisition module 101 is configured to acquire a multi-angle image set by multi-angle scanning of a traceability code region of a drug to be identified through a pre-deployed scanning device;

[0110] An abnormal pixel point detection module 102 is configured to detect abnormal pixel points according to the multi-view image set, and divide the multi-view image set into abnormal view images and normal view images according to the abnormal pixel point detection result;

[0111] An abnormality restoration module 103 is configured to splice and restore abnormal pixel points in the abnormal view images by using the traceability code composition features of the normal view images to obtain an image set to be analyzed;

[0112] A character extraction module 104 is configured to detect code type in the image set to be analyzed, and extract character data corresponding to each code type in parallel according to the number of code types;

[0113] A drug identification module 105 is configured to compare the character data corresponding to each code type by using a pre-set drug traceability code database, and output drug circulation information according to the comparison result.

[0114] Optionally, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed to implement the drug traceability code scanning and identifying method as described above.

[0115] Therefore, the embodiments should be regarded, at any point, as being exemplary and not limiting, the scope of the application being defined by the appended claims and not by the above description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

[0116] The foregoing is considered as illustrative only of the principles of the application. Numerous modifications and changes will readily occur to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for scanning and identifying drug traceability codes, characterized in that, Includes the following steps: Step S1: Scan the traceability code area of ​​the drug to be identified from multiple angles using a pre-deployed scanning device to obtain a multi-angle image set; Step S2: Detect abnormal pixels based on the multi-view image set, and divide the multi-view image set into abnormal view images and normal view images based on the abnormal pixel detection results. Step S3: Using the traceability code composition features of normal view images, stitch together and restore the abnormal pixels in abnormal view images to obtain the image set to be analyzed; Step S4: Detect the code type in the image to be analyzed, and extract the character data corresponding to each code type in parallel according to the number of code types. Step S5: Use the preset drug traceability code database to compare the character data corresponding to each code type, and output the drug circulation information based on the comparison results.

2. The drug traceability code scanning and identification method according to claim 1, characterized in that, Step S5, which outputs drug distribution information based on the comparison results, includes: If the character data corresponding to each code type in the comparison results matches the drug traceability code in the drug traceability code database, then the drug to be identified is determined to be a genuine drug, and the drug circulation information corresponding to the genuine drug is output. If the character data corresponding to each code type in the comparison results does not match the drug traceability code in the drug traceability code database, the drug to be identified is judged as a suspected counterfeit drug, and the scanning device is controlled to scan the traceability code area of ​​the suspected counterfeit drug, and steps S2 to S5 are executed again.

3. The drug traceability code scanning and identification method according to claim 2, characterized in that, Methods for identifying genuine medicines also include: If the character data corresponding to each code type matches the drug traceability code of the corresponding code type in the drug traceability code database, then the drug to be identified is determined to be a genuine drug. If the character data corresponding to each code type has a missing code type match with the drug traceability code of the corresponding code type in the drug traceability code database, then the drug to be identified is judged as a suspected abnormal drug. Control the scanning device to scan the traceability code area corresponding to the missing matching code type in the suspected abnormal drug, and re-execute steps S2 to S5.

4. The drug traceability code scanning and identification method according to claim 1, characterized in that, Abnormal pixels in step S2 include reflective abnormal pixels, blurry abnormal pixels, and dirty abnormal pixels; The pixels that detect reflective abnormalities include: The images in the multi-view image set are divided into several detection windows, and the mean gray value and standard deviation of the pixels in each detection window are calculated one by one. If the mean gray level in any detection window is higher than 30% of the mean gray level of the entire image and the standard deviation is lower than the preset standard deviation threshold, then the pixel in the detection window is determined to be a bright clustered pixel. Calculate the proportion of bright clustered pixels in each image. If the proportion of bright clustered pixels is higher than 10% of the total number of pixels in each image, then the bright clustered pixels are identified as reflective abnormal pixels.

5. The drug traceability code scanning and identification method according to claim 4, characterized in that, Detecting blurry or abnormal pixels includes: Perform edge gradient calculation and frequency domain energy distribution statistics on pixels within each detection window; If the mean edge gradient within any detection window is less than 40% of the mean edge gradient of the entire image, and the proportion of high-frequency components in the frequency domain energy distribution of the detection window is less than the preset high-frequency component proportion threshold, then the pixel within the detection window is determined to be a low-gradient blurred pixel. Calculate the proportion of low-gradient blurred pixels in each image. If the proportion of low-gradient blurred pixels is higher than 8% of the total number of pixels in each image, then the low-gradient blurred pixels are identified as blurred abnormal pixels.

6. The drug traceability code scanning and identification method according to claim 4, characterized in that, Detection of contaminated or abnormal pixels includes: Calculate the texture orientation dispersion and local contrast of each detection window; If the texture direction dispersion in any detection window is higher than the preset dispersion threshold, and the local contrast of the detection window is lower than 50% of the average contrast of the entire image, then the pixel in the detection window is determined to be a texture discontinuous pixel. Detect the boundary contact between discontinuous pixels in the texture and divide the connected regions based on the boundary contact. Calculate the number of pixels in each connected region. If the number of pixels in each connected region is higher than 6% of the total number of pixels in the image, then the pixels in each connected region are identified as dirty or abnormal pixels.

7. The drug traceability code scanning and identification method according to claim 1, characterized in that, The abnormal pixels in the image stitched and restored from the abnormal viewpoint in step S3 include: Mark the positions of abnormal pixels in the abnormal view image, and match the positions of the abnormal pixels with the corresponding positions in the normal view image. If the abnormal pixel is a reflective abnormal pixel, then the grayscale mean value of the corresponding symbol position in the normal view image is used to perform grayscale interpolation on the reflective abnormal pixel, and output a subset of the reflected abnormal restored image. If the abnormal pixel is a blurred abnormal pixel, then gradient constraint reconstruction is performed on the blurred abnormal pixel using the gradient information of the corresponding symbol position in the normal view image, and local frequency domain compensation is performed based on the constraint reconstruction result to output a subset of blurred abnormal restored image. If the abnormal pixel is a dirty abnormal pixel, then the local contrast of the corresponding symbol position in the normal view image is used to perform local contrast enhancement on the dirty abnormal pixel, and output a subset of the dirty abnormal restored image. The image subsets of the reflected anomaly restoration image subset, the blurred anomaly restoration image subset, and the dirt anomaly restoration image subset are merged with the normal viewpoint image, and edge texture reconstruction is performed based on the merging result to obtain the image set to be analyzed.

8. The drug traceability code scanning and identification method according to claim 1, characterized in that, Step S4 involves classifying and extracting character data corresponding to each code type in parallel, including: Based on the detection results of the code type, the image set to be analyzed is divided into image subsets of different code types; Assign preset decoding channels corresponding to each encoding type to a subset of images of each encoding type; Within each decoding channel, symbol decoding is performed in parallel, and character data corresponding to each code type is extracted based on the symbol decoding results.

9. A drug traceability code scanning and identification system, characterized in that, For performing the drug traceability code scanning and identification method as described in claim 1, the drug traceability code scanning and identification system includes: The data acquisition module is used to scan the traceability code area of ​​the drug to be identified from multiple angles using a pre-deployed scanning device to obtain a multi-angle image set; The abnormal pixel detection module is used to detect abnormal pixels based on a multi-view image set, and to divide the multi-view image set into abnormal view images and normal view images based on the abnormal pixel detection results. The anomaly restoration module is used to use the traceability code composition features of normal view images to stitch together and restore the abnormal pixels in abnormal view images to obtain the set of images to be analyzed. The character extraction module is used to detect the code type in the image to be analyzed, and extract the character data corresponding to each code type in parallel according to the number of code types. The drug identification module is used to compare the character data corresponding to each code type using a preset drug traceability code database, and output drug circulation information based on the comparison results.

10. A computer-readable storage medium, characterized in that, It contains a computer program that, when executed, implements the drug traceability code scanning and identification method as described in any one of claims 1-8.

Images