Drug supervision code batch identification method and system based on neural network

Abstract

The invention provides a drug supervision code batch identification method and system based on a neural network, and the method comprises the steps: collecting an original image set of drug supervision codes, inputting the original image set into a neural network model, and generating multi-scale supervision code feature data corresponding to a single drug package, and performing parallel decoding on the overlapped and shielded medicine supervision code areas of the multi-scale supervision code feature data, generating a supervision code text sequence uniquely associated with each medicine packaging body, comparing the supervision code text sequence with a medicine production batch number database, generating a batch recognition result map, and performing batch recognition on the batch recognition result map. The accuracy of medicine supervision code identification, the capability of processing complex conditions, the analysis speed and the transparency and the reliability of medicine tracking management are improved, and the working efficiency and the management level in large-scale application are greatly improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of drug supervision, and particularly to a method and system for batch recognition of drug supervision codes based on neural networks. Background Art

[0002] With the continuous development of modern drug production and circulation management, it has become crucial to ensure the accuracy of the origin and flow of each drug. The application scenario requires the ability to efficiently and accurately identify the supervision codes on drug packages to achieve effective tracking and monitoring of the entire process from drug production to sales. Especially in large-scale production and distribution environments, quickly and accurately reading and verifying the information on drug packages plays an irreplaceable role in preventing counterfeit drugs from entering the market and protecting public health. With the continuous growth of the types and quantities of drugs, the demand for batch recognition of drug supervision codes is increasing day by day;

[0003] With the application of traditional technologies in the field of drug supervision code recognition, existing solutions usually rely on traditional image processing technologies and simple OCR methods. These solutions capture images of the drug package surface and use predefined template matching or feature extraction algorithms to try to parse the supervision codes. Although they can meet basic requirements in some cases, their accuracy highly depends on factors such as the shooting angle, lighting conditions, and the flatness of the package surface. In addition, when there are overlaps or partial occlusions between supervision codes, it is often difficult to provide reliable decoding results, resulting in low recognition efficiency and high error rates. Summary of the Invention

[0004] The embodiments of the present application provide a method and system for batch recognition of drug supervision codes based on neural networks to solve the problems of low accuracy, low efficiency in drug supervision code recognition in the prior art, and difficulty in dealing with complex package surfaces (such as overlaps and occlusions).

[0005] In a first aspect, the embodiments of the present application provide a method for batch recognition of drug supervision codes based on neural networks, including:

[0006] Collecting a set of original images of the drug package surface containing densely arranged drug supervision codes;

[0007] Inputting the set of original images into a preset neural network model, and using the neural network model to generate multi-scale supervision code feature data corresponding to a single drug package through cross-view image matching;

[0008] Parallel decoding is performed on the drug supervision code regions with overlaps and occlusions in the multi-scale supervision code feature data to generate a supervision code text sequence uniquely associated with each drug package;

[0009] Compare the regulatory code text sequence with a preset database of drug production batch numbers in real time to generate a batch identification result map.

[0010] Optionally, perform parallel decoding on the drug regulatory code regions where the multi-scale regulatory code feature data has overlaps and occlusions to generate a regulatory code text sequence uniquely associated with each drug package, including:

[0011] Generate a dynamic attention mask weight matrix corresponding to the overlapping and occluded regions based on the character topological relationship after the drug package surface deformation compensation in the multi-scale regulatory code feature data;

[0012] Perform channel-by-channel fusion of the dynamic attention mask weight matrix with the character topological relationship after the surface deformation compensation in the multi-scale regulatory code feature data to obtain a fusion result, and extract the multi-level feature vector groups in each overlapping and occluded region in the fusion result;

[0013] According to the multi-level feature vector groups, perform two-way context association decoding on the drug regulatory code characters in each overlapping and occluded region through a parallel decoder to generate a set of candidate text segments including character position confidence;

[0014] Perform character space continuity verification based on surface deformation compensation on the set of candidate text segments, screen out the valid text segments that meet the preset continuity constraints, and merge the valid text segments according to the spatial arrangement order of the drug packages on the production line to generate a regulatory code text sequence uniquely associated with each drug package.

[0015] Optionally, perform character space continuity verification based on surface deformation compensation on the set of candidate text segments, screen out the valid text segments that meet the preset continuity constraints, and merge the valid text segments according to the spatial arrangement order of the drug packages on the production line to generate a regulatory code text sequence uniquely associated with each drug package, including:

[0016] Calculate the curvature matching index of each character in the set of candidate text segments based on the character topological relationship after surface deformation compensation;

[0017] Dynamically adjust the character spacing threshold range in the continuity constraint according to the character distribution statistical characteristics of the decoded drug regulatory codes in the same drug production batch to obtain an adjusted character spacing threshold range;

[0018] Mark the candidate text segments that meet the condition that the curvature matching index exceeds the upper limit of the surface deformation compensation error and the character spacing is within the adjusted character spacing threshold range as valid text segments, and record the spatial position mapping table of the valid text segments in the production line conveyor coordinate system;

[0019] According to the spatial position mapping table, perform character sequence splicing based on the conveyor belt movement direction on the valid text segments belonging to the same drug package to obtain the spliced character sequence, and eliminate the text segment conflicts of different drug packages according to the monotonically increasing order of the character position confidence levels in the spliced character sequence, and generate a supervision code text sequence uniquely associated with each drug package.

[0020] Optionally, input the original image set into a preset neural network model, and use the neural network model to generate multi-scale supervision code feature data corresponding to a single drug package through cross-view image matching, including:

[0021] Perform temporal alignment on the original image set based on the movement speed of the pipeline conveyor belt to generate a multi-view image frame sequence;

[0022] Perform cross-view feature point matching on each drug package in the multi-view image frame sequence through the neural network model to generate a cross-view region correlation matrix reflecting the spatial distribution of supervision code characters between adjacent views;

[0023] According to the cross-view region correlation matrix, construct a geometric deformation field associated with the geometric deformation parameters of the drug package surface, and use the geometric deformation field to map the multi-view image frame sequence to a unified reference coordinate system to generate multi-scale supervision code feature data.

[0024] Optionally, according to the cross-view region correlation matrix, construct a geometric deformation field associated with the geometric deformation parameters of the drug package surface, and use the geometric deformation field to map the multi-view image frame sequence to a unified reference coordinate system to generate multi-scale supervision code feature data, including:

[0025] Based on the spatial distribution characteristics of drug supervision code characters between adjacent views in the cross-view region correlation matrix, calculate a dynamic deformation offset positively correlated with the curvature gradient of the drug package surface;

[0026] According to the dynamic deformation offset and the surface curvature gradient, generate a geometric deformation field covering the entire domain of the drug package surface;

[0027] Input the spatial coordinates of each pixel point in the multi-view image frame sequence into the geometric deformation field, and use the curvature gradient matrix in the geometric deformation field to perform affine transformation compensation on the spatial coordinates of each pixel point to obtain the transformed spatial coordinates;

[0028] Use the dynamic deformation offset field to perform secondary correction on the transformed spatial coordinates to generate multi-scale supervision code feature data.

[0029] Optionally, compare the regulatory code text sequence with a preset database of drug production batch numbers in real time to generate a batch identification result graph, including:

[0030] Based on the character distribution characteristics of the decoded drug regulatory code text sequences in the same drug production batch, dynamically construct a batch character entropy value baseline associated with the change amount of the pipeline conveyor belt speed and the spatial arrangement density of drug packages;

[0031] Perform spatio-temporal alignment comparison between the decoding position confidence of each character in the regulatory code text sequence and the batch character entropy value baseline to generate an entropy value mutation coefficient matrix reflecting abnormal character distribution patterns;

[0032] According to the entropy value mutation coefficient matrix and the spatial position mapping table of drug packages in the pipeline conveyor belt coordinate system, perform edge position backtracking and positioning on drug regulatory codes exceeding the preset entropy value mutation threshold to generate an abnormal marking data cluster;

[0033] Perform weighted superposition of the abnormal marking data cluster and the surface deformation compensation parameters in the multi-scale regulatory code feature data to generate a batch identification result graph.

[0034] Optionally, based on the character distribution characteristics of the decoded drug regulatory code text sequences in the same drug production batch, dynamically construct a batch character entropy value baseline associated with the change amount of the pipeline conveyor belt speed and the spatial arrangement density of drug packages, including:

[0035] Based on the entropy value mutation amount of each abnormal character in the entropy value mutation coefficient matrix and the gradient change trend of adjacent characters, construct an abnormal area diffusion model positively correlated with the curvature gradient of the drug package surface;

[0036] Use the abnormal area diffusion model to locate the initial physical coordinate set of abnormal regulatory codes in the pipeline conveyor belt coordinate system, and perform point-by-point calibration on the initial physical coordinate set according to the reflectivity difference correction amount between adjacent drug package edges in the multi-scale regulatory code feature data to generate an abnormal physical coordinate cluster after reflectivity correction;

[0037] Match the abnormal physical coordinate cluster after reflectivity correction with the spatial arrangement order of the corresponding drug packages in the spatial position mapping table, screen out an abnormal coordinate subset that meets the preset spatial continuity constraint, and associate the corresponding character distribution abnormal level in the entropy value mutation coefficient matrix;

[0038] According to the abnormal coordinate subset and the associated character distribution abnormal level, generate an abnormal marking data cluster including deformation compensation residual amount, reflectivity difference correction amount and entropy value mutation amount.

[0039] Second aspect, an embodiment of the present application provides a batch identification system for drug supervision codes based on a neural network, including:

[0040] An acquisition module, configured to acquire an original image set including densely arranged drug supervision codes on the surface of a drug package;

[0041] An input module, configured to input the original image set into a preset neural network model, and use the neural network model to generate multi-scale supervision code feature data corresponding to a single drug package through cross-view image matching;

[0042] A decoding module, configured to perform parallel decoding on the drug supervision code regions with overlapping and occluded multi-scale supervision code feature data, and generate a supervision code text sequence uniquely associated with each drug package;

[0043] A comparison module, configured to compare the supervision code text sequence with a preset drug production batch number database in real time, and generate a batch identification result map.

[0044] Third aspect, an embodiment of the present application provides a computing device, including a processor and a memory, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the methods for batch identification of drug supervision codes based on a neural network in the first aspect.

[0045] Fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the methods for batch identification of drug supervision codes based on a neural network in any one of the first aspect are implemented.

[0046] In an embodiment of the present application, an original image set including densely arranged drug supervision codes on the surface of a drug package is acquired; the original image set is input into a preset neural network model, and the neural network model is used to generate multi-scale supervision code feature data corresponding to a single drug package through cross-view image matching; parallel decoding is performed on the drug supervision code regions with overlapping and occluded multi-scale supervision code feature data, and a supervision code text sequence uniquely associated with each drug package is generated; the supervision code text sequence is compared with a preset drug production batch number database in real time, and a batch identification result map is generated.

[0047] The technical solution of this application collects the original image set containing densely arranged drug supervision codes on the surface of drug packages, and uses a preset neural network model to process these images to generate multi-scale supervision code feature data. This method can effectively address the problems of complex geometric deformation and perspective change on the surface of drug packages, thereby improving the recognition accuracy. For the supervision code areas with overlapping and occlusion situations, a parallel decoding strategy is adopted to generate a supervision code text sequence uniquely associated with each drug package body, greatly enhancing the ability and efficiency of parsing drug supervision codes in complex situations. Finally, by comparing the generated supervision code text sequence with the drug production batch number database in real time, the drug tracking management task can be completed quickly and accurately, improving the transparency and reliability of the entire process.

[0048] Furthermore, a dynamic attention mask weight matrix is generated to compensate for the character topological relationship after the surface deformation of the drug package, ensuring that character information can be accurately extracted even under complex package surface conditions. Then, by combining multi-level feature vector groups in a per-channel fusion manner, the model's understanding ability of features at different scales is enhanced, which helps to more accurately locate and identify supervision code characters. Next, a bidirectional context-related decoding technology is adopted, which not only considers the information of a single character but also fully utilizes the context clues of its surrounding characters, significantly improving the accuracy and robustness of the recognition result. Finally, by verifying the spatial continuity of the candidate text segment set and merging valid text segments in the spatial order of the drug package body, the authenticity and integrity of the finally generated supervision code text sequence are ensured, greatly avoiding the possibility of misrecognition and providing strong technical support for drug supervision.

[0049] These aspects or other aspects of this application will be more clearly understood in the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0051] Figure 1 It is a flowchart of a method for batch recognition of drug supervision codes based on a neural network provided by an embodiment of this application;

[0052] Figure 2 It is a schematic structural diagram of a system for batch recognition of drug supervision codes based on a neural network provided by an embodiment of this application;

[0053] Figure 3A schematic structural diagram of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0054] In order to enable those skilled in the art of the present technology to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

[0055] In some processes described in the specification, claims and the above-mentioned accompanying drawings of the present application, a plurality of operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., do not represent a sequence, and do not limit that "first" and "second" are of different types.

[0056] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0057] Figure 1 A flowchart of a method for batch identification of drug supervision codes based on a neural network is provided for an embodiment of the present application. As Figure 1 shown, the method includes:

[0058] Step 101, collecting an original image set containing densely arranged drug supervision codes on the surface of a drug package;

[0059] In this step, the original image set refers to a series of two-dimensional images obtained by shooting or scanning through a specific device (such as an industrial camera or a scanner) and containing densely arranged drug supervision codes on the surface of the drug package. These images not only include the supervision codes themselves, but may also include other elements such as packaging backgrounds and text information. In order to ensure the effectiveness of subsequent processing, the quality of the images is crucial. Therefore, factors such as shooting angles, lighting conditions, and resolution need to be controlled to obtain a clear and consistent image data set.

[0060] In this embodiment, first, a high-resolution industrial camera is installed on the production line to ensure that it can cover the entire surface of the drug packaging. Then, the angle and focal length of the camera are adjusted so that it can accurately capture the supervision codes on each drug packaging. Next, a suitable light source is set to avoid image blurring caused by shadows or reflections. Then, the frame rate of the camera is configured according to the speed of the assembly line to ensure that each drug packaging can be completely photographed. Finally, the captured images are saved in a standard format (such as JPEG or PNG) using image acquisition software.

[0061] For example, a certain pharmaceutical factory deploys multiple high-resolution industrial cameras on its high-speed automated packaging line. These cameras are evenly distributed on both sides of the conveyor belt to ensure that all supervision codes on the drug packaging can be photographed from different angles. When the drug packaging passes under the camera, the camera will be automatically triggered to take pictures, generating a series of high-quality original image sets. For example, at a certain moment, a tray containing multiple drugs passes under the camera, and the camera takes an image set containing 50 drug packagings according to the preset parameters. Each image clearly shows the supervision code on the packaging surface and the details around it.

[0062] Step 102: Input the original image set into a preset neural network model, and use the neural network model to generate multi-scale supervision code feature data corresponding to a single drug packaging body through cross-view image matching.

[0063] In this step, cross-view image matching refers to using a neural network model to analyze multiple images of the same drug packaging taken from different angles and extract the common features among them.

[0064] In this embodiment, first, the collected original image set is imported into a pre-trained neural network model. This model is based on deep learning technology and has powerful feature extraction capabilities. The model will perform multi-scale feature extraction on each image and find the common feature points from different perspectives through a cross-view image matching algorithm. Then, combining the spatial relationships of these feature points, multi-scale supervision code feature data corresponding to a single drug packaging body is generated.

[0065] For example, continuing with the scenario of the pharmaceutical factory in the above example, after the central server receives the original image set of 50 drug packagings taken by the industrial camera, it immediately inputs these images into the neural network model. The model starts to process each image, extracts the supervision code feature information at various scales. For those supervision codes that are difficult to directly identify due to the packaging surface or occlusion, the model finds the consistent feature points from different perspectives through cross-view image matching technology and successfully generates the multi-scale supervision code feature data for each drug packaging.

[0066] Step 103: Parallelly decode the drug supervision code areas where there are overlaps and occlusions in the multi-scale supervision code feature data to generate a supervision code text sequence uniquely associated with each drug package body;

[0067] In this step, parallel decoding means using a neural network model and related algorithms to process multiple drug supervision code areas simultaneously, especially those with overlaps or partial occlusions. Through a dynamic attention mask weight matrix (used to compensate for the character topological relationship after the deformation of the drug package surface), a multi-level feature vector group (a set of key features extracted from the fusion result), and a bidirectional context-associated decoder (which can consider the context information around the characters), the system can efficiently parse each drug supervision code character and generate a set of candidate text segments containing the character position confidence.

[0068] In this embodiment, first, based on the multi-scale supervision code feature data, the system generates a dynamic attention mask weight matrix to compensate for the character deformation problem caused by the surface curvature deformation of the drug package. Then, this weight matrix is fused with the multi-scale supervision code feature data channel by channel to extract the multi-level feature vector group in each overlapping and occluding area. Next, the parallel decoder is used to perform bidirectional context-associated decoding on these feature vector groups to generate a set of candidate text segments containing the character position confidence. Finally, the system performs character space continuity verification based on surface deformation compensation on these candidate text segments, filters out the valid text segments that meet the preset continuity constraints, and merges them into a unique supervision code text sequence according to the spatial arrangement order of the drug package bodies.

[0069] For example, continuing with the above example of the pharmaceutical factory, after the central server receives the multi-scale supervision code feature data, the system starts to process those supervision code areas that are difficult to directly identify due to the bending of the packaging surface or partial occlusion. For example, on a certain drug package, the supervision code has some character deformations and slight overlaps due to slight wrinkles in the packaging material. The system uses the dynamic attention mask weight matrix to precisely compensate this area and extracts the key features through the multi-level feature vector group. Then, the system uses the parallel decoder to perform bidirectional context-associated decoding on these features and successfully generates a set of candidate text segments containing the character position confidence. Specifically, for a supervision code containing "ABCD12345678", the system first generates multiple possible candidate text segments, such as "ABCD1234", "BCD12345", etc. Through the character space continuity verification based on surface deformation compensation on these candidate text segments, the system filters out the valid text segments that meet the continuity constraints and merges them according to the order of the drug package bodies on the production line to generate a unique supervision code text sequence "ABCD12345678".

[0070] Step 104: Compare the supervision code text sequence with a preset drug production batch number database in real time to generate a batch recognition result graph;

[0071] In this step, the batch recognition result graph is a summary of the results generated by comparing the supervision code text sequence corresponding to each drug package with the drug production batch number database in real time. It not only includes the matching situation between the supervision code and the batch number, but also records any abnormal or unmatched data points for subsequent quality control and traceability management.

[0072] In this embodiment, first, the generated supervision code text sequence is compared with the preset drug production batch number database in real time. By comparing the characters in each supervision code text sequence with the corresponding records in the database, the system can quickly determine whether there are matching items. If abnormalities are found (such as unmatched supervision codes or batch numbers that do not meet expectations), the system will automatically generate marks and add relevant information to the batch recognition result graph. In this way, operators can quickly locate and solve potential problems by viewing the graph.

[0073] For example, continuing with the pharmaceutical factory scenario in the above example, the system compares the generated supervision code text sequence "ABCD12345678" with the drug production batch number database. The result shows that "ABCD12345678" exactly matches the production record of a certain batch, indicating that the drug package meets the standard. However, the supervision code "XYZW98765432" of another package fails to find the corresponding batch number record. The system immediately marks this package as abnormal and records detailed information (including package image, shooting time, production line position, etc.) in the batch recognition result graph. In this way, the quality control team can quickly take action to check whether there are risks of counterfeiting or other quality problems with this package.

[0074] In the process of identifying drug supervision codes, complex situations on the package surface (such as curved surface deformation, overlap, and occlusion) are the main challenges affecting the recognition accuracy. Traditional image processing techniques are difficult to handle these complex situations. Based on this, in some embodiments, according to what is described in step 103, parallel decoding is performed on the drug supervision code areas where there are overlaps and occlusions in the multi-scale supervision code feature data to generate a supervision code text sequence uniquely associated with each drug package, including:

[0075] Step 201: Generate a dynamic attention mask weight matrix corresponding to the overlapping and occluding areas based on the character topological relationship after compensation for the curved surface deformation of the drug package in the multi-scale supervision code feature data;

[0076] In this step, the multi-scale supervision code feature data refers to various feature information of the supervision code on the drug packaging surface extracted from different perspectives and resolutions. These feature data include the position, shape, size of the supervision code characters and their relationships with each other; the drug packaging surface deformation compensation refers to correcting the geometric deformation caused by the bending or wrinkling of the drug packaging surface through an algorithm to restore the true form of the supervision code characters; the character topological relationship describes the spatial arrangement and connection method between the supervision code characters, and can maintain certain structural features even in the case of deformation and occlusion; the dynamic attention mask weight matrix is an adaptive weight allocation mechanism that dynamically adjusts the importance weights of each character region according to the character topological relationship, thereby enhancing the recognition ability for overlapping and occluded regions.

[0077] In this embodiment, after receiving the multi-scale supervision code feature data, the processing of the supervision code on a certain drug packaging is started. Assuming that there is a slight bend on the packaging surface, resulting in deformation and slight overlap of some supervision code characters, first, a neural network model is used to analyze this situation, calculate the specific deformation parameters, and compensate the characters to restore them to a more standard form. Then, based on the compensated character topological relationship, a dynamic attention mask weight matrix is generated. In this matrix, higher weights are assigned to those regions with severe overlap and occlusion, while lower weights are assigned to the clearly visible character regions.

[0078] Step 202, perform channel-by-channel fusion of the dynamic attention mask weight matrix and the character topological relationship after surface deformation compensation in the multi-scale supervision code feature data to obtain a fusion result, and extract the multi-level feature vector groups in each overlapping and occluded region in the fusion result;

[0079] In this step, the dynamic attention mask weight matrix is an adaptive weight allocation mechanism that dynamically adjusts the importance weights of each character region according to the character topological relationship, thereby enhancing the recognition ability for overlapping and occluded regions; channel-by-channel fusion refers to combining the dynamic attention mask weight matrix with the different channel information in the multi-scale supervision code feature data on a channel-by-channel basis to generate a more accurate feature representation; the multi-level feature vector group is a set of key features at different levels extracted from the fusion result. These features can better describe the detailed features of the supervision code characters and contribute to the subsequent decoding process.

[0080] In this embodiment, first, the dynamic attention mask weight matrix is fused with the character topological relationship after surface deformation compensation in the multi-scale supervision code feature data channel by channel. For each channel (such as the color channel, depth channel, etc.), the dynamic attention mask weight matrix is applied to the corresponding feature data, and the fusion result is generated by weighted summation. The purpose of doing this is to highlight those regions that are more important during the recognition process (such as overlapping and occluded regions), while reducing the influence of unimportant regions. Next, multi-level feature vector groups within each overlapping and occluded region are extracted from the fusion result. These feature vector groups contain key feature information at different scales and can comprehensively describe each character and its surrounding context.

[0081] Step 203: According to the multi-level feature vector groups, use a parallel decoder to perform two-way context association decoding on the drug supervision code characters within each overlapping and occluded region, and generate a set of candidate text segments containing character position confidence levels.

[0082] In this step, the multi-level feature vector groups are key feature sets at different levels extracted from the fusion result, and these features can better describe the detailed features of the supervision code characters; the parallel decoder is a decoding mechanism that efficiently processes multiple character regions and can process the character information of multiple overlapping and occluded regions at the same time; two-way context association decoding means not only considering the information of a single character, but also combining the context clues of its preceding and following characters to improve the recognition accuracy; the character position confidence level is an evaluation value of the position and correctness of each character in the image and is used to measure the reliability of the decoding result. The set of candidate text segments is a series of possible supervision code text fragments generated by the system, and each fragment contains the characters and their position confidence levels.

[0083] In this embodiment, first, use the multi-level feature vector groups as inputs and send them into the parallel decoder. The parallel decoder will perform two-way context association decoding on each character, that is, not only consider the information of the current character itself, but also combine the information of its preceding and following characters, so as to more accurately identify each character. For example, for a partially occluded character "5", the system not only analyzes the shape features of this character itself, but also refers to the context information of its adjacent characters on the left and right (such as "4" and "6") to determine the most likely character form. In this way, the system can generate a set of candidate text segments containing character position confidence levels. Each candidate text segment not only contains the recognized character sequence, but also includes the position of each character in its image and its confidence score. This process helps the subsequent screening and verification steps to ensure that the finally generated supervision code text sequence has high accuracy.

[0084] Step 204: Perform character space continuity verification based on surface deformation compensation on the set of candidate text segments, screen out valid text segments that meet the preset continuity constraints, and merge the valid text segments in the spatial arrangement order of the drug packages on the production line to generate a supervision code text sequence uniquely associated with each drug package;

[0085] In this step, the character space continuity verification based on surface deformation compensation refers to verifying the spatial continuity and logical consistency between characters while considering the surface curvature deformation of the drug package to ensure the accuracy of the recognition result; the preset continuity constraints are a set of rules or thresholds used to determine whether a character sequence conforms to the expected spatial and logical relationships, such as character spacing, order, etc.; a valid text segment is a candidate text segment that meets the preset continuity constraints after verification.

[0086] First, the system performs character space continuity verification based on surface deformation compensation on the set of candidate text segments, adjusts the spatial positions of each character according to the actual surface curvature of the drug package, and verifies whether these characters conform to the preset continuity constraints. For cases where characters are deformed due to package bending, the true position and form of the characters are restored through a deformation compensation algorithm and then verified. Next, valid text segments that meet the preset continuity constraints are screened out.

[0087] For example, if the character spacing in a certain candidate text segment is too large or too small, or the character order does not meet the expectation, then this text segment will be excluded. Only those valid text segments with character spacing and order both meeting the expectation will be retained. Finally, these valid text segments are merged in the spatial arrangement order of the drug packages on the production line to generate a supervision code text sequence uniquely associated with each drug package.

[0088] During the drug production process, it is very important to ensure that the supervision codes on each drug package can be accurately recognized and associated. Due to the geometric deformation of the drug package surface (such as bending, twisting, etc.) and the spatial arrangement changes caused by the conveyor belt movement, traditional character recognition methods may not be able to guarantee high-precision recognition results. Based on this, as another embodiment, according to Step 204, perform character space continuity verification based on surface deformation compensation on the set of candidate text segments, screen out valid text segments that meet the preset continuity constraints, and merge the valid text segments in the spatial arrangement order of the drug packages on the production line to generate a supervision code text sequence uniquely associated with each drug package, including:

[0089] Step 301: Calculate the curvature matching index of each character in the set of candidate text segments based on the character topological relationship after surface deformation compensation;

[0090] In this step, the topological relationship of characters after surface deformation compensation refers to the spatial arrangement and connection mode between characters restored after compensating for the geometric deformation of the drug packaging surface; the curvature matching index is a numerical index used to measure the similarity between the shape of each character and the ideal shape, which reflects the accuracy of the character shape after deformation compensation.

[0091] In this embodiment, first, according to the topological relationship of characters after surface deformation compensation, analyze the morphological characteristics of each character, conduct a detailed analysis of the edge contour, bending degree, etc. of each character, and calculate its curvature value. Among them, the curvature value can be used to describe the bending degree of the character curve. A higher curvature value indicates a greater bend, and a lower curvature value indicates a smaller bend or a straight line. Next, calculate the curvature matching index for each character based on these curvature values. This index is usually a comprehensive score reflecting the similarity between the character shape and the ideal shape. Among them, in order to calculate the curvature matching index, the system will compare the actual curvature value of each character with the curvature value of its ideal shape and calculate the difference between the two. The smaller the difference, the higher the curvature matching index, indicating that the character shape is closer to the ideal shape.

[0092] Step 302, dynamically adjust the character spacing threshold range in the continuity constraint according to the statistical characteristics of the decoded drug supervision code characters in the same drug production batch to obtain the adjusted character spacing threshold range;

[0093] In this step, the statistical characteristics of character distribution refer to the statistical analysis of the characters of the successfully decoded drug supervision code, and extract the characteristic information such as the spacing and order between characters; the character spacing threshold range is an important parameter in the continuity constraint, which defines the maximum and minimum distances allowed between characters.

[0094] In this embodiment, first, conduct a statistical analysis of the successfully decoded drug supervision codes in the same drug production batch and extract the statistical characteristics of the character distribution. Specifically, the system will calculate statistical quantities such as the average spacing and standard deviation between each character, and dynamically adjust the character spacing threshold range in the continuity constraint based on these statistical data. Next, use the adjusted character spacing threshold range to re-verify whether the character spacing in the candidate text segment set meets the requirements. For those text segments whose character spacing exceeds the new threshold range, they will be excluded, and only the valid text segments that meet the new threshold range will be retained.

[0095] Step 303, mark the candidate text segments that meet the condition that the curvature matching index exceeds the upper limit of the surface deformation compensation error and the character spacing is within the adjusted character spacing threshold range as valid text segments, and record the spatial position mapping table of the valid text segments in the coordinate system of the pipeline conveyor belt;

[0096] In this step, the curvature matching index is a numerical index used to measure the similarity between each character form and the ideal form, which reflects the accuracy of the character shape after deformation compensation; the upper limit of the surface deformation compensation error refers to the maximum deformation error range allowed by the system, and character forms outside this range are considered unreliable; the spatial position mapping table is a data structure that records the specific positions of each valid text segment in the coordinate system of the pipeline conveyor belt, and is used for subsequent merging and tracking management.

[0097] In this embodiment, first, according to the previously calculated curvature matching index and the adjusted character spacing threshold range, candidate text segments that meet the conditions are screened out, and it is checked whether the characters in each candidate text segment meet the following two conditions (the curvature matching index exceeds the upper limit of the surface deformation compensation error and the character spacing is within the adjusted character spacing threshold range). Only the text segments that meet both conditions at the same time will be marked as valid text segments. Next, record the spatial positions of these valid text segments in the coordinate system of the pipeline conveyor belt, mark the specific positions of each drug package on the conveyor belt of the pipeline, and associate this position information with the valid text segments to generate a spatial position mapping table, which can be used for text segment merging and the traceability management of the entire system in subsequent steps.

[0098] Step 304, according to the spatial position mapping table, splice the valid text segments belonging to the same drug package based on the movement direction of the conveyor belt to obtain the spliced character sequence.

[0099] In this step, the movement direction of the conveyor belt refers to the direction in which the drug package moves on the pipeline conveyor belt, usually in the X-axis or Y-axis direction; character sequence splicing is to merge the valid text segments belonging to the same drug package in the order of their arrangement on the conveyor belt.

[0100] In this embodiment, first, according to the coordinate information in the spatial position mapping table, determine the drug package to which each valid text segment belongs, classify each valid text segment to the corresponding drug package according to its spatial position (such as X-axis and Y-axis coordinates), and then sort these valid text segments according to the movement direction of the conveyor belt. Sort the valid text segments according to this order to ensure that they are spliced together in the correct order. Finally, splice the sorted valid text segments to generate a complete supervision code text sequence. Among them, all the valid text segments belonging to the same drug package will be spliced one by one in the order of their arrangement on the conveyor belt to form a continuous character sequence.

[0101] Step 305: Eliminate the text segment conflicts of different drug packages according to the monotonically increasing order of the character position confidence levels in the spliced character sequence, and generate a supervision code text sequence uniquely associated with each drug package;

[0102] In this step, the character position confidence level is an evaluation value of the position and correctness of each character in its image, used to measure the reliability of the decoding result; the monotonically increasing order refers to sorting in ascending or descending order according to the character position confidence level; the text segment conflict means that since the character sequences on multiple drug packages may partially overlap or be confused, the system needs further processing to ensure that the supervision code information of each drug package is unique and accurate; the uniquely associated supervision code text sequence refers to the final generated, complete and unique supervision code text sequence corresponding to each drug package.

[0103] In this embodiment, first calculate the position confidence level of each character in the spliced character sequence. For example, if a character is recognized very clearly and unambiguously, its position confidence level will be relatively high; on the contrary, if a character is blurred or occluded, its position confidence level will be relatively low. Next, sort the characters in the monotonically increasing order of the character position confidence level, and select the order from high confidence level to low confidence level for sorting, so as to give priority to processing the most reliable characters, which can minimize the possibility of incorrect matching and conflicts. Then, gradually eliminate the text segment conflicts between different drug packages according to the sorted character position confidence levels, check the position confidence levels of each character one by one, and judge which characters should belong to which drug package according to these confidence scores. If there are some overlapping or similar characters on two adjacent drug packages, it will be determined which package these characters should belong to according to the high or low character position confidence level. Finally, splice the character sequence after conflict elimination again to generate a supervision code text sequence uniquely associated with each drug package. Among them, each drug package will have a complete and unique supervision code text sequence to ensure that each drug package can be accurately traced and managed throughout the production process.

[0104] During the drug package production process, due to the time and space differences in the images taken at different perspectives during the movement of the conveyor belt, the characters in the original image set may be misaligned or deformed. To solve this problem, in some embodiments, according to step 102, input the original image set into a preset neural network model, and use the neural network model to generate multi-scale supervision code feature data corresponding to a single drug package through cross-perspective image matching, including:

[0105] Step 401: Perform temporal alignment on the original image set based on the movement speed of the conveyor belt in the pipeline to generate a multi-perspective image frame sequence;

[0106] In this step, the original image set refers to the set of images of the drug packaging surface captured by multiple cameras or sensors within the same time period. These images may be from different perspectives, and due to the movement of the conveyor belt, there may be a time offset between the images captured at different time points; temporal alignment means adjusting the timestamps of image acquisition according to the movement speed of the conveyor belt to ensure that all images are arranged and synchronized in the correct order for subsequent processing; the multi-view image frame sequence is a sequence composed of image frames from multiple perspectives after temporal alignment, and these image frames can be used for further feature extraction and analysis.

[0107] In this embodiment, first, the original image set is obtained and the timestamp of each image is recorded. Then, the time interval between each image is calculated according to the movement speed of the pipeline conveyor belt. Next, the original image set is temporally aligned, and finally, a multi-view image frame sequence is generated. Among them, each image frame contains the drug packaging surface information from different perspectives, and this information can be used for subsequent feature point matching and geometric deformation field construction.

[0108] Step 402: Perform cross-view feature point matching on each drug packaging body in the multi-view image frame sequence through a neural network model to generate a cross-view region correlation matrix reflecting the spatial distribution of supervision code characters between adjacent views;

[0109] In this step, cross-view feature point matching means finding common feature points (such as key points of supervision code characters) between image frames from different perspectives to establish a correspondence between them; the cross-view region correlation matrix is a data structure representing the similarity and correspondence of the spatial distribution of supervision code characters between adjacent views. It reflects the spatial positions and morphological differences of supervision code characters from different perspectives and helps with subsequent geometric deformation compensation and feature extraction.

[0110] In this embodiment, first, the multi-view image frame sequence is input into a pre-trained neural network model. This model can automatically detect and extract feature points in each image frame, and perform matching based on the corresponding relationships of these feature points under different views. For each drug package, common feature points are searched for in the corresponding multi-view image frames, and the similarity and corresponding relationships between these feature points are calculated. For example, assume that the same regulatory code character '1' is included in the image frames of a certain drug package under two views. The feature points of the character '1' are found under these two views, and the similarity score between them is calculated. Finally, a cross-view region correlation matrix reflecting the spatial distribution of regulatory code characters between adjacent views is generated. This matrix records the spatial positions and morphological differences of regulatory code characters under different views, as well as their corresponding relationships. For example, if the positions of the character '1' under two views are (10, 20) and (12, 22) respectively, the corresponding relationship and its similarity score of these two positions are recorded in the cross-view region correlation matrix.

[0111] Step 403: According to the cross-view region correlation matrix, construct a geometric deformation field associated with the geometric deformation parameters of the drug package surface, and use the geometric deformation field to map the multi-view image frame sequence to a unified reference coordinate system to generate multi-scale regulatory code feature data;

[0112] In this step, the geometric deformation field is a mathematical model describing the geometric deformation of the drug package surface. It estimates the actual morphological changes of the package surface by analyzing the corresponding relationships of feature points under different views; the geometric deformation parameters refer to the specific numerical values used to describe the geometric deformation of the drug package surface, such as the bending degree, twisting angle, etc.; the unified reference coordinate system is a standard spatial coordinate system used to map image frames from different views into the same coordinate system for subsequent processing and analysis; the multi-scale regulatory code feature data is the regulatory code feature information at different scales extracted from the mapped image frames, and these features can more comprehensively describe the regulatory code characters and the surrounding background information.

[0113] In this embodiment, first, according to the cross-view region correlation matrix, a geometric deformation field associated with the geometric deformation parameters of the drug packaging curved surface is constructed. The corresponding relationship of feature points of each drug packaging body under different views is analyzed, and the corresponding geometric deformation parameters (such as bending degree, twisting angle, etc.) are calculated. For example, assume that the positions of the feature points of a certain drug packaging body under two views are (10, 20) and (12, 22) respectively. According to the position differences, the bending degree of the packaging surface is calculated and expressed as a geometric deformation parameter. Next, the multi-view image frame sequence is mapped to a unified reference coordinate system using the geometric deformation field. For each image frame, the deformation parameter in the geometric deformation field is used to perform deformation compensation on it to restore it to the shape in the ideal state. For example, if the regulatory code characters in a certain image frame are deformed due to the bending of the packaging surface, an inverse transformation will be performed on them according to the calculated geometric deformation parameters to restore their original shape. Then, all the image frames after deformation compensation are mapped to a unified reference coordinate system to ensure that they are aligned in the same spatial coordinate system. Finally, the key feature information of the regulatory code characters and their surrounding backgrounds, such as edges, textures, shapes, etc., will be extracted at different scales to generate multi-scale regulatory code feature data.

[0114] When dealing with the geometric deformation problem of the drug packaging surface, relying solely on preliminary feature point matching may not be sufficient to fully correct all deformations. To more precisely compensate for the deformations, it is necessary to further analyze the spatial distribution characteristics in the cross-view region correlation matrix, calculate the dynamic deformation offset, and construct a geometric deformation field covering the entire domain of the drug packaging curved surface. Based on this, as another embodiment, according to step 403, a geometric deformation field associated with the geometric deformation parameters of the drug packaging curved surface is constructed according to the cross-view region correlation matrix, and the multi-view image frame sequence is mapped to a unified reference coordinate system using the geometric deformation field to generate multi-scale regulatory code feature data, including:

[0115] Step 501, calculate a dynamic deformation offset that is positively correlated with the curvature gradient of the drug packaging curved surface based on the spatial distribution characteristics of the drug regulatory code characters between adjacent views in the cross-view region correlation matrix;

[0116] In this step, the curvature gradient refers to the speed and direction of the curvature change at a specific point. For the drug packaging curved surface, it describes the change in the bending degree of the packaging surface, that is, the difference in the bending degree at different positions; the dynamic deformation offset refers to the adjustment value calculated to compensate for the shape change of the drug packaging surface. This offset is calculated based on the curvature gradient of the packaging surface and dynamically adjusts as the shape of the curved surface changes.

[0117] In this embodiment, first, the spatial distribution characteristics of drug supervision code characters between adjacent perspectives recorded in the cross-perspective regional correlation matrix are analyzed. By comparing these spatial distribution characteristics, the bending pattern of the drug packaging surface and its influence on the relative positions of supervision code characters can be identified. If the position of the supervision code characters observed from one perspective has a displacement relative to another perspective, this may be caused by the bending of the packaging surface. Next, based on the above analysis results, the corresponding deformation offset amount is determined according to the curvature gradient at each position. This means that in regions with a larger curvature (such as the corners of the packaging bag), the deformation offset amount will be larger; while in relatively flat regions, the deformation offset amount will be smaller. Finally, by applying these dynamic deformation offset amounts, the position error of the supervision code characters caused by the geometric deformation of the packaging surface is effectively corrected.

[0118] Step 502: Generate a geometric deformation field covering the entire drug packaging surface according to the dynamic deformation offset amount and the surface curvature gradient.

[0119] In this embodiment, the dynamic deformation offset amount of each local region is combined with its corresponding curvature gradient to form a continuous and smooth deformation distribution model. For example, in regions with a larger curvature (such as the edges of the packaging bag), a larger deformation offset value will be assigned; while in relatively flat regions (such as the center of the packaging bag), a smaller deformation offset value will be assigned. Next, interpolation or fitting algorithms are used to extend the deformation information of the local regions to the entire drug packaging surface, thereby generating a complete geometric deformation field. The finally generated geometric deformation field can comprehensively describe the deformation conditions of each position on the drug packaging surface. Among them, this deformation field not only contains the deformation information of each local region but also can reflect the geometric characteristics of the overall surface, providing an important reference basis for subsequent image correction and feature extraction.

[0120] Step 503: Input the spatial coordinates of each pixel point in the multi-perspective image frame sequence into the geometric deformation field, and use the curvature gradient matrix in the geometric deformation field to perform affine transformation compensation on the spatial coordinates of each pixel point to obtain the transformed spatial coordinates.

[0121] In this step, the curvature gradient matrix is a mathematical matrix used to describe the bending degree and its change of the drug packaging surface at different positions. This matrix combines the global information of the geometric deformation field and the local curvature gradient characteristics, and can provide an accurate basis for deformation compensation for the spatial coordinates of each pixel point. Affine transformation compensation is a geometric transformation method that corrects the coordinate offset caused by surface deformation by performing linear transformations (such as translation, rotation, scaling, or shear) on the spatial coordinates of each pixel point. This method can effectively handle complex geometric deformation problems and make the image data closer to the ideal state.

[0122] In this embodiment, first, the original spatial coordinates of each pixel point (such as the X-axis and Y-axis positions) are read one by one, and the deformation compensation value of the pixel point is calculated according to the curvature gradient matrix in the geometric deformation field. For example, for a certain pixel point, if the curvature of the area where it is located is large (such as the edge of the packaging bag), a larger compensation value will be assigned to it according to the curvature gradient matrix; while for a relatively flat area (such as the center of the packaging bag), a smaller compensation value will be assigned. Next, the deformation offset of each pixel point is calculated according to the curvature gradient matrix and adjusted through the affine transformation formula. For example, assume that the original spatial coordinates of a certain pixel point are (10, 20), and it is calculated according to the curvature gradient matrix that it needs to move 2 units in the X-axis direction and 1 unit in the Y-axis direction. After affine transformation, the new coordinates of this pixel point become (12, 21). This affine transformation not only considers the position offset of the pixel point but may also involve operations such as rotation or scaling to further correct complex geometric deformations, and finally obtains the set of transformed spatial coordinates.

[0123] Step 504, use the dynamic deformation offset field to perform secondary correction on the transformed spatial coordinates to generate multi-scale supervision code feature data;

[0124] In this step, the geometric deformation parameters refer to the specific values used to describe the geometric deformation of the drug packaging surface, such as the degree of bending, the angle of twisting, etc.; the unified reference coordinate system is a standard spatial coordinate system used to map image frames from different perspectives to the same coordinate system for subsequent processing and analysis.

[0125] In this embodiment, first, the corresponding relationship of feature points of each drug packaging body under different perspectives is analyzed, and the corresponding geometric deformation parameters (such as the degree of bending, the angle of twisting, etc.) are calculated. For example, assume that the positions of the feature points of a certain drug packaging body under two perspectives are (10, 20) and (12, 22) respectively. The degree of bending of the packaging surface will be calculated according to these position differences and expressed as geometric deformation parameters. Next, for each image frame, the deformation compensation is performed on it using the deformation parameters in the geometric deformation field to restore it to the ideal state. For example, if the supervision code characters in a certain image frame are deformed due to the bending of the packaging surface, the inverse transformation will be performed on it according to the calculated geometric deformation parameters to restore its original form. Then, all the image frames after deformation compensation are mapped to a unified reference coordinate system to ensure that they are aligned in the same spatial coordinate system. Finally, multi-scale supervision code feature data is extracted from the mapped image frames.

[0126] In a pharmaceutical production batch, the character distribution in the supervision code text sequence may exhibit certain randomness and complexity. To ensure that the supervision code of each pharmaceutical package can be accurately identified and associated, it is necessary to dynamically construct a batch character entropy value baseline and compare it with the real-time decoded supervision code text sequence to detect abnormal character distribution patterns. Based on this, in some embodiments, as described in step 104, the supervision code text sequence is compared with a preset pharmaceutical production batch number database in real time to generate a batch recognition result map, including:

[0127] Step 601: Dynamically construct a batch character entropy value baseline associated with the change in the speed of the conveyor belt of the assembly line and the spatial arrangement density of the pharmaceutical packages based on the character distribution characteristics of the decoded pharmaceutical supervision code text sequence in the same pharmaceutical production batch;

[0128] In this step, the batch character entropy value baseline is a reference value reflecting the randomness and complexity of the character distribution in the pharmaceutical supervision code of a specific production batch. It combines the change in the conveyor belt speed and the density of the arrangement of the pharmaceutical packages on the conveyor belt and is used to evaluate the overall information entropy of the pharmaceutical supervision code of this batch. The higher the character entropy value baseline, the more random the character distribution; conversely, it indicates that the character distribution is more regular or orderly.

[0129] In this embodiment, first, information such as the frequency of each character appearance, the interval between characters, and the patterns in the character sequence is statistically analyzed. For example, in a batch, it may be found that some characters appear frequently while others appear less frequently, or certain character sequences exhibit a certain degree of repetition. Next, the weight of the character distribution characteristics is adjusted according to the change in the conveyor belt speed because a higher conveyor belt speed may lead to more recognition errors and a more random character distribution; at the same time, the spatial arrangement density of the pharmaceutical packages is considered. Densely arranged packages may increase the possibility of mutual interference between characters, thereby affecting the character entropy value. For example, if the conveyor belt speed is fast and the packages are arranged densely, a higher character entropy value baseline will be calculated; on the contrary, if the conveyor belt speed is slow and the packages are arranged sparsely, a lower character entropy value baseline will be calculated.

[0130] Step 602: Perform spatio-temporal alignment comparison between the decoding position confidence of each character in the supervision code text sequence and the batch character entropy value baseline to generate an entropy value mutation coefficient matrix reflecting abnormal character distribution patterns;

[0131] In this step, the entropy value mutation coefficient matrix is a data structure used to represent the degree of difference between the decoding position confidence of each character and the batch character entropy value baseline in a specific production batch. Each element in this matrix represents the entropy value mutation coefficient of a character position, and the larger the value, the more the character distribution at this position deviates from the normal pattern and may be abnormal.

[0132] In this embodiment, first, according to the timestamp and spatial coordinate information, the position confidence of each character is matched with its corresponding entropy value baseline. For example, assume that the decoding position confidence of a certain character is 0.95, while its corresponding entropy value baseline is 0.85. The difference between the two will be calculated. Next, based on these differences, an entropy value mutation coefficient is calculated, and an entropy value mutation coefficient matrix is generated. For each character position, the difference between its decoding position confidence and the entropy value baseline is calculated and converted into a mutation coefficient. For example, if the decoding confidence of a certain character position is significantly lower than the expected entropy value baseline, the entropy value mutation coefficient at this position will be higher, reflecting a possible abnormal character distribution pattern.

[0133] Step 603: According to the entropy value mutation coefficient matrix and the spatial position mapping table of the drug package in the coordinate system of the pipeline conveyor belt, perform edge position backtracking and positioning on the drug supervision code exceeding the preset entropy value mutation threshold, and generate an abnormal marking data cluster.

[0134] In this step, the abnormal marking data cluster is a set of data containing the abnormal drug supervision code and its detailed information. These information include the specific position of the abnormal character, the time point when it appears, and the corresponding spatial position on the conveyor belt.

[0135] In this embodiment, first, an entropy value mutation threshold (such as 0.1) is set, and all character positions in the entropy value mutation coefficient matrix exceeding this threshold are marked. For example, if the entropy value mutation coefficient of a certain character position is 0.15, exceeding the preset threshold, this position is marked as abnormal. Next, edge position backtracking and positioning are performed on these abnormal characters to determine their specific physical positions. Using the spatial position mapping table, the specific coordinates of each abnormal character on the conveyor belt are found, and their timestamps are recorded. For example, assume that an abnormal character is located at the X-axis position of 200 mm and the Y-axis position of 50 mm on the conveyor belt and appears at the time point t = 0.5 s. These information are recorded to generate an abnormal marking data cluster, which contains the relevant information of all abnormal characters.

[0136] Step 604: Weightedly superimpose the abnormal marking data cluster and the surface deformation compensation parameter in the multi-scale supervision code feature data to generate a batch recognition result map.

[0137] In this embodiment, first, corresponding surface deformation compensation parameters are applied to each abnormal character position to correct the error caused by the bending or distortion of the packaging surface. For example, assume that the position of a certain abnormal character is offset by 2 pixels due to the bending of the packaging surface. The system will adjust it according to the surface deformation compensation parameters. Next, the corrected abnormal marker data clusters are combined with the multi-scale supervision code feature data to generate a batch recognition result map. This batch recognition result map not only shows the specific positions and time points of the abnormal characters, but also demonstrates the geometric deformation conditions of each drug package in the entire production batch. For example, color coding is used to represent the recognition accuracy and deformation degree at different positions, with red indicating high-risk areas and green indicating low-risk areas.

[0138] During the drug production process, the supervision codes on some drug packages may show abnormal character distribution patterns. In order to accurately locate these abnormalities and conduct effective quality control, it is necessary to construct an abnormal area diffusion model and perform point-by-point calibration and spatial matching in combination with the multi-scale supervision code feature data. Based on this, as another embodiment, according to step 601, the supervision code text sequence is compared with a preset drug production batch number database in real time to generate a batch recognition result map, including:

[0139] Step 701, based on the entropy value mutation amount of each abnormal character and the gradient change trend of adjacent characters in the entropy value mutation coefficient matrix, construct an abnormal area diffusion model that is positively correlated with the curvature gradient of the drug packaging surface;

[0140] In this step, the abnormal area diffusion model is a mathematical model used to describe the expansion of abnormal characters and their surrounding areas on the drug packaging surface.

[0141] In this embodiment, first, calculate the difference degree between each abnormal character and its adjacent characters (such as the position confidence or the change in character form), and combine this information to evaluate the expansion possibility of the abnormal area. For example, if the entropy value mutation amount of a certain character is relatively high and its adjacent characters also show a large gradient change, then this area may have a relatively high diffusion risk. Next, construct an abnormal area diffusion model based on the above analysis results.

[0142] Step 702, use the abnormal area diffusion model to locate the initial physical coordinate set of the abnormal supervision code in the coordinate system of the pipeline conveyor belt, and perform point-by-point calibration on the initial physical coordinate set according to the reflectivity difference correction amount between the edges of adjacent drug packages in the multi-scale supervision code feature data to generate a cluster of abnormal physical coordinates after reflectivity correction;

[0143] In this step, the cluster of abnormal physical coordinates after reflectivity correction is a set of specific physical coordinates of abnormal characters that have been corrected for reflectivity differences.

[0144] In this embodiment, first, the diffusion range of each abnormal character is predicted according to the abnormal area diffusion model, and its initial physical coordinates on the conveyor belt are determined. For example, assume that an abnormal character is located at the position of 200 mm on the X-axis and 50 mm on the Y-axis of the conveyor belt, and the system will record these initial coordinates to generate an initial physical coordinate set. Next, the initial physical coordinate set is calibrated point by point according to the reflectivity difference correction amount between the edges of adjacent drug packages in the multi-scale supervision code feature data. For each initial physical coordinate, it is adjusted according to the reflectivity difference correction amount around it, and finally, an abnormal physical coordinate cluster after reflectivity correction is obtained.

[0145] Step 703: Match the abnormal physical coordinate cluster after reflectivity correction with the spatial arrangement order of the corresponding drug packages in the spatial position mapping table, screen out an abnormal coordinate subset that meets the preset spatial continuity constraint, and associate the corresponding character distribution abnormal level in the entropy mutation coefficient matrix;

[0146] The abnormal coordinate subset is a specific physical coordinate set of a group of abnormal characters that meet the preset spatial continuity constraint after screening.

[0147] In this embodiment, first, according to the physical coordinates of each abnormal character, its corresponding drug package in the spatial position mapping table is searched, and it is checked whether these coordinates meet the preset spatial continuity constraint. For example, assume that the coordinates of an abnormal character are located at the position of 200 mm on the X-axis and 50 mm on the Y-axis of the conveyor belt, and it is checked whether this position is consistent with the spatial arrangement order of the corresponding drug package. Next, an abnormal coordinate subset that meets the preset spatial continuity constraint is screened out. For each abnormal character, it is evaluated whether its physical coordinates meet the spatial continuity requirement. If the coordinates of an abnormal character deviate too much from the expected position, then this coordinate will not be included in the abnormal coordinate subset. In addition, the corresponding character distribution abnormal level in the entropy mutation coefficient matrix is also associated to add additional abnormal information to each abnormal coordinate subset.

[0148] Step 704: Generate an abnormal marking data cluster including the deformation compensation residual amount, the reflectivity difference correction amount, and the entropy mutation amount according to the abnormal coordinate subset and the associated character distribution abnormal level;

[0149] In this embodiment, first, a comprehensive analysis is performed on the characters in each abnormal coordinate subset to extract the deformation compensation residual amount (such as the error due to incomplete correction of the packaging surface deformation), the reflectivity difference correction amount (such as the coordinate offset caused by the reflectivity difference of the packaging material), and the entropy value mutation amount (such as the difference between the character position confidence and the batch character entropy value baseline). Next, for each abnormal character, the system records its physical coordinates, deformation compensation residual amount, reflectivity difference correction amount, and entropy value mutation amount, and associates them with the abnormal level of character distribution. For example, assume that the deformation compensation residual amount of an abnormal character is 0.5 pixels, the reflectivity difference correction amount is 1 pixel, the entropy value mutation amount is 0.15, and its abnormal level of character distribution is high. Integrating these information, an abnormal marking data cluster is obtained.

[0150] Figure 2 FIG. shows a schematic structural diagram of a batch identification system for drug supervision codes based on a neural network provided by an embodiment of the present application. Figure 2 As shown, the system includes:

[0151] An acquisition module 21, which acquires a set of original images of the drug packaging surface containing densely arranged drug supervision codes.

[0152] An input module 22, which inputs the set of original images into a preset neural network model, and uses the neural network model to generate multi-scale supervision code feature data corresponding to a single drug packaging body through cross-view image matching.

[0153] A decoding module 23, which parallelly decodes the drug supervision code regions with overlapping and occluded multi-scale supervision code feature data to generate a supervision code text sequence uniquely associated with each drug packaging body.

[0154] A comparison module 24, which is used to compare the supervision code text sequence with a preset drug production batch number database in real time to generate a batch identification result map.

[0155] Figure 2 The batch identification system for drug supervision codes based on a neural network can execute Figure 1 The batch identification method for drug supervision codes based on a neural network described in the embodiment shown. The implementation principle and technical effects will not be elaborated. For the batch identification system for drug supervision codes based on a neural network in the above embodiment, the specific ways for each module and unit to perform operations have been described in detail in the embodiment related to the method, and will not be elaborated here.

[0156] In a possible design, Figure 2 The batch identification system for drug supervision codes based on a neural network in the embodiment shown can be implemented as a computing device, such as Figure 3As shown, the computing device may include a storage component 31 and a processing component 32;

[0157] The storage component 31 stores one or more computer instructions, and among them, the one or more computer instructions are for the processing component 32 to call and execute.

[0158] The processing component 32 is used to collect an original image set of drug supervision codes densely arranged on the surface of a drug package; input the original image set into a preset neural network model, and use the neural network model to generate multi-scale supervision code feature data corresponding to a single drug package body through cross-view image matching; perform parallel decoding on the drug supervision code areas where the multi-scale supervision code feature data has overlap and occlusion, and generate a supervision code text sequence uniquely associated with each drug package body; compare the supervision code text sequence with a preset drug production batch number database in real time to generate a batch identification result map.

[0159] Among them, the processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for executing the above method.

[0160] The storage component 31 is configured to store various types of data to support the operation of the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0161] Of course, the computing device may also necessarily include other components, such as an input / output interface, a display component, a communication component, etc.

[0162] The input / output interface provides an interface between the processing component and the peripheral interface module, and the above peripheral interface module may be an output device, an input device, etc.

[0163] The communication component is configured to facilitate communication between the computing device and other devices in a wired or wireless manner, etc.

[0164] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server. The above processing components, storage components, etc. can be basic server resources rented or purchased from a cloud computing platform.

[0165] The embodiment of the present application also provides a computer storage medium storing a computer program, and when the computer program is executed by a computer, it can implement the above Figure 1 A method for batch recognition of drug supervision codes based on a neural network in the shown embodiment.

[0166] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0168] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A method for batch recognition of drug regulatory codes based on neural networks, characterized in that: include: Collecting a set of original images containing densely arranged drug regulatory codes on the surface of drug packaging; Inputting the original image set into a preset neural network model, and using the neural network model to generate multi-scale supervision code feature data corresponding to a single drug package through cross-view image matching; Parallel decoding is performed on the drug supervision code regions where the multi-scale supervision code feature data overlap and are blocked, to generate a supervision code text sequence uniquely associated with each drug package; The regulatory code text sequence is compared with a preset drug production batch number database in real time to generate a batch recognition result map.

2. The method according to claim 1, characterized in that Parallel decoding is performed on the drug supervision code regions where the multi-scale supervision code feature data overlaps and is blocked, to generate a supervision code text sequence uniquely associated with each drug package, including: Based on the character topological relationship after the deformation compensation of the drug packaging surface in the multi-scale regulatory code feature data, a dynamic attention mask weight matrix corresponding to the overlapping and occluded areas is generated; The dynamic attention mask weight matrix is ​​fused with the character topological relationship after surface deformation compensation in the multi-scale supervision code feature data channel by channel to obtain a fusion result, and a multi-level feature vector group in each overlapping and occluded area in the fusion result is extracted; According to the multi-level feature vector group, bidirectional context-associative decoding is performed on each drug regulatory code character in the overlapping and occluded area by a parallel decoder to generate a candidate text segment set including character position confidences; The character space continuity verification based on surface deformation compensation is performed on the candidate text segment set, and valid text segments that meet the preset continuity constraints are screened. The valid text segments are merged according to the spatial arrangement order of the drug packages on the assembly line to generate a supervision code text sequence uniquely associated with each drug package.

3. The method according to claim 2, characterized in that The character space continuity verification based on surface deformation compensation is performed on the candidate text segment set, valid text segments that meet the preset continuity constraints are screened, and the valid text segments are merged according to the spatial arrangement order of the drug packaging bodies on the assembly line to generate a regulatory code text sequence uniquely associated with each drug packaging body, including: Calculating the curvature matching index of each character in the candidate text segment set based on the character topological relationship after surface deformation compensation; Dynamically adjust the character spacing threshold range in the continuity constraint according to the character distribution statistical characteristics of the decoded drug regulatory code in the same drug production batch to obtain an adjusted character spacing threshold range; Marking the candidate text segments that satisfy the curvature matching index exceeding the upper limit of the surface deformation compensation error and whose character spacing is within the adjusted character spacing threshold range as valid text segments, and recording the spatial position mapping table of the valid text segments in the assembly line conveyor belt coordinate system; According to the spatial position mapping table, the valid text segments belonging to the same drug package are spliced ​​in a character sequence based on the conveyor belt movement direction to obtain a spliced ​​character sequence; The text segment conflicts of different drug packages are eliminated according to the monotonically increasing order of the character position confidences in the spliced ​​character sequence, and a supervision code text sequence uniquely associated with each drug package is generated.

4. The method according to claim 1, characterized in that: The original image set is input into a preset neural network model, and the neural network model is used to generate multi-scale supervision code feature data corresponding to a single drug package through cross-view image matching, including: Performing time sequence alignment on the original image set based on the moving speed of the pipeline conveyor belt to generate a multi-view image frame sequence; Performing cross-view feature point matching on each drug package in the multi-view image frame sequence through a neural network model to generate a cross-view regional correlation matrix reflecting the spatial distribution of regulatory code characters between adjacent views; According to the cross-viewing area correlation matrix, a geometric deformation field associated with the geometric deformation parameters of the drug packaging surface is constructed, and the multi-viewing image frame sequence is mapped to a unified reference coordinate system using the geometric deformation field to generate multi-scale supervision code feature data.

5. The method according to claim 4, characterized in that According to the cross-view area correlation matrix, a geometric deformation field associated with the geometric deformation parameters of the drug packaging surface is constructed, and the multi-view image frame sequence is mapped to a unified reference coordinate system using the geometric deformation field to generate multi-scale supervision code feature data, including: Based on the spatial distribution characteristics of the drug regulatory code characters between adjacent viewing angles in the cross-viewing area correlation matrix, a dynamic deformation offset positively correlated with the curvature gradient of the drug packaging surface is calculated; Generate a geometric deformation field covering the entire surface of the drug packaging according to the dynamic deformation offset and the surface curvature gradient; Inputting the spatial coordinates of each pixel point in the multi-view image frame sequence into the geometric deformation field, and performing affine transformation compensation on the spatial coordinates of each pixel point using the curvature gradient matrix in the geometric deformation field to obtain the transformed spatial coordinates; The transformed spatial coordinates are corrected twice using the dynamic deformation offset field to generate multi-scale supervision code feature data.

6. The method according to claim 1, characterized in that The regulatory code text sequence is compared with a preset drug production batch number database in real time to generate a batch recognition result map, including: Based on the character distribution characteristics of the decoded drug regulatory code text sequence in the same drug production batch, a batch character entropy value baseline associated with the speed change of the assembly line conveyor belt and the spatial arrangement density of the drug packaging body is dynamically constructed; Performing a spatiotemporal alignment comparison of the decoding position confidence of each character in the regulatory code text sequence with the batch character entropy value baseline to generate an entropy value mutation coefficient matrix reflecting the distribution pattern of abnormal characters; According to the entropy mutation coefficient matrix and the spatial position mapping table of the drug packaging body in the assembly line conveyor belt coordinate system, the edge position of the drug supervision code exceeding the preset entropy mutation threshold is traced back to generate an abnormal marking data cluster; The abnormal mark data cluster is weightedly superimposed with the surface deformation compensation parameters in the multi-scale supervision code feature data to generate a batch recognition result map.

7. The method according to claim 6, characterized in that Based on the character distribution characteristics of the decoded drug regulatory code text sequence in the same drug production batch, a batch character entropy value baseline associated with the change in the conveyor belt speed of the assembly line and the spatial arrangement density of the drug packaging is dynamically constructed, including: Based on the entropy mutation amount of each abnormal character in the entropy mutation coefficient matrix and the gradient change trend of adjacent characters, an abnormal area diffusion model positively correlated with the curvature gradient of the drug packaging surface is constructed; The abnormal region diffusion model is used to locate the initial physical coordinate set of the abnormal supervision code in the assembly line conveyor belt coordinate system, and the initial physical coordinate set is calibrated point by point according to the correction amount of the reflectivity difference of the edges of adjacent drug packaging bodies in the multi-scale supervision code feature data to generate an abnormal physical coordinate cluster after reflectivity correction; Matching the reflectivity-corrected abnormal physical coordinate cluster with the spatial arrangement order of the corresponding drug packaging body in the spatial position mapping table, screening out the abnormal coordinate subset that meets the preset spatial continuity constraint, and associating the corresponding character distribution abnormality level in the entropy mutation coefficient matrix; According to the abnormal coordinate subset and the associated character distribution abnormal level, an abnormal marking data cluster including a deformation compensation residual, a reflectivity difference correction amount and an entropy value mutation amount is generated.

8. A drug regulatory code batch recognition system based on neural network, characterized in that: include: The acquisition module collects the original image set of densely arranged drug regulatory codes on the drug packaging surface; An input module, inputting the original image set into a preset neural network model, and using the neural network model to generate multi-scale supervision code feature data corresponding to a single drug package through cross-view image matching; A decoding module, which decodes the overlapping and occluded drug regulatory code regions of the multi-scale regulatory code feature data in parallel, thereby generating a regulatory code text sequence uniquely associated with each drug package; The comparison module is used to compare the regulatory code text sequence with a preset drug production batch number database in real time to generate a batch recognition result map.

9. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a neural network-based drug regulatory code batch identification method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a method for batch identification of drug regulatory codes based on a neural network as described in any one of claims 1 to 7 is implemented.

Images