A method and system for batch identification of drug regulatory codes based on neural network

Abstract

The present application provides a method and system for batch recognition of drug regulatory codes based on a neural network, wherein the present application collects a set of original images of drug regulatory codes and inputs the original image set into a neural network model to generate multi-scale regulatory code feature data corresponding to a single drug package, performs parallel decoding on drug regulatory code areas where the multi-scale regulatory code feature data overlaps or is obscured, generates a regulatory code text sequence uniquely associated with each drug package, compares the regulatory code text sequence with a drug production batch number database, and generates a batch recognition result map, thereby improving the accuracy of drug regulatory code recognition, the ability to handle complex situations, the parsing speed, and the transparency and reliability of drug tracking management, greatly improving work efficiency and management level in large-scale applications.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of drug supervision technology, and in particular to a method and system for batch identification of drug supervision codes based on a neural network. Background Art

[0002] With the continuous development of modern pharmaceutical production and distribution management, ensuring the accurate source and destination of each drug has become crucial. Application scenarios require the ability to efficiently and accurately identify the regulatory codes on drug packaging to effectively track and monitor the entire drug process from production to sales. Especially in large-scale production and distribution environments, quickly and accurately reading and verifying the information on drug packaging plays an irreplaceable role in preventing counterfeit drugs from entering the market and protecting public health. As the variety and quantity of drugs continue to grow, the demand for batch identification of drug regulatory codes is increasing.

[0003] With the application of traditional technologies in the field of drug regulatory code recognition, existing solutions generally rely on traditional image processing technology and simple OCR methods. These solutions capture images of the drug packaging surface and use predefined template matching or feature extraction algorithms to attempt to decode the regulatory code. Although they can meet basic needs in some cases, their accuracy is highly dependent on factors such as shooting angle, lighting conditions, and the flatness of the packaging surface. In addition, when encountering overlap or partial occlusion between regulatory codes, it is often difficult to provide reliable decoding results, resulting in low recognition efficiency and increased error rate. Summary of the Invention

[0004] The embodiments of the present application provide a method and system for batch recognition of drug regulatory codes based on a neural network, which is used to solve the problems of low accuracy, low efficiency and difficulty in processing complex packaging surfaces (such as overlap and occlusion) in the prior art of drug regulatory code recognition.

[0005] In a first aspect, an embodiment of the present application provides a method for batch identification of drug regulatory codes based on a neural network, comprising:

[0006] Collecting a set of original images containing densely arranged drug regulatory codes on the surface of drug packaging;

[0007] Inputting the original image set into a preset neural network model, and using the neural network model to generate multi-scale supervision code feature data corresponding to a single drug package through cross-view image matching;

[0008] Parallel decoding is performed on the drug regulatory code regions where the multi-scale regulatory code feature data overlaps or is obscured, to generate a regulatory code text sequence uniquely associated with each drug package;

[0009] The regulatory code text sequence is compared with the preset drug production batch number database in real time to generate a batch recognition result map.

[0010] Optionally, parallel decoding is performed on the drug regulatory code regions where the multi-scale regulatory code feature data overlaps and is obscured, to generate a regulatory code text sequence uniquely associated with each drug package, including:

[0011] Based on the character topological relationship after compensation for the drug packaging surface deformation in the multi-scale regulatory code feature data, a dynamic attention mask weight matrix corresponding to the overlapping and occluded areas is generated;

[0012] The dynamic attention mask weight matrix is ​​fused with the character topology relationship after surface deformation compensation in the multi-scale supervision code feature data channel by channel to obtain a fusion result, and a multi-level feature vector group in each overlapping and occluded area in the fusion result is extracted;

[0013] Based on the multi-level feature vector group, a parallel decoder is used to perform bidirectional context-associative decoding on each drug regulatory code character in the overlapping and occluded area to generate a set of candidate text segments including character position confidences;

[0014] The set of candidate text segments is subjected to character space continuity verification based on surface deformation compensation, and valid text segments that meet preset continuity constraints are screened. The valid text segments are merged according to the spatial arrangement order of the drug packaging bodies on the assembly line to generate a regulatory code text sequence uniquely associated with each drug packaging body.

[0015] Optionally, character space continuity verification based on surface deformation compensation is performed on the candidate text segment set to screen valid text segments that meet preset continuity constraints, and the valid text segments are merged according to the spatial arrangement order of the drug packages on the assembly line to generate a regulatory code text sequence uniquely associated with each drug package, including:

[0016] Calculating a curvature matching index for each character in the candidate text segment set based on the character topological relationship after surface deformation compensation;

[0017] Dynamically adjusting the character spacing threshold range in the continuity constraint according to the character distribution statistical characteristics of the decoded drug regulatory codes in the same drug production batch to obtain an adjusted character spacing threshold range;

[0018] Marking candidate text segments that satisfy the curvature matching index exceeding the upper limit of the surface deformation compensation error and whose character spacing is within the adjusted character spacing threshold range as valid text segments, and recording a spatial position mapping table of the valid text segments in the assembly line conveyor coordinate system;

[0019] According to the spatial position mapping table, valid text segments belonging to the same drug package are spliced ​​in a character sequence based on the direction of conveyor belt movement to obtain a spliced ​​character sequence, and text segment conflicts between different drug packages are eliminated according to the monotonically increasing order of character position confidence in the spliced ​​character sequence to generate a regulatory code text sequence uniquely associated with each drug package.

[0020] Optionally, the original image set is input into a preset neural network model, and the neural network model is used to generate multi-scale regulatory code feature data corresponding to a single drug package through cross-view image matching, including:

[0021] Performing time sequence alignment on the original image set based on the movement speed of the pipeline conveyor belt to generate a multi-view image frame sequence;

[0022] Performing cross-view feature point matching on each drug package in the multi-view image frame sequence using a neural network model to generate a cross-view regional correlation matrix reflecting the spatial distribution of regulatory code characters between adjacent views;

[0023] According to the cross-view area correlation matrix, a geometric deformation field associated with the geometric deformation parameters of the drug packaging surface is constructed, and the multi-view image frame sequence is mapped to a unified reference coordinate system using the geometric deformation field to generate multi-scale supervision code feature data.

[0024] Optionally, a geometric deformation field associated with the geometric deformation parameters of the drug packaging surface is constructed based on the cross-view region correlation matrix, and the multi-view image frame sequence is mapped to a unified reference coordinate system using the geometric deformation field to generate multi-scale supervision code feature data, including:

[0025] Based on the spatial distribution characteristics of the drug regulatory code characters between adjacent viewing angles in the cross-view region correlation matrix, a dynamic deformation offset positively correlated with the curvature gradient of the drug packaging surface is calculated;

[0026] generating a geometric deformation field covering the entire surface of the pharmaceutical packaging according to the dynamic deformation offset and the surface curvature gradient;

[0027] Inputting the spatial coordinates of each pixel point in the multi-view image frame sequence into the geometric deformation field, and performing affine transformation compensation on the spatial coordinates of each pixel point using the curvature gradient matrix in the geometric deformation field to obtain transformed spatial coordinates;

[0028] The transformed spatial coordinates are corrected twice using the dynamic deformation offset field to generate multi-scale supervision code feature data.

[0029] Optionally, the regulatory code text sequence is compared with a preset drug production batch number database in real time to generate a batch recognition result map, including:

[0030] Based on the character distribution characteristics of the decoded drug regulatory code text sequence in the same drug production batch, a batch character entropy value baseline associated with the change in the conveyor belt speed and the spatial arrangement density of the drug packaging is dynamically constructed;

[0031] Performing a spatiotemporal alignment comparison of the decoding position confidence of each character in the regulatory code text sequence with the batch character entropy baseline to generate an entropy mutation coefficient matrix reflecting the distribution pattern of abnormal characters;

[0032] Based on the entropy mutation coefficient matrix and the spatial position mapping table of the drug packaging body in the assembly line conveyor coordinate system, the edge position of the drug supervision code exceeding the preset entropy mutation threshold is traced back to generate an abnormal mark data cluster;

[0033] The abnormal mark data cluster is weightedly superimposed with the surface deformation compensation parameters in the multi-scale supervision code feature data to generate a batch recognition result map.

[0034] Optionally, based on the character distribution characteristics of the decoded drug regulatory code text sequence in the same drug production batch, a batch character entropy value baseline associated with the change in the speed of the assembly line conveyor belt and the spatial arrangement density of the drug packaging is dynamically constructed, including:

[0035] Based on the entropy mutation coefficient matrix of each abnormal character and the gradient change trend of adjacent characters, an abnormal area diffusion model positively correlated with the curvature gradient of the drug packaging surface is constructed;

[0036] The abnormal region diffusion model is used to locate the initial physical coordinate set of the abnormal supervision code in the assembly line conveyor coordinate system, and the initial physical coordinate set is calibrated point by point based on the reflectivity difference correction value of the edges of adjacent drug packages in the multi-scale supervision code feature data to generate a reflectivity-corrected abnormal physical coordinate cluster;

[0037] Matching the reflectivity-corrected abnormal physical coordinate cluster with the spatial arrangement order of the corresponding drug packaging in the spatial position mapping table, screening out an abnormal coordinate subset that meets the preset spatial continuity constraint, and associating the corresponding character distribution abnormality level in the entropy mutation coefficient matrix;

[0038] According to the abnormal coordinate subset and the associated character distribution abnormality level, an abnormal mark data cluster including a deformation compensation residual, a reflectivity difference correction amount and an entropy value mutation amount is generated.

[0039] In a second aspect, an embodiment of the present application provides a drug regulatory code batch recognition system based on a neural network, comprising:

[0040] The acquisition module collects a set of raw images containing densely arranged drug regulatory codes on the surface of drug packaging;

[0041] An input module, which inputs the original image set into a preset neural network model and uses the neural network model to generate multi-scale supervision code feature data corresponding to a single drug package through cross-view image matching;

[0042] A decoding module, which decodes the overlapping and occluded drug regulatory code regions of the multi-scale regulatory code feature data in parallel to generate a regulatory code text sequence uniquely associated with each drug package;

[0043] The comparison module is used to compare the regulatory code text sequence with the preset drug production batch number database in real time to generate a batch recognition result map.

[0044] In a third aspect, an embodiment of the present application provides a computing device comprising a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a neural network-based drug regulatory code batch identification method as described in any one of the first aspects.

[0045] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, which, when executed by a processor, implements a neural network-based drug regulatory code batch identification method as described in any one of the first aspects.

[0046] In an embodiment of the present application, a set of original images containing densely arranged drug regulatory codes on the surface of drug packaging is collected; the set of original images is input into a preset neural network model, and the neural network model is used to generate multi-scale regulatory code feature data corresponding to a single drug package through cross-view image matching; the drug regulatory code areas where there is overlap and occlusion in the multi-scale regulatory code feature data are decoded in parallel to generate a regulatory code text sequence uniquely associated with each drug package; the regulatory code text sequence is compared with a preset drug production batch number database in real time to generate a batch recognition result map.

[0047] The technical solution of this application collects a set of original images containing densely arranged drug regulatory codes on the surface of drug packaging, and processes these images using a preset neural network model to generate multi-scale regulatory code feature data. This method can effectively cope with the complex geometric deformation and perspective changes of the drug packaging surface, thereby improving recognition accuracy. For regulatory code areas with overlapping and occlusion, a parallel decoding strategy is used to generate a regulatory code text sequence uniquely associated with each drug package, greatly improving the ability and efficiency of parsing drug regulatory codes in complex situations. Finally, by comparing the generated regulatory code text sequence with the drug production batch number database in real time, the drug tracking management task can be completed quickly and accurately, improving the transparency and reliability of the entire process.

[0048] Furthermore, a dynamic attention mask weight matrix is ​​generated to compensate for the character topological relationship after the deformation of the pharmaceutical packaging surface, ensuring that character information can be accurately extracted even under complex packaging surface conditions. Then, the model's ability to understand features of different scales is enhanced by combining multi-level feature vector groups in a channel-by-channel fusion manner, which helps to more accurately locate and identify regulatory code characters. Next, bidirectional context association decoding technology is adopted, which not only considers the information of a single character, but also makes full use of the context clues of its surrounding characters, significantly improving the accuracy and robustness of the recognition results. Finally, by verifying the spatial continuity of the candidate text segment set and merging the valid text segments according to the spatial order of the pharmaceutical packaging body, the authenticity and integrity of the finally generated regulatory code text sequence are ensured, which greatly avoids the possibility of misidentification and provides strong technical support for pharmaceutical supervision.

[0049] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0051] Figure 1 A flowchart of a method for batch identification of drug regulatory codes based on a neural network provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of the structure of a neural network-based drug regulatory code batch recognition system provided in an embodiment of the present application;

[0053] Figure 3A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0055] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0056] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0057] Figure 1 A flowchart of a method for batch identification of drug regulatory codes based on a neural network is provided for the embodiment of this application. Figure 1 As shown, the method includes:

[0058] Step 101: collecting a set of original images of drug packaging surfaces containing densely arranged drug regulatory codes;

[0059] In this step, the original image set refers to a series of two-dimensional images containing densely arranged drug regulatory codes on the surface of drug packaging, which are captured or scanned by specific equipment (such as industrial cameras or scanners). These images include not only the regulatory codes themselves, but may also contain other elements such as packaging background and text information. In order to ensure the effectiveness of subsequent processing, the quality of the image is crucial. Therefore, it is necessary to control factors such as shooting angle, lighting conditions and resolution to obtain a clear and consistent image dataset.

[0060] In this embodiment, a high-resolution industrial camera is first installed on the production line to ensure that it can cover the entire surface of the drug packaging. The camera's angle and focal length are adjusted to accurately capture the regulatory code on each drug package. Next, a suitable light source is set to avoid image blur caused by shadows or reflections. The camera's frame rate is then configured according to the speed of the assembly line to ensure that each drug package can be fully photographed. Finally, the captured image is saved in a standard format (such as JPEG or PNG) using image acquisition software.

[0061] For example, a pharmaceutical factory has deployed multiple high-resolution industrial cameras on its high-speed automated packaging line. These cameras are evenly distributed along both sides of the conveyor belt, ensuring that all regulatory codes on the drug packaging can be captured from different angles. When the drug packaging passes under the camera, the camera automatically triggers the capture, generating a series of high-quality original image sets. For example, at a certain moment, a pallet containing multiple drugs passes under the camera, and the camera captures a set of images containing 50 drug packages according to preset parameters. Each image clearly shows the regulatory code on the packaging surface and the detailed information around it.

[0062] Step 102: Input the original image set into a preset neural network model, and use the neural network model to generate multi-scale regulatory code feature data corresponding to a single drug package through cross-view image matching;

[0063] In this step, cross-view image matching refers to using a neural network model to analyze multiple images of the same drug packaging taken from different angles and extract common features.

[0064] In this embodiment, first, the collected original image set is imported into a pre-trained neural network model. This model is based on deep learning technology and has powerful feature extraction capabilities. The model will perform multi-scale feature extraction on each image and find common feature points under different perspectives through a cross-perspective image matching algorithm. Then, the spatial relationship of these feature points is combined to generate multi-scale regulatory code feature data corresponding to a single drug package.

[0065] For example, continuing with the pharmaceutical factory scenario in the above example, after the central server receives a set of original images of 50 drug packages taken by an industrial camera, it immediately inputs these images into the neural network model. The model begins to process each image and extracts regulatory code feature information at various scales. For those regulatory codes that are difficult to directly identify due to package curves or occlusions, the model finds consistent feature points under different perspectives through cross-view image matching technology, successfully generating multi-scale regulatory code feature data for each drug package.

[0066] Step 103: Parallel decoding is performed on the drug regulatory code regions where the multi-scale regulatory code feature data overlaps or is blocked, to generate a regulatory code text sequence uniquely associated with each drug package.

[0067] In this step, parallel decoding refers to the use of neural network models and related algorithms to simultaneously process multiple drug regulatory code areas, especially those with overlapping or partial occlusion. Through the dynamic attention mask weight matrix (used to compensate for the character topological relationship after the deformation of the drug packaging surface), the multi-level feature vector group (the key feature set extracted from the fusion result) and the bidirectional context association decoder (which can consider the contextual information around the character), the system can efficiently parse each drug regulatory code character and generate a set of candidate text segments containing the character position confidence.

[0068] In this embodiment, first, based on the multi-scale regulatory code feature data, the system generates a dynamic attention mask weight matrix to compensate for the character deformation problem caused by the surface deformation of the drug packaging. Then, this weight matrix is ​​fused with the multi-scale regulatory code feature data channel by channel to extract the multi-level feature vector group in each overlapping and occluded area. Next, the parallel decoder is used to perform bidirectional context association decoding on these feature vector groups to generate a set of candidate text segments containing character position confidence. Finally, the system performs character space continuity verification based on surface deformation compensation on these candidate text segments, screens valid text segments that meet the preset continuity constraints, and merges them into a unique regulatory code text sequence according to the spatial arrangement order of the drug packaging body.

[0069] For example, continuing with the pharmaceutical factory example above, after the central server receives the multi-scale regulatory code feature data, the system begins processing those regulatory code areas that are difficult to directly identify due to the curvature or partial occlusion of the packaging surface. For example, the regulatory code on a certain drug package has some characters deformed and slightly overlapping due to slight wrinkles in the packaging material. The system uses a dynamic attention mask weight matrix to accurately compensate for this area and extracts key features through a multi-level feature vector group. Next, the system uses a parallel decoder to perform bidirectional context-associative decoding on these features, successfully generating a set of candidate text segments containing character position confidence. Specifically, for a regulatory code containing "ABCD12345678", the system first generates multiple possible candidate text segments, such as "ABCD1234" and "BCD12345". By performing character space continuity verification on these candidate text segments based on surface deformation compensation, the system selects valid text segments that meet the continuity constraints and merges them according to the order of the drug packaging on the assembly line to generate a unique regulatory code text sequence "ABCD12345678".

[0070] Step 104: performing a real-time comparison between the regulatory code text sequence and a preset drug production batch number database to generate a batch recognition result map;

[0071] In this step, the batch recognition result map is a summary of the results generated by real-time comparison of the regulatory code text sequence corresponding to each drug package with the drug production batch number database. It not only includes the matching status of the regulatory code and batch number, but also records any abnormal or mismatched data points to facilitate subsequent quality control and traceability management.

[0072] In this embodiment, the generated regulatory code text sequence is first compared in real time with a preset drug production batch number database. By comparing the characters in each regulatory code text sequence with the corresponding records in the database, the system can quickly determine whether there is a match. If an anomaly is found (such as an unmatched regulatory code or an unexpected batch number), the system will automatically generate a mark and add the relevant information to the batch recognition result map. In this way, the operator can quickly locate and solve potential problems by viewing the map.

[0073] For example, continuing with the pharmaceutical factory scenario in the above example, the system compares the generated regulatory code text sequence "ABCD12345678" with the drug production batch number database. The results show that "ABCD12345678" completely matches the production record of a certain batch, indicating that the drug packaging meets the standards. However, the regulatory code "XYZW98765432" of another package cannot find the corresponding batch number record. The system immediately marks the package as abnormal and records detailed information (including packaging image, shooting time, assembly line location, etc.) in the batch recognition result map, so that the quality control team can take quick action to check whether the package has the risk of counterfeiting or other quality issues.

[0074] During the recognition process of drug regulatory codes, complexities on the packaging surface (such as curved surface deformation, overlap, and occlusion) are major challenges that affect recognition accuracy. Traditional image processing techniques are unable to cope with these complexities. Therefore, in some embodiments, according to step 103, the drug regulatory code regions with overlap and occlusion in the multi-scale regulatory code feature data are decoded in parallel to generate a regulatory code text sequence uniquely associated with each drug package, including:

[0075] Step 201: generating a dynamic attention mask weight matrix corresponding to overlapping and occluded areas based on the character topological relationship after compensation for the drug package surface deformation in the multi-scale supervision code feature data;

[0076] In this step, multi-scale regulatory code feature data refers to various feature information of the regulatory code on the surface of the drug packaging extracted from different perspectives and resolutions. These feature data include the position, shape, size and relationship of the regulatory code characters; drug packaging surface deformation compensation refers to the correction of geometric deformation caused by bending or wrinkling of the drug packaging surface through an algorithm to restore the true shape of the regulatory code characters; character topology describes the spatial arrangement and connection method between regulatory code characters, and can maintain certain structural characteristics even in the case of deformation and occlusion; dynamic attention mask weight matrix is ​​an adaptive weight distribution mechanism, which dynamically adjusts the importance weight of each character area according to the character topology relationship, thereby enhancing the recognition ability of overlapping and occluded areas.

[0077] In this embodiment, after receiving the multi-scale regulatory code feature data, the regulatory code on a certain drug package is processed. Assuming that there is a slight curvature on the packaging surface, causing some regulatory code characters to be deformed and slightly overlapped, a neural network model is first used to analyze this situation, calculate the specific deformation parameters, and compensate the characters to restore them to a more standard form. Then, based on the compensated character topology, a dynamic attention mask weight matrix is ​​generated. In this matrix, higher weights are assigned to those areas with severe overlap and occlusion, while lower weights are assigned to clearly visible character areas.

[0078] Step 202: fusing the dynamic attention mask weight matrix with the character topology after surface deformation compensation in the multi-scale supervision code feature data channel by channel to obtain a fusion result, and extracting a multi-level feature vector group in each overlapping and occluded area in the fusion result;

[0079] In this step, the dynamic attention mask weight matrix is ​​an adaptive weight distribution mechanism that dynamically adjusts the importance weight of each character area according to the character topological relationship, thereby enhancing the recognition ability of overlapping and occluded areas; channel-by-channel fusion refers to combining the dynamic attention mask weight matrix with the different channel information in the multi-scale supervision code feature data channel by channel to generate a more accurate feature representation; the multi-level feature vector group is a set of key features at different levels extracted from the fusion results. These features can better describe the detailed features of the supervision code characters and help the subsequent decoding process.

[0080] In this embodiment, the dynamic attention mask weight matrix is ​​first fused channel by channel with the character topological relationship after surface deformation compensation in the multi-scale supervision code feature data. For each channel (such as color channel, depth channel, etc.), the dynamic attention mask weight matrix is ​​applied to the corresponding feature data, and the fusion result is generated by weighted summation. The purpose of this is to highlight those areas that are more important in the recognition process (such as overlapping and occluded areas) while reducing the influence of unimportant areas. Next, multi-level feature vector groups in each overlapping and occluded area are extracted from the fusion results. These feature vector groups contain key feature information at different scales and can comprehensively describe each character and its surrounding context.

[0081] Step 203: Based on the multi-level feature vector group, a parallel decoder is used to perform bidirectional context-sensitive decoding on each drug regulatory code character in the overlapping and occluded areas to generate a set of candidate text segments including character position confidences;

[0082] In this step, the multi-level feature vector group is a set of key features at different levels extracted from the fusion results. These features can better describe the detailed features of the regulatory code characters; the parallel decoder is a decoding mechanism that efficiently processes multiple character areas and can process character information of multiple overlapping and occluded areas at the same time; bidirectional context-related decoding refers to not only considering the information of a single character, but also combining the context clues of the previous and next characters to improve recognition accuracy; the character position confidence is an evaluation value of the position of each character in the image and its correctness, which is used to measure the reliability of the decoding results. The candidate text segment set is a series of possible regulatory code text fragments generated by the system, each of which contains a character and its position confidence.

[0083] In this embodiment, a multi-level feature vector group is first used as input and sent to a parallel decoder. The parallel decoder performs bidirectional context-related decoding on each character, that is, it not only considers the information of the current character itself, but also combines the information of the characters before and after it, so as to more accurately identify each character. For example, for a partially occluded character "5", the system not only analyzes the shape characteristics of the character itself, but also refers to the context information of its left and right adjacent characters (such as "4" and "6") to determine the most likely character form. In this way, the system can generate a series of candidate text segment sets containing character position confidences. Each candidate text segment not only contains the recognized character sequence, but also includes the position of each character in its image and its confidence score. This process helps in subsequent screening and verification steps to ensure that the final generated regulatory code text sequence has high accuracy.

[0084] Step 204: Verify the character space continuity of the candidate text segments based on surface deformation compensation, select valid text segments that meet the preset continuity constraints, and merge the valid text segments according to the spatial arrangement order of the drug packages on the assembly line to generate a regulatory code text sequence uniquely associated with each drug package.

[0085] In this step, character spatial continuity verification based on surface deformation compensation refers to verifying the spatial continuity and logical consistency between characters while taking into account the surface deformation of the pharmaceutical packaging to ensure the accuracy of the recognition results; the preset continuity constraints are a set of rules or thresholds used to determine whether the character sequence conforms to the expected spatial and logical relationships, such as character spacing and order; the valid text segment is a candidate text segment that meets the preset continuity constraints after verification.

[0086] First, the system verifies the character space continuity of the candidate text segment set based on surface deformation compensation. According to the actual surface shape of the drug packaging surface, the spatial position of each character is adjusted, and it is verified whether these characters meet the preset continuity constraints. For those cases where characters are deformed due to packaging bending, the deformation compensation algorithm is used to restore the true position and shape of the characters, and then verification is performed. Next, valid text segments that meet the preset continuity constraints are screened out.

[0087] For example, if the character spacing in a candidate text segment is too large or too small, or the character order is not as expected, the text segment will be excluded. Only valid text segments with expected character spacing and order will be retained. Finally, these valid text segments are merged according to the spatial arrangement order of the drug packaging on the assembly line to generate a regulatory code text sequence uniquely associated with each drug packaging.

[0088] During the pharmaceutical production process, it is extremely important to ensure that the regulatory code on each pharmaceutical package can be accurately identified and associated. Due to geometric deformations (such as bending and twisting) on ​​the pharmaceutical package surface and spatial arrangement changes caused by conveyor belt movement, traditional character recognition methods may not guarantee high-precision recognition results. Based on this, as another embodiment, according to step 204, the character space continuity verification based on surface deformation compensation is performed on the candidate text segment set, and valid text segments that meet the preset continuity constraints are selected. The valid text segments are then merged according to the spatial arrangement order of the pharmaceutical packages on the assembly line to generate a regulatory code text sequence uniquely associated with each pharmaceutical package, including:

[0089] Step 301, calculating a curvature matching index of each character in the candidate text segment set based on the character topological relationship after surface deformation compensation;

[0090] In this step, the character topological relationship after surface deformation compensation refers to the spatial arrangement and connection method between characters restored after compensating for the geometric deformation of the pharmaceutical packaging surface; the curvature matching index is a numerical indicator used to measure the degree of similarity between each character shape and the ideal shape, which reflects the accuracy of the character shape after deformation compensation.

[0091] In this embodiment, first, the morphological features of each character are analyzed based on the character topological relationship after surface deformation compensation, and the edge contour, curvature, etc. of each character are analyzed in detail, and its curvature value is calculated, wherein the curvature value can be used to describe the curvature of the character curve. A higher curvature value indicates a larger curvature, and a lower curvature value indicates a smaller curvature or a straight line. Next, the curvature matching index of each character is calculated based on these curvature values. This index is usually a comprehensive score that reflects the degree of similarity between the character morphology and the ideal morphology. In order to calculate the curvature matching index, the system compares the actual curvature value of each character with the curvature value of its ideal morphology, and calculates the difference between the two. The smaller the difference, the higher the curvature matching index, indicating that the character morphology is closer to the ideal morphology.

[0092] Step 302: dynamically adjusting the character spacing threshold range in the continuity constraint based on the character distribution statistical characteristics of the decoded drug regulatory codes in the same drug production batch to obtain an adjusted character spacing threshold range;

[0093] In this step, character distribution statistical features refer to statistical analysis of the characters of the successfully decoded drug regulatory code, extracting characteristic information such as the spacing and order between characters; the character spacing threshold range is an important parameter in the continuity constraint, which defines the maximum and minimum distances allowed between characters.

[0094] In this example, a statistical analysis of successfully decoded drug regulatory codes from the same drug production batch is first performed to extract statistical characteristics of character distribution. Specifically, the system calculates statistics such as the average spacing and standard deviation between each character. Based on these statistics, the character spacing threshold range in the continuity constraint is dynamically adjusted. Next, the adjusted character spacing threshold range is used to re-verify that the character spacing in the candidate text segment set meets the requirements. Text segments with character spacing exceeding the new threshold range are eliminated, and only valid text segments that meet the new threshold range are retained.

[0095] Step 303: Mark candidate text segments that satisfy a curvature matching index exceeding the upper limit of the surface deformation compensation error and whose character spacing is within the adjusted character spacing threshold as valid text segments, and record a spatial position mapping table of the valid text segments in the conveyor belt coordinate system;

[0096] In this step, the curvature matching index is a numerical indicator used to measure the degree of similarity between each character shape and the ideal shape. It reflects the accuracy of the character shape after deformation compensation; the upper limit of the surface deformation compensation error refers to the maximum deformation error range allowed by the system. Character shapes beyond this range are considered unreliable; the spatial position mapping table is a data structure that records the specific position of each valid text segment in the assembly line conveyor belt coordinate system, which is used for subsequent merging and tracking management.

[0097] In this embodiment, first, based on the previously calculated curvature matching index and the adjusted character spacing threshold range, candidate text segments that meet the conditions are screened out, and the characters in each candidate text segment are checked to see whether they meet the following two conditions (the curvature matching index exceeds the upper limit of the surface deformation compensation error and the character spacing is within the adjusted character spacing threshold range). Only text segments that meet both conditions will be marked as valid text segments. Next, the spatial positions of these valid text segments in the coordinate system of the assembly line conveyor belt are recorded, and the specific position of each drug package is marked in the coordinate system of the assembly line conveyor belt. These position information are associated with the valid text segments to generate a spatial position mapping table. This mapping table can be used for text segment merging in subsequent steps and traceability management of the entire system.

[0098] Step 304: splicing valid text segments belonging to the same medicine package into character sequences based on the conveyor belt movement direction according to the spatial position mapping table to obtain a spliced ​​character sequence;

[0099] In this step, the conveyor belt movement direction refers to the direction in which the drug packaging body moves on the assembly line, usually the X-axis or Y-axis direction; character sequence splicing is to merge the valid text segments belonging to the same drug packaging body according to their arrangement order on the conveyor belt.

[0100] In this embodiment, first, the drug packaging to which each valid text segment belongs is determined based on the coordinate information in the spatial position mapping table, and each valid text segment is classified into the corresponding drug packaging according to its spatial position (such as the X-axis and Y-axis coordinates). Next, these valid text segments are sorted according to the movement direction of the conveyor belt, and the valid text segments are sorted according to this order to ensure that they are spliced ​​together in the correct order. Finally, the sorted valid text segments are spliced ​​into character sequences to generate a complete regulatory code text sequence, wherein all valid text segments belonging to the same drug packaging will be spliced ​​together one by one according to their arrangement order on the conveyor belt to form a continuous character sequence.

[0101] Step 305: Eliminate text segment conflicts between different drug packages based on the monotonically increasing order of character position confidences in the spliced ​​character sequence, and generate a regulatory code text sequence uniquely associated with each drug package.

[0102] In this step, the character position confidence is an evaluation value of the position of each character in its image and its correctness, which is used to measure the reliability of the decoding results; the monotonically increasing order refers to sorting in order from low to high or from high to low according to the character position confidence; the text segment conflict refers to the situation where the character sequences on multiple drug packages may partially overlap or be confused, which requires the system to further process to ensure that the regulatory code information of each drug package is unique and accurate; the uniquely associated regulatory code text sequence refers to the final generated complete and unique regulatory code text sequence that corresponds one-to-one to each drug package.

[0103] In this embodiment, the position confidence of each character in the spliced ​​character sequence is first calculated. For example, if a character is recognized very clearly and without ambiguity, its position confidence will be high; on the contrary, if a character is blurred or blocked, its position confidence will be low; then the characters are sorted in the monotonically increasing order of the character position confidence, and the order from high confidence to low confidence is selected to give priority to the most reliable characters, which can minimize the possibility of false matching and conflict; then, different drug packaging bodies are gradually eliminated according to the sorted character position confidence. The system checks the position confidence of each character one by one to check which characters belong to which drug package based on these confidence scores. If some characters on two adjacent drug packages overlap or are similar, the confidence scores of the characters will be used to determine which package these characters belong to. Finally, the character sequence after conflict elimination will be reassembled to generate a regulatory code text sequence uniquely associated with each drug package. Each drug package will have a complete and unique regulatory code text sequence, ensuring that each drug package can be accurately tracked and managed throughout the production process.

[0104] During the pharmaceutical packaging production process, due to the movement of the conveyor belt and the temporal and spatial differences between images taken from different perspectives, characters in the original image set may be misplaced or deformed. To address this issue, in some embodiments, according to step 102, the original image set is input into a preset neural network model, and the neural network model is used to generate multi-scale regulatory code feature data corresponding to a single pharmaceutical package through cross-perspective image matching, including:

[0105] Step 401, performing time alignment on the original image set based on the movement speed of the pipeline conveyor belt to generate a multi-view image frame sequence;

[0106] In this step, the original image set refers to a set of images of the surface of the pharmaceutical packaging captured by multiple cameras or sensors within the same time period. These images may come from different perspectives, and due to the movement of the conveyor belt, there may be a time offset between the images captured at different time points; timing alignment refers to adjusting the timestamp of image acquisition according to the movement speed of the conveyor belt to ensure that all images are arranged in the correct order and synchronized for subsequent processing; a multi-view image frame sequence is a sequence of image frames from multiple perspectives that have undergone timing alignment, and these image frames can be used for further feature extraction and analysis.

[0107] In this embodiment, the original image set is first acquired and the timestamp of each image is recorded. Then, the time interval between each image is calculated based on the movement speed of the assembly line conveyor belt. Next, the original image set is time-series aligned, and finally a multi-view image frame sequence is generated. Each image frame contains surface information of the drug packaging from different perspectives. This information can be used for subsequent feature point matching and geometric deformation field construction.

[0108] Step 402: performing cross-view feature point matching on each drug package in the multi-view image frame sequence using a neural network model to generate a cross-view regional correlation matrix reflecting the spatial distribution of regulatory code characters between adjacent views.

[0109] In this step, cross-view feature point matching refers to finding common feature points (such as key points of regulatory code characters) between image frames of different viewpoints to establish a correspondence between them; the cross-view regional correlation matrix is ​​a data structure that represents the similarity and correspondence of the spatial distribution of regulatory code characters between adjacent viewpoints. It reflects the spatial position and morphological differences of regulatory code characters under different viewpoints, which helps in subsequent geometric deformation compensation and feature extraction.

[0110] In this embodiment, a multi-view image frame sequence is first input into a pre-trained neural network model. The model automatically detects and extracts feature points in each image frame and matches them based on the correspondence between these feature points across different viewpoints. For each pharmaceutical package, common feature points are found in the corresponding image frames across multiple viewpoints, and the similarity and correspondence between these feature points are calculated. For example, if a pharmaceutical package contains the same regulatory code character "1" in image frames from two different viewpoints, the feature points of the character "1" are found in both viewpoints and the similarity score between them is calculated. Finally, a cross-view regional correlation matrix is ​​generated that reflects the spatial distribution of the regulatory code characters between adjacent viewpoints. This matrix records the spatial position and morphological differences of the regulatory code characters from different viewpoints, as well as the correspondence between them. For example, if the character "1" is located at positions (10, 20) and (12, 22) in two different viewpoints, the correspondence between these two positions and their similarity score are recorded in the cross-view regional correlation matrix.

[0111] Step 403: constructing a geometric deformation field associated with the geometric deformation parameters of the drug packaging surface based on the cross-view regional correlation matrix, and using the geometric deformation field to map the multi-view image frame sequence to a unified reference coordinate system to generate multi-scale supervision code feature data;

[0112] In this step, the geometric deformation field is a mathematical model that describes the geometric deformation of the pharmaceutical packaging surface. It estimates the actual morphological changes of the packaging surface by analyzing the correspondence between feature points under different perspectives. The geometric deformation parameters refer to the specific numerical values ​​used to describe the geometric deformation of the pharmaceutical packaging surface, such as the degree of curvature and the twist angle. The unified reference coordinate system is a standard spatial coordinate system used to map image frames from different perspectives into the same coordinate system for subsequent processing and analysis. The multi-scale regulatory code feature data is the regulatory code feature information at different scales extracted from the mapped image frames. These features can more comprehensively describe the regulatory code characters and the surrounding background information.

[0113] In this embodiment, a geometric deformation field associated with the geometric deformation parameters of the drug packaging surface is first constructed based on the cross-viewing regional correlation matrix. The corresponding relationship between the feature points of each drug package at different viewing angles is analyzed, and the corresponding geometric deformation parameters (such as curvature, distortion angle, etc.) are calculated. For example, assuming that the feature point positions of a drug package at two viewing angles are (10, 20) and (12, 22), the curvature of the packaging surface is calculated based on these position differences and expressed as geometric deformation parameters. Next, the geometric deformation field is used to map the multi-viewing image frame sequence to a unified reference coordinate system. For each image frame, the deformation parameters in the geometric deformation field are used to compensate for its deformation and restore it to its ideal shape. For example, if the regulatory code characters in a certain image frame are deformed due to the curvature of the packaging surface, they will be inversely transformed according to the calculated geometric deformation parameters to restore their original shape. Then, all the image frames that have undergone deformation compensation are mapped to a unified reference coordinate system to ensure that they are aligned in the same spatial coordinate system. Finally, key feature information of the regulatory code characters and their surrounding background, such as edges, textures, shapes, etc., will be extracted at different scales to generate multi-scale regulatory code feature data.

[0114] When dealing with the geometric deformation problem of the pharmaceutical packaging surface, relying solely on preliminary feature point matching may not be sufficient to completely correct all deformations. In order to more accurately compensate for the deformation, it is necessary to further analyze the spatial distribution characteristics in the cross-view region correlation matrix, calculate the dynamic deformation offset, and construct a geometric deformation field covering the entire pharmaceutical packaging surface. Based on this, as another embodiment, according to step 403, a geometric deformation field associated with the geometric deformation parameters of the pharmaceutical packaging surface is constructed based on the cross-view region correlation matrix, and the geometric deformation field is used to map the multi-view image frame sequence to a unified reference coordinate system to generate multi-scale supervision code feature data, including:

[0115] Step 501, based on the spatial distribution characteristics of drug regulatory code characters between adjacent viewing angles in the cross-view region correlation matrix, calculate a dynamic deformation offset that is positively correlated with the curvature gradient of the drug packaging surface;

[0116] In this step, the curvature gradient refers to the speed and direction of curvature change at a specific point. For the pharmaceutical packaging surface, it describes the change in the degree of curvature of the packaging surface, that is, the difference in curvature at different positions; the dynamic deformation offset refers to the adjustment value calculated to compensate for the change in the shape of the pharmaceutical packaging surface. This offset is calculated based on the curvature gradient of the packaging surface and is dynamically adjusted as the surface shape changes.

[0117] In this embodiment, the spatial distribution characteristics of the drug regulatory code characters between adjacent viewing angles recorded in the cross-view area correlation matrix are first analyzed. By comparing these spatial distribution characteristics, the curvature pattern of the drug packaging surface and its influence on the relative position of the regulatory code characters can be identified. If the position of the regulatory code characters observed from one viewing angle is displaced relative to another viewing angle, this may be due to the curvature of the packaging surface. Next, based on the above analysis results, the corresponding deformation offset is determined according to the curvature gradient at each position. This means that in areas with larger curvature (such as the corners of the packaging bag), the deformation offset will be larger; while in relatively flat areas, the deformation offset will be smaller. Finally, by applying these dynamic deformation offsets, the regulatory code character position error caused by the geometric deformation of the packaging surface is effectively corrected.

[0118] Step 502: Generate a geometric deformation field covering the entire surface of the drug packaging based on the dynamic deformation offset and the surface curvature gradient;

[0119] In this embodiment, the dynamic deformation offset of each local region is combined with its corresponding curvature gradient to form a continuous and smooth deformation distribution model. For example, larger deformation offset values ​​are assigned to regions with greater curvature (such as the edge of the packaging bag), while smaller deformation offset values ​​are assigned to flatter regions (such as the center of the packaging bag). Next, an interpolation or fitting algorithm is used to extend the deformation information of the local region to the entire pharmaceutical packaging surface, thereby generating a complete geometric deformation field. The resulting geometric deformation field comprehensively describes the deformation at every location on the pharmaceutical packaging surface. This deformation field not only includes the deformation information of each local region but also reflects the geometric characteristics of the entire surface, providing an important reference for subsequent image correction and feature extraction.

[0120] Step 503: inputting the spatial coordinates of each pixel point in the multi-view image frame sequence into the geometric deformation field, and performing affine transformation compensation on the spatial coordinates of each pixel point using the curvature gradient matrix in the geometric deformation field to obtain transformed spatial coordinates;

[0121] In this step, the curvature gradient matrix is ​​a mathematical matrix used to describe the degree of curvature and its variation at different locations on the pharmaceutical packaging surface. This matrix combines the global information of the geometric deformation field with the local curvature gradient characteristics, providing a precise deformation compensation basis for the spatial coordinates of each pixel. Affine transformation compensation is a geometric transformation method that corrects coordinate offsets caused by surface deformation by performing linear transformations (such as translation, rotation, scaling, or shearing) on ​​the spatial coordinates of each pixel. This method can effectively handle complex geometric deformation problems and bring image data closer to the ideal state.

[0122] In this embodiment. First, the original spatial coordinates of each pixel point (such as the X-axis and Y-axis position) are read one by one, and the deformation compensation value of the pixel point is calculated according to the curvature gradient matrix in the geometric deformation field. For example, for a certain pixel point, if the curvature of the area where it is located is large (such as the edge of the packaging bag), it will be assigned a larger compensation value according to the curvature gradient matrix; while for a relatively flat area (such as the center of the packaging bag), a smaller compensation value is assigned. Next, the deformation offset of each pixel point is calculated according to the curvature gradient matrix, and it is adjusted by the affine transformation formula. For example, assuming that the original spatial coordinates of a pixel point are (10, 20), it is calculated according to the curvature gradient matrix that it needs to move 2 units in the X-axis direction and 1 unit in the Y-axis direction. After the affine transformation, the new coordinates of the pixel point become (12, 21). This affine transformation not only takes into account the position offset of the pixel point, but may also involve operations such as rotation or scaling to further correct complex geometric deformations, and finally obtain the transformed spatial coordinate set.

[0123] Step 504: performing secondary correction on the transformed spatial coordinates using the dynamic deformation offset field to generate multi-scale supervision code feature data;

[0124] In this step, geometric deformation parameters refer to specific numerical values ​​used to describe the geometric deformation of the pharmaceutical packaging surface, such as the degree of bending, twisting angle, etc.; the unified reference coordinate system is a standard spatial coordinate system used to map image frames from different perspectives into the same coordinate system for subsequent processing and analysis.

[0125] In this embodiment, the correspondence between the feature points of each drug package under different viewing angles is first analyzed, and the corresponding geometric deformation parameters (such as bending degree, distortion angle, etc.) are calculated. For example, assuming that the feature point positions of a drug package under two viewing angles are (10, 20) and (12, 22) respectively, the bending degree of the packaging surface is calculated based on these position differences and expressed as geometric deformation parameters. Next, for each image frame, the deformation parameters in the geometric deformation field are used to compensate for its deformation and restore it to its ideal shape. For example, if the regulatory code characters in a certain image frame are deformed due to the bending of the packaging surface, they are inversely transformed according to the calculated geometric deformation parameters to restore their original shape. Then, all the image frames that have undergone deformation compensation are mapped to a unified reference coordinate system to ensure that they are aligned in the same spatial coordinate system. Finally, the multi-scale regulatory code feature data is extracted from the mapped image frames.

[0126] In a drug production batch, the character distribution of the regulatory code text sequence may be somewhat random and complex. To ensure that the regulatory code of each drug package can be accurately identified and associated, it is necessary to dynamically construct a batch character entropy baseline and compare it with the real-time decoded regulatory code text sequence to detect abnormal character distribution patterns. Based on this, in some embodiments, according to step 104, the regulatory code text sequence is compared with a preset drug production batch number database in real time to generate a batch recognition result map, including:

[0127] Step 601: Based on the character distribution characteristics of the decoded drug regulatory code text sequence in the same drug production batch, dynamically construct a batch character entropy value baseline associated with the speed change of the assembly line conveyor belt and the spatial arrangement density of the drug packaging body;

[0128] In this step, the batch character entropy baseline is a benchmark value that reflects the randomness and complexity of the character distribution of the drug regulatory code in a specific production batch. It combines the changes in conveyor belt speed and the density of drug packaging on the conveyor belt to evaluate the overall information entropy of the drug regulatory code of the batch. The higher the character entropy baseline, the more random the character distribution; conversely, it means that the character distribution is more regular or orderly.

[0129] In this embodiment, information such as the frequency of occurrence of each character, the interval between characters, and the pattern in the character sequence is first counted. For example, in a batch, some characters may appear frequently while other characters appear less frequently, or some character sequences may show a certain degree of repetitiveness. Next, the weights of the character distribution features are adjusted according to the changes in the conveyor belt speed, because a higher conveyor belt speed may lead to more recognition errors and a more random character distribution. At the same time, the spatial arrangement density of the drug packaging bodies is taken into consideration. Densely arranged packaging bodies may increase the possibility of mutual interference between characters, thereby affecting the character entropy value. For example, if the conveyor belt speed is faster and the packaging bodies are densely arranged, a higher character entropy value baseline will be calculated; on the contrary, if the conveyor belt speed is slower and the packaging bodies are sparsely arranged, a lower character entropy value baseline will be calculated.

[0130] Step 602: performing a spatiotemporal alignment comparison of the decoding position confidence of each character in the regulatory code text sequence with the batch character entropy baseline to generate an entropy mutation coefficient matrix reflecting the distribution pattern of abnormal characters;

[0131] In this step, the entropy mutation coefficient matrix is ​​a data structure used to represent the degree of difference between the confidence of each character decoding position in a specific production batch and the batch character entropy baseline. Each element in the matrix represents the entropy mutation coefficient of a character position. The larger the value, the more the character distribution at that position deviates from the normal pattern, and there may be anomalies.

[0132] In this embodiment, the position confidence of each character is first matched with its corresponding entropy baseline based on the timestamp and spatial coordinate information. For example, assuming that the decoding position confidence of a character is 0.95 and its corresponding entropy baseline is 0.85, the difference between the two is calculated. Next, the entropy mutation coefficient is calculated based on these differences, and an entropy mutation coefficient matrix is ​​generated. For each character position, the difference between its decoding position confidence and the entropy baseline is calculated and converted into a mutation coefficient. For example, if the decoding confidence of a character position is significantly lower than the expected entropy baseline, the entropy mutation coefficient of the position will be higher, reflecting the possible existence of an abnormal character distribution pattern.

[0133] Step 603: Based on the entropy mutation coefficient matrix and the spatial position mapping table of the drug packaging in the assembly line conveyor coordinate system, the edge position of the drug supervision code exceeding the preset entropy mutation threshold is traced back to generate an abnormal mark data cluster;

[0134] In this step, the abnormal marking data cluster is a data set containing abnormal drug regulatory codes and their detailed information, which includes the specific location of the abnormal characters, the time point of occurrence, and the corresponding spatial position on the conveyor belt.

[0135] In this embodiment, an entropy mutation threshold (such as 0.1) is first set, and all character positions that exceed this threshold in the entropy mutation coefficient matrix are marked. For example, if the entropy mutation coefficient of a character position is 0.15, which exceeds the preset threshold, the position is marked as abnormal. Next, these abnormal characters are edge-positioned to determine their specific physical positions. The spatial position mapping table is used to find the specific coordinates of each abnormal character on the conveyor belt, and their timestamps are recorded. For example, assuming that an abnormal character is located on the conveyor belt at an X-axis position of 200 mm and a Y-axis position of 50 mm, and appears at a time point t = 0.5 s, this information is recorded to generate an abnormal marking data cluster, which contains relevant information of all abnormal characters.

[0136] Step 604: Perform weighted superposition on the abnormality mark data cluster and the surface deformation compensation parameter in the multi-scale supervision code feature data to generate a batch recognition result map;

[0137] In this embodiment, a corresponding surface deformation compensation parameter is first applied to each abnormal character position to correct errors caused by the curvature or distortion of the packaging surface. For example, if an abnormal character is offset by 2 pixels due to the curvature of the packaging surface, the system will adjust it based on the surface deformation compensation parameter. Next, the corrected abnormal mark data cluster is combined with the multi-scale regulatory code feature data to generate a batch recognition result map. This batch recognition result map not only shows the specific location and time point of the abnormal character, but also the geometric deformation of each drug packaging body in the entire production batch. For example, color coding is used to indicate the recognition accuracy and deformation degree of different locations, with red indicating high-risk areas and green indicating low-risk areas.

[0138] During the pharmaceutical production process, regulatory codes on certain pharmaceutical packaging may exhibit abnormal character distribution patterns. To accurately locate these anomalies and conduct effective quality control, it is necessary to construct an abnormal region diffusion model and perform point-by-point calibration and spatial matching in combination with multi-scale regulatory code feature data. Based on this, as another embodiment, according to step 601, the regulatory code text sequence is compared with a preset pharmaceutical production batch number database in real time to generate a batch recognition result map, including:

[0139] Step 701: construct an abnormal region diffusion model that is positively correlated with the curvature gradient of the drug packaging surface based on the entropy mutation coefficient matrix of each abnormal character and the gradient change trend of adjacent characters;

[0140] In this step, the abnormal area diffusion model is a mathematical model used to describe the expansion of abnormal characters and their surrounding areas on the surface of the drug packaging.

[0141] In this embodiment, first, the degree of difference between each abnormal character and its adjacent characters (such as position confidence or change in character morphology) is calculated, and this information is combined to evaluate the expansion possibility of the abnormal area. For example, if the entropy value mutation of a character is high and its adjacent characters also show a large gradient change, then the area may have a higher risk of diffusion. Next, an abnormal area diffusion model is constructed based on the above analysis results.

[0142] Step 702: Using the abnormal region diffusion model, locate the initial physical coordinate set of the abnormal supervision code in the assembly line conveyor coordinate system, and calibrate the initial physical coordinate set point by point based on the reflectivity difference correction value of the edges of adjacent drug packages in the multi-scale supervision code feature data to generate a reflectivity-corrected abnormal physical coordinate cluster;

[0143] In this step, the abnormal physical coordinate cluster after reflectivity correction is a set of specific physical coordinates of abnormal characters after reflectivity difference correction.

[0144] In this embodiment, the diffusion range of each abnormal character is first predicted based on the abnormal area diffusion model, and its initial physical coordinates on the conveyor belt are determined. For example, assuming that an abnormal character is located at an X-axis position of 200 mm and a Y-axis position of 50 mm on the conveyor belt, the system will record these initial coordinates to generate an initial physical coordinate set. Next, the initial physical coordinate set is calibrated point by point based on the reflectivity difference correction amount of the edges of adjacent drug packaging bodies in the multi-scale supervision code feature data. For each initial physical coordinate, it is adjusted according to the reflectivity difference correction amount of its surroundings, and finally a reflectivity-corrected abnormal physical coordinate cluster is obtained.

[0145] Step 703: Match the reflectivity-corrected abnormal physical coordinate cluster with the spatial arrangement order of the corresponding drug packages in the spatial position mapping table, screen out the abnormal coordinate subset that meets the preset spatial continuity constraint, and associate it with the corresponding character distribution abnormality level in the entropy mutation coefficient matrix;

[0146] The abnormal coordinate subset is a set of specific physical coordinates of abnormal characters that meet the preset spatial continuity constraints after screening.

[0147] In this embodiment, first, the corresponding drug packaging in the spatial position mapping table will be searched based on the physical coordinates of each abnormal character, and the coordinates will be checked to see if they meet the preset spatial continuity constraints. For example, assuming that the coordinates of an abnormal character are located on the conveyor belt at an X-axis position of 200 mm and a Y-axis position of 50 mm, it will be checked whether the position is consistent with the spatial arrangement order of the corresponding drug packaging. Next, a subset of abnormal coordinates that meet the preset spatial continuity constraints will be screened out. For each abnormal character, its physical coordinates will be evaluated to see if they meet the spatial continuity requirements. If the coordinates of an abnormal character deviate significantly from the expected position, the coordinates will not be included in the abnormal coordinate subset. In addition, the corresponding character distribution abnormality level in the entropy mutation coefficient matrix will be associated to add additional abnormal information for each abnormal coordinate subset.

[0148] Step 704: generating an abnormality mark data cluster including a deformation compensation residual, a reflectivity difference correction amount, and an entropy value mutation amount according to the abnormal coordinate subset and the associated character distribution abnormality level;

[0149] In this embodiment, a comprehensive analysis is first performed on the characters in each abnormal coordinate subset to extract their deformation compensation residuals (such as errors due to incomplete correction of packaging surface deformation), reflectivity difference corrections (such as coordinate offsets due to differences in reflectivity of packaging materials) and entropy mutations (such as the difference between the character position confidence and the batch character entropy baseline). Next, for each abnormal character, the system records its physical coordinates, deformation compensation residuals, reflectivity difference corrections and entropy mutations, and associates them with the character distribution abnormality level. For example, assuming that the deformation compensation residual of an abnormal character is 0.5 pixels, the reflectivity difference correction is 1 pixel, the entropy mutation is 0.15, and its character distribution abnormality level is high, this information is integrated to obtain an abnormal labeling data cluster.

[0150] Figure 2 The present invention provides a schematic diagram of a drug regulatory code batch recognition system based on a neural network, as shown in FIG. Figure 2 As shown, the system includes:

[0151] The acquisition module 21 acquires a set of original images of drug packaging surfaces containing densely arranged drug regulatory codes;

[0152] An input module 22 inputs the original image set into a preset neural network model, and uses the neural network model to generate multi-scale supervision code feature data corresponding to a single drug package through cross-view image matching;

[0153] The decoding module 23 decodes the overlapping and occluded drug regulatory code regions of the multi-scale regulatory code feature data in parallel to generate a regulatory code text sequence uniquely associated with each drug package;

[0154] The comparison module 24 is used to compare the regulatory code text sequence with the preset drug production batch number database in real time to generate a batch recognition result map.

[0155] Figure 2 The drug regulatory code batch recognition system based on neural network can be executed Figure 1 The implementation principles and technical effects of the neural network-based batch identification method for drug regulatory codes described in the illustrated embodiment will not be elaborated on here. The specific manner in which each module and unit performs operations in the neural network-based batch identification system for drug regulatory codes in the above embodiment has been described in detail in the relevant embodiments of the method and will not be elaborated on here.

[0156] In one possible design, Figure 2 The drug regulatory code batch recognition system based on neural network of the embodiment shown can be implemented as a computing device, such as Figure 3As shown, the computing device may include a storage component 31 and a processing component 32;

[0157] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0158] The processing component 32 is used to collect a set of original images containing densely arranged drug regulatory codes on the surface of drug packaging; input the set of original images into a preset neural network model, and use the neural network model to generate multi-scale regulatory code feature data corresponding to a single drug package through cross-view image matching; parallel decode the drug regulatory code areas where there is overlap and occlusion in the multi-scale regulatory code feature data to generate a regulatory code text sequence uniquely associated with each drug package; and compare the regulatory code text sequence with a preset drug production batch number database in real time to generate a batch recognition result map.

[0159] The processing component 32 may include one or more processors to execute computer instructions to perform all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0160] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0161] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0162] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0163] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0164] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0165] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment is a method for batch identification of drug regulatory codes based on a neural network.

[0166] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0168] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for batch identification of drug regulatory codes based on neural networks, characterized in that: include: Collecting a set of original images containing densely arranged drug regulatory codes on the surface of drug packaging; The original image set is time-sequentially aligned based on the movement speed of the assembly line conveyor belt to generate a multi-view image frame sequence; cross-view feature point matching is performed on each drug package in the multi-view image frame sequence using a neural network model to generate a cross-view regional correlation matrix reflecting the spatial distribution of regulatory code characters between adjacent views; Based on the spatial distribution characteristics of the drug regulatory code characters between adjacent viewing angles in the cross-view region correlation matrix, a dynamic deformation offset positively correlated with the curvature gradient of the drug packaging surface is calculated; generating a geometric deformation field covering the entire surface of the pharmaceutical packaging according to the dynamic deformation offset and the surface curvature gradient; Inputting the spatial coordinates of each pixel point in the multi-view image frame sequence into the geometric deformation field, performing affine transformation compensation on the spatial coordinates of each pixel point using the curvature gradient matrix in the geometric deformation field to obtain transformed spatial coordinates; performing secondary correction on the transformed spatial coordinates using the dynamic deformation offset field to generate multi-scale supervision code feature data; Parallel decoding is performed on the drug regulatory code regions where the multi-scale regulatory code feature data overlaps or is obscured, to generate a regulatory code text sequence uniquely associated with each drug package; The regulatory code text sequence is compared with the preset drug production batch number database in real time to generate a batch recognition result map.

2. The method according to claim 1, characterized in that Parallel decoding is performed on the drug regulatory code regions where the multi-scale regulatory code feature data overlaps and is obscured, to generate a regulatory code text sequence uniquely associated with each drug package, including: Based on the character topological relationship after compensation for the drug packaging surface deformation in the multi-scale regulatory code feature data, a dynamic attention mask weight matrix corresponding to the overlapping and occluded areas is generated; The dynamic attention mask weight matrix is ​​fused with the character topology relationship after surface deformation compensation in the multi-scale supervision code feature data channel by channel to obtain a fusion result, and a multi-level feature vector group in each overlapping and occluded area in the fusion result is extracted; Based on the multi-level feature vector group, a parallel decoder is used to perform bidirectional context-associative decoding on each drug regulatory code character in the overlapping and occluded area to generate a set of candidate text segments including character position confidences; The set of candidate text segments is subjected to character space continuity verification based on surface deformation compensation, and valid text segments that meet preset continuity constraints are screened. The valid text segments are merged according to the spatial arrangement order of the drug packaging bodies on the assembly line to generate a regulatory code text sequence uniquely associated with each drug packaging body.

3. The method according to claim 2, characterized in that The candidate text segment set is subjected to character space continuity verification based on surface deformation compensation, valid text segments that meet preset continuity constraints are screened, and the valid text segments are merged according to the spatial arrangement order of the drug packages on the assembly line to generate a regulatory code text sequence uniquely associated with each drug package, including: Calculating a curvature matching index for each character in the candidate text segment set based on the character topological relationship after surface deformation compensation; Dynamically adjusting the character spacing threshold range in the continuity constraint according to the character distribution statistical characteristics of the decoded drug regulatory codes in the same drug production batch to obtain an adjusted character spacing threshold range; Marking candidate text segments that satisfy the curvature matching index exceeding the upper limit of the surface deformation compensation error and whose character spacing is within the adjusted character spacing threshold range as valid text segments, and recording a spatial position mapping table of the valid text segments in the assembly line conveyor coordinate system; According to the spatial position mapping table, valid text segments belonging to the same medicine package are spliced ​​into character sequences based on the conveyor belt movement direction to obtain a spliced ​​character sequence; The text segment conflicts of different drug packages are eliminated according to the monotonically increasing order of the character position confidences in the spliced ​​character sequence, and a supervision code text sequence uniquely associated with each drug package is generated.

4. The method according to claim 1, wherein The regulatory code text sequence is compared with the preset drug production batch number database in real time to generate a batch recognition result map, including: Based on the character distribution characteristics of the decoded drug regulatory code text sequence in the same drug production batch, a batch character entropy value baseline associated with the change in the conveyor belt speed and the spatial arrangement density of the drug packaging is dynamically constructed; Performing a spatiotemporal alignment comparison of the decoding position confidence of each character in the regulatory code text sequence with the batch character entropy baseline to generate an entropy mutation coefficient matrix reflecting the distribution pattern of abnormal characters; Based on the entropy mutation coefficient matrix and the spatial position mapping table of the drug packaging body in the assembly line conveyor coordinate system, the edge position of the drug supervision code exceeding the preset entropy mutation threshold is traced back to generate an abnormal mark data cluster; The abnormal mark data cluster is weightedly superimposed with the surface deformation compensation parameters in the multi-scale supervision code feature data to generate a batch recognition result map.

5. The method according to claim 4, characterized in that Based on the character distribution characteristics of the decoded drug regulatory code text sequence in the same drug production batch, a batch character entropy value baseline associated with the change in conveyor belt speed and the spatial arrangement density of drug packaging is dynamically constructed, including: Based on the entropy mutation coefficient matrix of each abnormal character and the gradient change trend of adjacent characters, an abnormal area diffusion model positively correlated with the curvature gradient of the drug packaging surface is constructed; The abnormal region diffusion model is used to locate the initial physical coordinate set of the abnormal supervision code in the assembly line conveyor coordinate system, and the initial physical coordinate set is calibrated point by point based on the reflectivity difference correction value of the edges of adjacent drug packages in the multi-scale supervision code feature data to generate a reflectivity-corrected abnormal physical coordinate cluster; Matching the reflectivity-corrected abnormal physical coordinate cluster with the spatial arrangement order of the corresponding drug packaging in the spatial position mapping table, screening out an abnormal coordinate subset that meets the preset spatial continuity constraint, and associating the corresponding character distribution abnormality level in the entropy mutation coefficient matrix; According to the abnormal coordinate subset and the associated character distribution abnormality level, an abnormal mark data cluster including a deformation compensation residual, a reflectivity difference correction amount and an entropy value mutation amount is generated.

6. A drug regulatory code batch recognition system based on neural network, characterized in that: include: An acquisition module, used to acquire a set of original images containing densely arranged drug regulatory codes on the surface of drug packaging; An input module is configured to perform time-series alignment on the original image set based on the movement speed of the assembly line conveyor belt to generate a multi-view image frame sequence; perform cross-view feature point matching on each pharmaceutical package in the multi-view image frame sequence using a neural network model to generate a cross-view regional correlation matrix reflecting the spatial distribution of regulatory code characters between adjacent views; Based on the spatial distribution characteristics of the drug regulatory code characters between adjacent viewing angles in the cross-view region correlation matrix, a dynamic deformation offset positively correlated with the curvature gradient of the drug packaging surface is calculated; generating a geometric deformation field covering the entire surface of the pharmaceutical packaging according to the dynamic deformation offset and the surface curvature gradient; Inputting the spatial coordinates of each pixel point in the multi-view image frame sequence into the geometric deformation field, performing affine transformation compensation on the spatial coordinates of each pixel point using the curvature gradient matrix in the geometric deformation field to obtain transformed spatial coordinates; performing secondary correction on the transformed spatial coordinates using the dynamic deformation offset field to generate multi-scale supervision code feature data; A decoding module is used to decode in parallel the drug regulatory code areas where the multi-scale regulatory code feature data overlaps and is blocked, and generate a regulatory code text sequence uniquely associated with each drug package; The comparison module is used to compare the regulatory code text sequence with the preset drug production batch number database in real time to generate a batch recognition result map.

7. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a neural network-based drug regulatory code batch identification method as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, it implements a neural network-based drug regulatory code batch recognition method as described in any one of claims 1 to 5.

Images