A method for detecting surface defects of omeprazole enteric-coated capsules

Abstract

The present application provides a method for detecting surface defects of omeprazole enteric-coated capsules, which belongs to the field of artificial intelligence technology. The method comprises: an electronic device acquires M images of the omeprazole enteric-coated capsules, where the M images are multiple images taken during the process of the omeprazole enteric-coated capsules rotating on the same axis, and M is an integer greater than 1; the electronic device extracts features of each of the M images through a neural network model, and fuses the features of each of the M images to obtain fused features of the M images; the electronic device processes the fused features of the M images through the neural network model to obtain a processing result, and the processing result indicates whether there are defects on the surface of the omeprazole enteric-coated capsules.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for detecting surface defects of omeprazole enteric-coated capsules. Background Art

[0002] Capsules, a common oral medication form, are widely used in clinical treatments due to their portability and ease of use. However, any drug may develop surface defects during production and use, such as damage, deformation, or color inconsistency in the capsule shell. These issues not only affect the drug's appearance and quality but also pose potential risks to patient safety. For example, consider omeprazole enteric-coated capsules, a commonly used medication for gastrointestinal diseases. Surface defects in omeprazole enteric-coated capsules can affect the drug's release rate and efficacy, and may even cause leakage or deterioration of the drug's ingredients. Therefore, detecting and controlling surface defects in capsules is particularly important.

[0003] Currently, methods for detecting surface defects in pharmaceutical capsules primarily include visual inspection, microscopic observation, X-ray testing, and ultrasonic testing. However, these methods can be cumbersome, inefficient, or require specialized equipment and personnel. At the same time, there is a growing demand for rapid and accurate detection of surface defects in pharmaceutical capsules on large-scale production lines. Summary of the Invention

[0004] The embodiment of the present application provides a method for detecting surface defects of omeprazole enteric-coated capsules, which is used to achieve highly robust surface defect detection of omeprazole enteric-coated capsules.

[0005] To achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a method for detecting surface defects of omeprazole enteric-coated capsules is provided, which is applied to an electronic device. The method comprises: the electronic device acquires M images of the omeprazole enteric-coated capsules, where the M images are multiple images taken while the omeprazole enteric-coated capsules are rotating on the same axis, and M is an integer greater than 1; the electronic device extracts features of each of the M images through a neural network model, and fuses the features of each of the M images to obtain fused features of the M images; the electronic device processes the fused features of the M images through the neural network model to obtain a processing result, and the processing result indicates whether there are defects on the surface of the omeprazole enteric-coated capsules.

[0007] Optionally, there are multiple omeprazole enteric-coated capsules, and the multiple omeprazole enteric-coated capsules are arranged along a first direction and clamped by their respective corresponding clamping components. The first direction is the axis direction of each of the multiple omeprazole enteric-coated capsules. When the motor drives the multiple omeprazole enteric-coated capsules to rotate along the first direction, the shooting device shoots the multiple omeprazole enteric-coated capsules multiple times to obtain corresponding M images.

[0008] Optionally, when the motor drives the multiple omeprazole enteric-coated capsules to rotate 30° along the first direction, the shooting device takes an image of the multiple omeprazole enteric-coated capsules, and stops rotating and shooting when the motor drives the multiple omeprazole enteric-coated capsules to rotate 180° along the first direction, that is, M=6.

[0009] Optionally, the neural network model is a decoupled deployment model, which means that the neural network model includes a decoupled feature extraction sub-model and a feature processing sub-model. The electronic device extracts the features of each of the M images through the neural network model, and fuses the features of the M images to obtain the fusion features of the M images, including: the electronic device preprocesses each of the M images to obtain a picture containing only the edge area of ​​the omeprazole enteric-coated capsules, and a total of M pictures are obtained; the electronic device inputs the M pictures into the feature extraction sub-model, and obtains the feature sequence corresponding to each of the M pictures output by the feature extraction sub-model, and a total of M feature sequences are obtained; the electronic device classifies and fuses the M feature sequences according to the matching degree to obtain N fused feature sequences, where N is an integer greater than 1, and the N fused feature sequences are the fusion features of the M images.

[0010] Optionally, the electronic device preprocesses each of the M images to obtain a picture containing only the edge area of ​​the omeprazole enteric-coated capsules, including: the electronic device determines the pixel points located at the edge of the omeprazole enteric-coated capsules in each image by binarizing or grayscale processing each of the M images; the electronic device extracts a pattern containing only the portion of the omeprazole enteric-coated capsules from each image based on the pixel points located at the edge of the omeprazole enteric-coated capsules in each image, obtaining a total of M patterns; the electronic device reduces the size of the outline of any one of the M patterns by a preset size to obtain a target outline; the electronic device aligns the center point of each pattern with the center point of the target outline, and cuts off the portion of each pattern covered by the target outline to obtain a corresponding picture, obtaining a total of M patterns.

[0011] Optionally, each of the M feature sequences contains L elements, that is, the length of each feature sequence in the M feature sequences is L, the preset sequence length is X, L is an integer greater than 3, and X is an integer greater than 1 and less than L. The electronic device classifies and fuses the M feature sequences according to the matching degree to obtain N fused feature sequences, including: the electronic device divides each feature sequence in the M feature sequences into K feature subsequences according to the preset sequence length, K=ceiling(LmodX), ceiling() represents rounding up, and a total of K*M feature subsequences are obtained; the electronic device determines the correlation between every two feature subsequences in the K*M feature subsequences, and divides the feature subsequences with high correlation into the same feature sequence group, and a total of N feature sequence groups are obtained; the electronic device splices the feature subsequences contained in each feature sequence group in the N feature sequence groups to obtain a corresponding fused feature sequence, and a total of N fused feature sequences are obtained.

[0012] Optionally, the electronic device divides the feature subsequences with high correlation into the same feature sequence group by determining the correlation between every two feature subsequences in the K*M feature subsequences, including: for the i-th feature subsequence in the K*M feature subsequences, i is an integer traversing from 1 to K*M: if the i-th feature subsequence has been divided into a feature sequence group, the electronic device does not process the i-th feature subsequence; or, if the i-th feature subsequence has not been divided into a feature sequence group, the electronic device determines the correlation between the i-th feature subsequence and each of the (K*M)-1 feature subsequences, and obtains (K*M)-1 correlations, (K*M)-1 feature subsequences. The sequence is a feature subsequence other than the i-th feature subsequence in the K*M feature subsequences; the electronic device selects the highest correlation from the (K*M)-1 correlations, and the highest correlation is the correlation between the i-th feature subsequence and the j-th feature subsequence in the (K*M)-1 feature subsequences, where j is an integer from 1 to (K*M)-1; if the j-th feature subsequence has been divided into a feature sequence group, the i-th feature subsequence is divided into the feature sequence group to which the j-th feature subsequence belongs; if the j-th feature subsequence has not been divided into a feature sequence group, the i-th feature subsequence and the j-th feature subsequence are divided into a new feature sequence group.

[0013] Optionally, the electronic device processes the fusion features of M images through a neural network model to obtain a processing result, including: the electronic device inputs N fusion feature sequences into a feature processing sub-model to obtain a processing result output by the feature processing sub-model.

[0014] Optionally, the electronic device inputs N fused feature sequences into a feature processing sub-model to obtain a processing result output by the feature processing sub-model, including: the electronic device instructs the feature processing sub-model to select the top N classifiers with the highest performance from the classifiers of the feature processing sub-model based on the number of the N fused feature sequences being N; the electronic device sorts the N fused feature sequences from short to long according to sequence length, adds identification elements to the top P fused feature sequences with the highest ranking, and obtains P fused feature sequences with added identification elements, and NP fused feature sequences without added identification elements, where P is an integer greater than or equal to 1 and less than N; the electronic device sorts the N fused feature sequences from short to long according to sequence length, and inputs the P fused feature sequences with added identification elements and the NP fused feature sequences without added identification elements into the N classifiers of the feature processing sub-model one by one to obtain the processing result output by the feature processing sub-model, wherein, when inputting the N classifiers, the N classifiers need to be input in order from high to low according to the performance of the classifiers.

[0015] Optionally, the electronic device sorts N fusion feature sequences from short to long according to sequence length, adds identification elements to the top P fusion feature sequences, and obtains P fusion feature sequences with added identification elements, including: the electronic device sorts the N fusion feature sequences from short to long according to sequence length, randomly extracts P elements from the lowest-ranked fusion feature sequence, and adds P elements to the top P fusion feature sequences, and obtains P fusion feature sequences with added identification elements, where the P elements are identification elements.

[0016] In a second aspect, a device for detecting surface defects of omeprazole enteric-coated capsules is provided, the device being configured to: obtain M images of the omeprazole enteric-coated capsules, where the M images are multiple images taken while the omeprazole enteric-coated capsules are rotating on the same axis, and M is an integer greater than 1; extract features of each of the M images through a neural network model, and fuse the features of each of the M images to obtain fused features of the M images; process the fused features of the M images through the neural network model to obtain a processing result, and the processing result indicates whether there are defects on the surface of the omeprazole enteric-coated capsules.

[0017] In a third aspect, a computer-readable storage medium is provided, comprising: a computer program or instructions; when the computer program or instructions are run on a computer, the computer is caused to execute the method described in the first aspect.

[0018] The above method and device have the following specific technical effects:

[0019] Considering that the shape of omeprazole enteric-coated capsules is similar to a cylinder, M images can be captured while the omeprazole enteric-coated capsules are rotating on the same axis. The electronic device can obtain these M images and extract the features of each of the M images through a neural network model. The features of each of the M images are then fused to obtain the fused features of the M images. At this time, since the features are fused, the neural network model can perform a holistic analysis of the M images, so that the processing results output by the neural network model can accurately indicate whether there are defects on the surface of the omeprazole enteric-coated capsules, thereby achieving highly robust surface defect detection of omeprazole enteric-coated capsules. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic flow chart of a method for detecting surface defects of omeprazole enteric-coated capsules provided in an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram of the Omeprazole Enteric-coated Capsules in the examples of this application;

[0022] Figure 3 This is a schematic diagram of the structure of the neural network model in the embodiment of the present application;

[0023] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solution in this application will be described below with reference to the accompanying drawings.

[0025] This application will present various aspects, embodiments, or features in the context of systems that may include multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in conjunction with the figures. Furthermore, combinations of these aspects may also be used.

[0026] Additionally, in the embodiments of this application, words such as "exemplary" and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner.

[0027] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field will know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0028] To facilitate understanding of the embodiments of the present application, first Figure 2 The smart photovoltaic power storage system based on the Internet of Things shown in the figure is used as an example to describe in detail the smart photovoltaic power storage system based on the Internet of Things applicable to the embodiments of the present application.

[0029] For ease of understanding, the following will be combined Figure 1 The surface defect detection method for omeprazole enteric-coated capsules provided in the examples of the present application is specifically described.

[0030] For example, Figure 1 The present invention provides a flow chart of a method for detecting surface defects of omeprazole enteric-coated capsules. The method can be applied to electronic devices, i.e., performed by electronic devices.

[0031] The electronic device may specifically be a terminal, which may also be referred to as a terminal device, user equipment (UE), mobile station, mobile terminal, etc. The terminal can be widely used in various scenarios, for example, device-to-device (D2D), vehicle to everything (V2X) communication, machine-type communication (MTC), Internet of Things (IOT), virtual reality, augmented reality, industrial control, autonomous driving, telemedicine, smart grid, smart furniture, smart office, smart wearable, smart transportation, smart city, etc. The terminal may be a mobile phone, a tablet computer, a computer with wireless transceiver function, a wearable device, etc. The embodiments of the present application do not limit the device form of the terminal.

[0032] like Figure 1 As shown, the process of this method is as follows:

[0033] S101, the electronic device obtains M images of omeprazole enteric-coated capsules.

[0034] M images are multiple images taken when the omeprazole enteric-coated capsule is rotated on the same axis, and M is an integer greater than 1. Figure 2As shown, there are multiple omeprazole enteric-coated capsules arranged along a first direction and held by corresponding clamping assemblies. The first direction is the axis direction of each of the multiple omeprazole enteric-coated capsules. As the motor drives the multiple omeprazole enteric-coated capsules to rotate along the first direction, the camera captures the multiple omeprazole enteric-coated capsules multiple times to obtain M corresponding images. Specifically, the camera captures one image of the multiple omeprazole enteric-coated capsules every 30° rotation along the first direction driven by the motor. The camera stops rotating and capturing images when the motor drives the multiple omeprazole enteric-coated capsules to rotate 180° along the first direction, i.e., M = 6.

[0035] It should be understood that Figure 2 As shown, the omeprazole enteric-coated capsules can also be arranged in multiple rows, with each row arranged in the above manner and rotated to capture the image. This allows a single image to capture a larger number of omeprazole enteric-coated capsules. In one example, a row of omeprazole enteric-coated capsules can contain 12 capsules, and the number of rows can be 6 or 8, meaning that 72 or 96 omeprazole enteric-coated capsules can be inspected for surface defects at a time. In practical applications, the omeprazole enteric-coated capsules can be transported by a conveyor belt to a positioning device, where they are sorted and placed on corresponding clamping assemblies, thereby improving subsequent inspection efficiency.

[0036] It can be understood that the purpose of the above-mentioned rotational shooting is to enable the subsequent processing of the neural network model to analyze the surface characteristics of the omeprazole enteric-coated capsules, such as whether there are uneven and easily broken areas on the surface, that is, to achieve surface defect detection.

[0037] S202: The electronic device extracts features of each of the M images through a neural network model, and fuses the features of the M images to obtain fused features of the M images.

[0038] The neural network model may be a deep neural network (DNN), specifically a DNN with improved structure. For example, the neural network model is a decoupled deployment model, and the decoupled deployment model means that the neural network model includes a decoupled feature extraction sub-model, an intermediate processing sub-model, and a feature processing sub-model. Among them, the feature extraction sub-model includes an input layer and a feature extraction layer, and the input layer is understood as a data transmission channel from the input port to the feature extraction layer, and the feature extraction layer can perform a convolution operation. The feature processing sub-model may include an input layer, multiple classifiers, and an output layer, and the input layer is understood as a feature transmission channel from the input port to the multiple classifiers. Each of the multiple classifiers contains neurons of a certain size, and each neuron performs a weighted sum operation on its input value, and generates an output through a nonlinear function on the weighted sum result, that is, the feature processing process, and the output layer can perform a deconvolution operation. Decoupling the feature extraction sub-model and the feature processing sub-model means that the features output by the feature extraction sub-model are not directly input into the feature processing sub-model within the model, but are output to the intermediate processing sub-model, which then processes them further before inputting them into the feature processing sub-model. This is described in detail below.

[0039] The electronic device can pre-process each of the M images to obtain an image containing only the edge region of the omeprazole enteric-coated capsule, and obtain a total of M images.

[0040] For example, the electronic device can perform binarization or grayscale processing on each of the M images to determine the pixel points located at the edge of the omeprazole enteric-coated capsule in each image, that is, the pixel points with relatively large pixel differences after binarization or grayscale processing, that is, the pixel points located at the edge of the omeprazole enteric-coated capsule, so as to determine the edge of the omeprazole enteric-coated capsule in each image. In this way, the electronic device extracts a pattern containing only the omeprazole enteric-coated capsule portion from each image based on the pixel points located at the edge of the omeprazole enteric-coated capsule in each image, and obtains a total of M patterns (each pattern can contain patterns of multiple omeprazole enteric-coated capsules contained in the image. In other words, the pattern is not a complete pattern, but a pattern of multiple omeprazole enteric-coated capsules, that is, a collection of patterns), that is, stripping off the background elements in the image. On the one hand, it is convenient to subsequently concentrate on extracting and analyzing the features located on the surface of the omeprazole enteric-coated capsule. On the other hand, it can also reduce the amount of data processing and improve detection efficiency.

[0041] The electronic device can reduce the size of the outline of any one of the M patterns by a preset size (for example, the preset size can be 2 / 3 of the original size) to obtain the target outline, thereby aligning the center point of each pattern with the center point of the target outline, and cutting off the portion of each pattern covered by the target outline to obtain a corresponding image, and a total of M images are obtained. In other words, each omeprazole enteric-coated capsule is rotated and photographed, and the edge outline of each omeprazole enteric-coated capsule in the obtained image can reflect the corresponding area on the surface of the omeprazole enteric-coated capsule, and the outlines in different images can reflect the omeprazole enteric-coated capsule. The M images can essentially encompass the entire surface of the omeprazole enteric-coated capsules by analyzing the areas near the edge contours of the omeprazole enteric-coated capsules in the images. Therefore, with this particular shooting method, the subsequent neural network model only needs to analyze the edge areas of the omeprazole enteric-coated capsule pattern in any image, without having to analyze the center area of ​​the pattern. Therefore, the center area can be cropped out using the aforementioned method. This means that each omeprazole enteric-coated capsule pattern in each resulting image is a ring-like pattern, also known as a quasi-annular pattern. Furthermore, each image can contain multiple quasi-annular patterns of omeprazole enteric-coated capsules contained in the image. In other words, the image is not a complete image, but rather a collection of multiple quasi-annular patterns of omeprazole enteric-coated capsules, i.e., a collection of quasi-annular patterns.

[0042] The electronic device can input M pictures into the feature extraction sub-model, and obtain the feature sequence corresponding to each of the M pictures output by the feature extraction sub-model, and obtain a total of M feature sequences. That is, the feature sequence is obtained through convolution extraction, and the size of the convolution for each picture is consistent, so the length of each feature sequence is also fixed.

[0043] The electronic device (or the electronic device through the intermediate processing sub-model) can classify and fuse M feature sequences according to the matching degree to obtain N fused feature sequences, where N is an integer greater than 1. The N fused feature sequences are the fused features of the M images.

[0044] For example, each of the M feature sequences contains L elements, that is, the length of each feature sequence in the M feature sequences is L, the preset sequence length is X, L is an integer greater than 3, and X is an integer greater than 1 and less than L. On this basis,

[0045] The electronic device divides each feature sequence in the M feature sequences into K feature subsequences according to the preset sequence length, K=ceiling(LmodX), ceiling() represents rounding up, and a total of K*M feature subsequences are obtained. Among them, the size relationship between X and L needs to meet certain conditions, such as X is basically 1 / 9-1 / 10 of L, to ensure that the number of features of each feature subsequence is moderate, which is convenient for the subsequent feature classification of the features extracted from the defective part. The electronic device can determine the correlation between each two feature subsequences in the K*M feature subsequences (such as inner product, Euclidean distance, etc.), and divide the feature subsequences with high correlation into the same feature sequence group, thereby obtaining a total of N feature sequence groups.

[0046] Specifically, for the i-th feature subsequence in the K*M feature subsequences, i is an integer ranging from 1 to K*M: if the i-th feature subsequence has been divided into a feature sequence group, the electronic device does not process the i-th feature subsequence; or, if the i-th feature subsequence has not been divided into a feature sequence group, the electronic device determines the correlation between the i-th feature subsequence and each of the (K*M)-1 feature subsequences, and obtains (K*M)-1 correlations. The (K*M)-1 feature subsequences are feature subsequences in the K*M feature subsequences except the i-th feature subsequence. ; The electronic device selects the highest correlation from the (K*M)-1 correlations, the highest correlation being the correlation between the ith feature subsequence and the jth feature subsequence in the (K*M)-1 feature subsequences, where j is an integer ranging from 1 to (K*M)-1; if the jth feature subsequence has been divided into a feature sequence group, the ith feature subsequence is divided into the feature sequence group to which the jth feature subsequence belongs; if the jth feature subsequence has not been divided into a feature sequence group, the ith feature subsequence and the jth feature subsequence are divided into a new feature sequence group.

[0047] For example, assuming that K*M=10, i.e., including characteristic subsequences 1 to 10, the electronic device calculates the correlations between characteristic subsequence 1 and characteristic subsequences 2 to 10, thereby determining that characteristic subsequence 1 has the highest correlation with characteristic subsequence 3, and generates characteristic sequence group 1, which includes characteristic subsequence 1 and characteristic subsequence 3. Thereafter, the electronic device calculates the correlations between characteristic subsequence 2 and characteristic subsequences 1 and characteristic subsequences 3 to 10 (or may only calculate the correlations between characteristic subsequence 2 and characteristic subsequences 3 to 10), thereby determining that characteristic subsequence 2 has the highest correlation with characteristic subsequence 3, and classifies characteristic subsequence 2 into characteristic sequence group 1. The electronic device then calculates the correlations between characteristic subsequence 4 and characteristic subsequences 1 to 3, and characteristic subsequences 5 to 10 (or may only calculate the correlations between characteristic subsequence 4 and characteristic subsequence 3, and characteristic subsequence 5 to 10), thereby determining that characteristic subsequence 4 has the highest correlation with characteristic subsequence 6, and generates characteristic sequence group 2, which includes characteristic subsequence 4 and characteristic subsequence 6. The same procedure is repeated.

[0048] It should be understood that by calculating the correlation, the feature sequences extracted from the same defect can be divided into the same group, and the feature sequences extracted from non-defective sequences can be divided into other groups for easy subsequent processing. Of course, at this time, it is not yet certain which feature sequences are extracted from defects.

[0049] The electronic device splices the characteristic subsequences contained in each characteristic sequence group in the N characteristic sequence groups to obtain a corresponding fused characteristic sequence, and obtains a total of N fused characteristic sequences. The embodiment of the present application does not limit the order of splicing.

[0050] S203: The electronic device processes the fusion features of the M images through a neural network model to obtain a processing result.

[0051] The processing result indicates whether there is a defect on the surface of the omeprazole enteric-coated capsule. For example, the processing result may include M marked images, where the mark may be located at the position of the surface defect of the omeprazole enteric-coated capsule in the image, indicating at which position on the surface of the omeprazole enteric-coated capsule the defect is present.

[0052] For example, the electronic device inputs N fused feature sequences into the feature processing sub-model to obtain a processing result output by the feature processing sub-model.

[0053] Specifically, the electronic device can, based on the number of N fused feature sequences being N, instruct the feature processing sub-model to select the N classifiers with the top N performance rankings from the classifiers of the feature processing sub-model, that is, the N classifiers with the top N neuron sizes. The electronic device can sort the N fused feature sequences from short to long according to the sequence length, add identification elements to the top P fused feature sequences with the highest ranking, and obtain P fused feature sequences with added identification elements, and NP fused feature sequences without added identification elements, where P is an integer greater than or equal to 1 and less than N. For example, the electronic device can sort the N fused feature sequences from short to long according to the sequence length, randomly extract P elements from the lowest-ranked fused feature sequence, and add P elements to the top P fused feature sequences with the highest ranking, to obtain P fused feature sequences with added identification elements, where the P elements are identification elements. The effect of this is to facilitate the subsequent classifier to analyze the elements, and the slight differences between the elements can assist the classifier in identifying whether the feature sequence is a feature sequence extracted from a defect. The electronic device sorts the N fused feature sequences from short to long according to the sequence length (i.e., defects are usually smaller, so there are fewer features. A shorter feature sequence means that it is more likely that a defect is extracted and requires a classifier with better performance to process). The P fused feature sequences with added identification elements and the NP fused feature sequences without added identification elements are input one by one into the N classifiers of the feature processing sub-model to obtain the processing results output by the feature processing sub-model. Among them, the N classifiers can output N analysis results one by one, and each analysis result can indicate whether a corresponding fused feature sequence is extracted by a defect, that is, identify the feature sequence extracted by the defect. Among them, when inputting N classifiers, the N classifiers need to be input in order from high to low according to the performance of the classifier.

[0054] It should be understood that the output layer can mark the feature sequence extracted from the defect based on the N analysis results, and then the output layer can obtain the process information of the above fusion from the above intermediate processing sub-model, execute the reverse order of the above fusion, and restore the M feature sequences carrying the mark, and then perform deconvolution on the M feature sequences carrying the mark to obtain M marked images, that is, the processing results.

[0055] In summary, considering that the shape of the omeprazole enteric-coated capsules is similar to a cylinder, M images can be captured while the omeprazole enteric-coated capsules are rotating on the same axis. The electronic device can obtain these M images and extract the features of each of the M images through a neural network model. The features of each of the M images are then fused to obtain the fused features of the M images. At this time, since the features are fused, the neural network model can perform a holistic analysis of the M images, so that the processing results output by the neural network model can accurately indicate whether there are defects on the surface of the omeprazole enteric-coated capsules, thereby achieving highly robust surface defect detection of omeprazole enteric-coated capsules.

[0056] Combination of the above Figure 1 The surface defect detection method for omeprazole enteric-coated capsules provided by the embodiment of the present application is described in detail. The surface defect detection device for omeprazole enteric-coated capsules used to perform the above method is described below.

[0057] The device is configured to: obtain M images of omeprazole enteric-coated capsules, where the M images are multiple images taken during the process of the omeprazole enteric-coated capsules rotating on the same axis, and M is an integer greater than 1; extract the features of each of the M images through a neural network model, and fuse the features of each of the M images to obtain fused features of the M images; process the fused features of the M images through the neural network model to obtain a processing result, and the processing result indicates whether there are defects on the surface of the omeprazole enteric-coated capsules.

[0058] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. For example, the electronic device may be a terminal device, or a chip (system) or other component or assembly that can be provided in the terminal device. Figure 4 As shown, electronic device 400 may include a processor 401. Optionally, electronic device 400 may further include a memory 402 and / or a transceiver 403. Processor 401 is coupled to memory 402 and transceiver 403, for example, via a communication bus. Furthermore, electronic device 400 may be a chip, such as one including processor 401. In this case, the transceiver may be the chip's input / output interface.

[0059] The following combination Figure 4 The components of the electronic device 400 are described in detail.

[0060] The processor 401 is the control center of the electronic device 400 and can be a single processor or a collective term for multiple processing elements. For example, the processor 401 can be one or more central processing units (CPUs), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0061] Optionally, the processor 401 can execute various functions of the electronic device 400 by running or executing the software program stored in the memory 402 and calling the data stored in the memory 402, such as executing the above Figure 4 The method shown is used for detecting surface defects of omeprazole enteric-coated capsules.

[0062] In a specific implementation, as an embodiment, the processor 401 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 are shown in FIG.

[0063] In a specific implementation, as an embodiment, the electronic device 400 may also include multiple processors. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer programs or instructions).

[0064] The memory 402 is used to store the software program for executing the solution of the present application, and the execution is controlled by the processor 401. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0065] Alternatively, the memory 402 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 402 may be integrated with the processor 401 or exist independently and accessed through the interface circuit ( Figure 4 (not shown) is coupled to the processor 401, which is not specifically limited in this embodiment of the present application.

[0066] Transceiver 403 is used for communication with other electronic devices. For example, if electronic device 400 is a terminal device, transceiver 403 can be used to communicate with a network device or another terminal device. For another example, if electronic device 400 is a network device, transceiver 403 can be used to communicate with a terminal device or another network device.

[0067] Optionally, the transceiver 403 may include a receiver and a transmitter ( Figure 4 (not shown separately in the figure). The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0068] Optionally, the transceiver 403 may be integrated with the processor 401 or may exist independently and communicate with the electronic device 400 through the interface circuit ( Figure 4 (not shown) is coupled to the processor 401, which is not specifically limited in this embodiment of the present application.

[0069] It is understandable that Figure 4 The structure of the electronic device 400 shown in the figure does not constitute a limitation on the electronic device. The actual electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0070] In addition, the technical effects of the electronic device 400 can refer to the technical effects of the methods described in the above method embodiments, and will not be repeated here.

[0071] It should be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0072] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0073] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (such as infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0074] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0075] In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0076] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0077] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0078] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0079] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0080] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0081] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0082] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0083] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for detecting surface defects of omeprazole capsules, characterized in that: Applied to electronic equipment, the method includes: The electronic device acquires M images of the omeprazole capsule, wherein the M images are multiple images taken during the process of the omeprazole capsule rotating on the same axis, and M is an integer greater than 1; The electronic device preprocesses each of the M images to obtain an image containing only the edge region of the omeprazole capsule, obtaining a total of M images; the neural network model is a decoupled deployment model, wherein the decoupled deployment model means that the neural network model includes a decoupled feature extraction submodel and a feature processing submodel; The electronic device inputs the M pictures into the feature extraction sub-model, and obtains a feature sequence corresponding to each of the M pictures output by the feature extraction sub-model, obtaining a total of M feature sequences; The electronic device divides each of the M feature sequences into K feature subsequences according to a preset sequence length, where K=ceiling(LmodX), where ceiling() represents rounding up, to obtain a total of K*M feature subsequences; each of the M feature sequences contains L elements, that is, the length of each feature sequence in the M feature sequences is L, the preset sequence length is X, L is an integer greater than 3, and X is an integer greater than 1 and less than L; The electronic device determines the correlation between every two feature subsequences in the K*M feature subsequences, and groups the feature subsequences with high correlation into the same feature sequence group, thereby obtaining N feature sequence groups in total; The electronic device splices the feature subsequences contained in each feature sequence group in the N feature sequence groups to obtain a corresponding fused feature sequence, and obtains a total of N fused feature sequences, where N is an integer greater than 1, and the N fused feature sequences are the fused features of the M images; The electronic device processes the fusion features of the M images through the neural network model to obtain a processing result, and the processing result indicates whether there is a defect on the surface of the omeprazole capsule.

2. The method according to claim 1, characterized in that There are multiple omeprazole capsules, which are arranged along a first direction and clamped by their respective corresponding clamping assemblies. The first direction is the axial direction of each of the multiple omeprazole capsules. When the motor drives the multiple omeprazole capsules to rotate along the first direction, the shooting device shoots the multiple omeprazole capsules multiple times to obtain the corresponding M images.

3. The method according to claim 2, characterized in that When the motor drives the multiple omeprazole capsules to rotate 30° along the first direction, the shooting device shoots an image of the multiple omeprazole capsules. When the motor drives the multiple omeprazole capsules to rotate 180° along the first direction, the rotation and shooting are stopped, that is, M=6.

4. The method according to claim 1, wherein The electronic device preprocesses each of the M images to obtain an image containing only the edge region of the omeprazole capsule, including: The electronic device determines a pixel point located at an edge of the omeprazole capsule in each image by performing binarization or grayscale processing on each of the M images; The electronic device extracts a pattern containing only the omeprazole capsule portion from each image based on pixel points located at the edge of the omeprazole capsule in each image, to obtain a total of M patterns; The electronic device reduces the size of the outline of any one of the M patterns by a preset size to obtain a target outline; The electronic device aligns the center point of each pattern with the center point of the target outline, cuts off the portion of each pattern covered by the target outline, and obtains a corresponding picture, thereby obtaining the M pictures in total.

5. The method according to claim 1, wherein The electronic device determines the correlation between every two feature subsequences in the K*M feature subsequences, and classifies the feature subsequences with high correlation into the same feature sequence group, including: For the i-th characteristic subsequence in the K*M characteristic subsequences, i is an integer ranging from 1 to K*M: If the i-th characteristic subsequence has been classified into a characteristic sequence group, the electronic device does not process the i-th characteristic subsequence; or, If the i-th characteristic subsequence is not classified into a characteristic sequence group, the electronic device determines the correlation between the i-th characteristic subsequence and each of (K*M)-1 characteristic subsequences to obtain (K*M)-1 correlations, where the (K*M)-1 characteristic subsequences are characteristic subsequences in the K*M characteristic subsequences excluding the i-th characteristic subsequence; The electronic device selects a highest correlation from the (K*M)-1 correlations, where the highest correlation is the correlation between the i-th characteristic subsequence and the j-th characteristic subsequence in the (K*M)-1 characteristic subsequences, where j is an integer ranging from 1 to (K*M)-1; If the j-th feature subsequence has been classified into a feature sequence group, the i-th feature subsequence is classified into the feature sequence group to which the j-th feature subsequence belongs; if the j-th feature subsequence has not been classified into a feature sequence group, the i-th feature subsequence and the j-th feature subsequence are classified into a new feature sequence group.

6. The method according to claim 5, characterized in that The electronic device processes the fusion features of the M images through the neural network model to obtain a processing result, including: The electronic device inputs the N fused feature sequences into the feature processing sub-model to obtain the processing result output by the feature processing sub-model.

7. The method according to claim 6, characterized in that The electronic device inputs the N fused feature sequences into the feature processing sub-model to obtain the processing result output by the feature processing sub-model, including: The electronic device instructs the feature processing sub-model to select N classifiers with the top N performance rankings from the classifiers of the feature processing sub-model based on the number of the N fusion feature sequences being N; The electronic device sorts the N fused feature sequences from short to long according to sequence length, adds identification elements to the top P fused feature sequences, and obtains P fused feature sequences with the identification elements added, and N / P fused feature sequences without the identification elements added, where P is an integer greater than or equal to 1 and less than N. The electronic device sorts the N fused feature sequences according to sequence length from short to long, and inputs the P fused feature sequences with added identification elements and the NP fused feature sequences without added identification elements into the N classifiers of the feature processing sub-model one by one to obtain the processing results output by the feature processing sub-model, wherein when inputting the N classifiers, the N classifiers need to be input in order from high to low according to the performance of the classifier.

8. The method according to claim 7, characterized in that The electronic device sorts the N fused feature sequences from short to long according to sequence length, adds identification elements to the top P fused feature sequences with the highest ranking, and obtains P fused feature sequences with the added identification elements, including: The electronic device sorts the N fused feature sequences from short to long according to sequence length, randomly extracts P elements from the lowest-ranked fused feature sequence, and adds the P elements to the top P fused feature sequences with the highest ranking, to obtain the P fused feature sequences with added identification elements, where the P elements are the identification elements.

Images