Method for detecting surface defects of omeprazole enteric capsule

Abstract

The invention provides a method for detecting surface defects of an omeprazole enteric capsule, and belongs to the technical field of artificial intelligence, the method comprises the following steps: an electronic device obtains M images of the omeprazole enteric capsule, the M images are multiple images shot in the process that the omeprazole enteric capsule rotates on the same axis, and M is an integer greater than 1; the electronic equipment extracts respective features of the M images through a neural network model, and fuses the respective features of the M images to obtain fused features of the M images; the electronic equipment processes the fusion features of the M images through a neural network model to obtain a processing result, and the processing result indicates whether the surface of the omeprazole enteric capsule has defects or not.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to a method for detecting surface defects of omeprazole enteric-coated capsules. Background Art

[0002] As a common form of oral medication, capsule drugs are widely used in clinical treatment due to their convenience in carrying and use. However, any drug may have surface defect problems during production and use, such as cracks, deformations, or inconsistent colors in the capsule shell. These problems not only affect the appearance and quality of the drug but also pose potential risks to the safety of patients' medication. Taking omeprazole enteric-coated capsules as an example, which is a commonly used drug for treating gastrointestinal diseases. The surface defect problems of omeprazole enteric-coated capsules may affect the drug release rate and efficacy, and may even lead to leakage or deterioration of the drug ingredients. Therefore, it is particularly important to detect and control the surface defects of capsule drugs.

[0003] Currently, the detection methods for surface defects of capsule drugs mainly include manual visual inspection, microscopic observation, X-ray detection, and ultrasonic detection, etc. However, these methods either have cumbersome operations and low efficiency, or require professional equipment and operators. At the same time, the demand for quickly and accurately detecting surface defects of capsule drugs on large-scale production lines is increasing. Summary of the Invention

[0004] An embodiment of this application provides a method for detecting surface defects of omeprazole enteric-coated capsules to achieve highly robust detection of surface defects of omeprazole enteric-coated capsules.

[0005] To achieve the above object, this application adopts the following technical solutions:

[0006] In a first aspect, a method for detecting surface defects of omeprazole enteric-coated capsules is provided, which is applied to an electronic device. The method includes: the electronic device acquires M images of omeprazole enteric-coated capsules, where the M images are multiple images taken during the rotation of the omeprazole enteric-coated capsules on the same axis, and M is an integer greater than 1; the electronic device extracts the features of each of the M images through a neural network model and fuses the features of each of the M images to obtain the fused features of the M images; the electronic device processes the fused features of the M images through a neural network model to obtain a processing result, and the processing result indicates whether there are defects on the surface of the omeprazole enteric-coated capsules.

[0007] Optionally, there are multiple omeprazole enteric-coated capsules. The multiple omeprazole enteric-coated capsules are arranged along a first direction and are respectively clamped by corresponding clamping components. The first direction is the axial direction of each of the multiple omeprazole enteric-coated capsules. During the process of the motor driving the multiple omeprazole enteric-coated capsules to rotate self - clockwise along the first direction, the imaging device takes multiple shots of the multiple omeprazole enteric-coated capsules to obtain corresponding M images.

[0008] Optionally, when the motor drives the multiple omeprazole enteric-coated capsules to rotate 30° along the first direction each time, the imaging device takes one image of the multiple omeprazole enteric-coated capsules. When the motor drives the multiple omeprazole enteric-coated capsules to rotate 180° along the first direction, the rotation and shooting stop, that is, M = 6.

[0009] Optionally, the neural network model is a decoupled deployment model. The decoupled deployment model means that the neural network model includes a decoupled feature extraction sub - model and a feature processing sub - model. The electronic device extracts the features of each of the M images through the neural network model and fuses the features of each of the M images to obtain the fused features of the M images, including: The electronic device pre - processes each of the M images to obtain one picture containing only the edge region of the omeprazole enteric-coated capsule, and a total of M pictures are obtained; The electronic device inputs the M pictures into the feature extraction sub - model to obtain the feature sequences respectively extracted from each of the M pictures output by the feature extraction sub - model, and a total of M feature sequences are obtained; The electronic device classifies and fuses the M feature sequences according to the matching degree to obtain N fused feature sequences, where N is an integer greater than 1, and the N fused feature sequences are the fused features of the M images.

[0010] Optionally, the electronic device pre - processes each of the M images to obtain one picture containing only the edge region of the omeprazole enteric-coated capsule, including: The electronic device determines the pixel points located at the edge of the omeprazole enteric-coated capsule in each of the M images by performing binary or grayscale processing on each of the M images; The electronic device extracts the patterns containing only the omeprazole enteric-coated capsule part from each of the M images according to the pixel points located at the edge of the omeprazole enteric-coated capsule in each image, and a total of M patterns are obtained; The electronic device reduces the size of the contour of any one of the M patterns by a preset size to obtain a target contour; The electronic device aligns the center point of each pattern with the center point of the target contour and cuts off the part of each pattern covered by the target contour to obtain a corresponding picture, and a total of M pictures are obtained.

[0011] Optionally, each of the M feature sequences contains L elements, that is, the length of each of the M feature sequences is L, the preset sequence length is X, L is an integer greater than 3, and X is an integer greater than 1 and less than L. The electronic device classifies and fuses the M feature sequences according to the matching degree to obtain N fused feature sequences, including: The electronic device divides each of the M feature sequences into K feature subsequences according to the preset sequence length, where K = ceiling(L mod X), and ceiling() represents rounding up. A total of K * M feature subsequences are obtained; The electronic device determines the correlation between every two of the K * M feature subsequences, and divides the feature subsequences with high correlation into the same feature sequence group, obtaining a total of N feature sequence groups; The electronic device splices the feature subsequences included in each of the N feature sequence groups to obtain a corresponding fused feature sequence, obtaining a total of N fused feature sequences.

[0012] Optionally, the electronic device determines the correlation between every two of the K * M feature subsequences and divides the feature subsequences with high correlation into the same feature sequence group, including: For the i-th feature subsequence among the K * M feature subsequences, where i is an integer traversing from 1 to K * M: If the i-th feature subsequence has been divided into a feature sequence group, the electronic device does not process the i-th feature subsequence; Or, if the i-th feature subsequence has not been divided into a feature sequence group, the electronic device determines the correlation between the i-th feature subsequence and each of the (K * M) - 1 feature subsequences, obtaining (K * M) - 1 correlations. The (K * M) - 1 feature subsequences are the feature subsequences other than the i-th feature subsequence among the K * M feature subsequences; The electronic device selects the highest correlation from the (K * M) - 1 correlations. The highest correlation is the correlation between the i-th feature subsequence and the j-th feature subsequence among the (K * M) - 1 feature subsequences, where j is an integer taking values from 1 to (K * M) - 1; If the j-th feature subsequence has been divided into a feature sequence group, the i-th feature subsequence is divided into the feature sequence group to which the j-th feature subsequence belongs. If the j-th feature subsequence has not been divided into a feature sequence group, the i-th feature subsequence and the j-th feature subsequence are divided into a new feature sequence group.

[0013] Optionally, the electronic device processes the fused features of M images through a neural network model to obtain a processing result, including: The electronic device inputs the N fused feature sequences into a feature processing sub-model to obtain the processing result output by the feature processing sub-model.

[0014] Optionally, the electronic device inputs N fused feature sequences into the feature processing sub-model to obtain the processing result output by the feature processing sub-model, including: the electronic device indicates that the feature processing sub-model selects the top N classifiers with the highest performance from the classifiers of the feature processing sub-model according to the number of the N fused feature sequences being N; the electronic device sorts the N fused feature sequences in ascending order of sequence length, adds recognition elements to the top P fused feature sequences with the highest ranking to obtain P fused feature sequences with recognition elements added, and N - P fused feature sequences without recognition elements added, where P is an integer greater than or equal to 1 and less than N; the electronic device sorts the N fused feature sequences in ascending order of sequence length, and inputs the P fused feature sequences with recognition elements added and the N - P fused feature sequences without recognition elements added into the N classifiers of the feature processing sub-model one by one to obtain the processing result output by the feature processing sub-model, where when inputting into the N classifiers, the N classifiers need to be input in descending order of classifier performance.

[0015] Optionally, the electronic device sorts the N fused feature sequences in ascending order of sequence length, and adds recognition elements to the top P fused feature sequences with the highest ranking to obtain P fused feature sequences with recognition elements added, including: the electronic device sorts the N fused feature sequences in ascending order of sequence length, randomly extracts P elements from the fused feature sequence with the lowest ranking, and adds the P elements to the top P fused feature sequences with the highest ranking to obtain P fused feature sequences with recognition elements added, and the P elements are recognition elements.

[0016] In a second aspect, there is provided a device for detecting surface defects of omeprazole enteric-coated capsules, and the device is configured to: acquire M images of omeprazole enteric-coated capsules, where the M images are multiple images taken during the rotation of the omeprazole enteric-coated capsules on the same axis, and M is an integer greater than 1; extract the features of each of the M images through a neural network model, and fuse the features of each of the M images to obtain the fused features of the M images; process the fused features of the M images through a neural network model to obtain a processing result, and the processing result indicates whether there are defects on the surface of the omeprazole enteric-coated capsules.

[0017] In a third aspect, there is provided a computer-readable storage medium, including: a computer program or instruction; when the computer program or instruction runs on a computer, the computer is caused to execute the method described in the first aspect.

[0018] The above method and device have the following specific technical effects:

[0019] Considering that the omeprazole enteric-coated capsules are similar in shape to cylinders, M images can be obtained by photographing the omeprazole enteric-coated capsules during the rotation process on the same axis. The electronic device can acquire these M images, extract the features of each of the M images through a neural network model, and then fuse the features of each of the M images to obtain the fused features of the M images. At this time, since the features are fused, the neural network model can perform an overall analysis on the M images, enabling the processing result output by the neural network model to accurately indicate whether there are defects on the surface of the omeprazole enteric-coated capsules, that is, realizing the surface defect detection of omeprazole enteric-coated capsules with high robustness. Brief Description of the Drawings

[0020] Figure 1 It is a schematic flow chart of a method for detecting surface defects of omeprazole enteric-coated capsules provided by an embodiment of the present application;

[0021] Figure 2 It is a schematic diagram of photographing an omeprazole enteric-coated capsule in an embodiment of the present application;

[0022] Figure 3 It is a schematic structural diagram of a neural network model in an embodiment of the present application;

[0023] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0024] Next, the technical solutions in the present application will be described in conjunction with the accompanying drawings.

[0025] The present application will present various aspects, embodiments or features around a system that may include multiple devices, components, modules, etc. It should be understood and clear that each system may include additional devices, components, modules, etc., and / or may not include all the devices, components, modules, etc. discussed in conjunction with the accompanying drawings. In addition, combinations of these solutions can also be used.

[0026] In addition, in the embodiments of the present application, words such as "exemplary" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of the word "exemplary" is intended to present concepts in a specific way.

[0027] The network architecture and business scenarios described in the embodiments of the present application are for more clearly explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those of ordinary skill in the art can know that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0028] To facilitate the understanding of the embodiments of the present application, first, Figure 2 taking the intelligent photovoltaic power energy storage system based on the Internet of Things shown in

[0029] as an example, the intelligent photovoltaic power energy storage system based on the Internet of Things applicable to the embodiments of the present application will be described in detail. Figure 1 For ease of understanding, the method for detecting surface defects of omeprazole enteric-coated capsules provided by the embodiments of the present application will be specifically elaborated below.

[0030] Exemplarily, Figure 1 the flowchart of the method for detecting surface defects of omeprazole enteric-coated capsules provided by the embodiments of the present application. This method can be applicable to an electronic device, that is, executed by the electronic device.

[0031] The electronic device can specifically be a terminal. A terminal can also be referred to as a terminal device, user equipment (UE), mobile station, mobile terminal, etc. Terminals can be widely applied to various scenarios. For example, device-to-device (D2D), vehicle to everything (V2X) communication, machine-type communication (MTC), Internet of Things (IOT), virtual reality, augmented reality, industrial control, autonomous driving, remote medical treatment, smart grid, smart furniture, smart office, smart wearables, smart transportation, smart city, etc. The terminal can be a mobile phone, a tablet computer, a computer with wireless transceiver function, a wearable device, etc. The embodiments of the present application do not limit the device form of the terminal.

[0032] As Figure 1 shown, the process of this method is as follows:

[0033] S101, the electronic device acquires M images of omeprazole enteric-coated capsules.

[0034] The M images are multiple images taken during the rotation of the omeprazole enteric-coated capsule on the same axis. M is an integer greater than 1. For example, as Figure 2As shown, there are multiple omeprazole enteric-coated capsules. The multiple omeprazole enteric-coated capsules are arranged in the first direction and are clamped by their respective corresponding clamping components. The first direction is the axial direction of each of the multiple omeprazole enteric-coated capsules. During the process of the motor driving the multiple omeprazole enteric-coated capsules to rotate self - clockwise in the first direction, the imaging device takes multiple pictures of the multiple omeprazole enteric-coated capsules to obtain corresponding M images. Specifically, when the motor drives the multiple omeprazole enteric-coated capsules to rotate 30° in the first direction, the imaging device takes one picture of the multiple omeprazole enteric-coated capsules. When the motor drives the multiple omeprazole enteric-coated capsules to rotate 180° in the first direction, the rotation and shooting stop, that is, M = 6.

[0035] It should be understood that, as Figure 2 shown, the omeprazole enteric-coated capsules can also be arranged in multiple rows, that is, each row is arranged in the above - mentioned manner, and the rotation and shooting are carried out in the above - mentioned manner, so that the images taken at one time can contain a larger number of omeprazole enteric-coated capsules. In one example, there can be 12 omeprazole enteric-coated capsules in one row, and the number of rows can be 6 or 8, that is, surface defect detection of 72 or 96 omeprazole enteric-coated capsules can be achieved at one time. In practical applications, the omeprazole enteric-coated capsules can be conveyed to the positioning device by a conveyor belt, and the positioning device sorts and respectively sets them on the corresponding clamping components, thereby improving the subsequent detection efficiency.

[0036] It can be understood that the purpose of the above rotation and shooting is to enable the subsequent processing of the neural network model to analyze the surface characteristics of the omeprazole enteric-coated capsules, such as whether there are uneven or easily cracked areas on the surface, that is, to achieve surface defect detection.

[0037] S202, the electronic device extracts the features of each of the M images through the neural network model and fuses the features of each of the M images to obtain the fused features of the M images.

[0038] The neural network model can be a deep neural network (DNN), specifically a DNN with an improved structure. For example, the neural network model is a decoupled deployment model, and a decoupled deployment model means that the neural network model includes a decoupled feature extraction sub-model, an intermediate processing sub-model, and a feature processing sub-model. Among them, the feature extraction sub-model includes an input layer and a feature extraction layer. The input layer is understood as the data transmission channel from the input port to the feature extraction layer, and the feature extraction layer can perform convolutional operations. The feature processing sub-model can include an input layer, multiple classifiers, and an output layer. The input layer is understood as the feature transmission channel from the input port to the multiple classifiers. Each classifier in the multiple classifiers contains a certain number of neurons. Each neuron performs a weighted sum operation on its input value and generates an output through a non-linear function, that is, the process of feature processing. The output layer can perform deconvolution operations. The decoupling of the feature extraction sub-model and the feature processing sub-model means that the features output by the feature extraction sub-model are not directly input into the feature processing sub-model within the model, but are output to the intermediate processing sub-model, and then input into the feature processing sub-model after being further processed by the intermediate processing sub-model. The following is a specific introduction.

[0039] The electronic device can preprocess each of the M images to obtain one picture containing only the edge region of the omeprazole enteric-coated capsules, and a total of M pictures are obtained.

[0040] For example, the electronic device can perform binarization or grayscale processing on each of the M images to determine the pixel points located at the edge of the omeprazole enteric-coated capsules in each image, that is, the pixel points with relatively large pixel differences after binarization or grayscale processing are the pixel points located at the edge of the omeprazole enteric-coated capsules, so as to determine the edge of the omeprazole enteric-coated capsules in each image. In this way, the electronic device extracts the patterns containing only the omeprazole enteric-coated capsule parts from each image according to the pixel points located at the edge of the omeprazole enteric-coated capsules in each image, and a total of M patterns are obtained (each pattern can contain the patterns of multiple omeprazole enteric-coated capsules contained in the image. In other words, the pattern is not a complete pattern, but a set of patterns of multiple omeprazole enteric-coated capsules), that is, the background elements in the image are stripped. On the one hand, it is convenient for subsequent centralized extraction and analysis of the features located on the surface of the omeprazole enteric-coated capsules. On the other hand, it can also reduce the amount of data processing and improve the detection efficiency.

[0041] The electronic device can reduce the size of the outline of any one of the M patterns by a preset size (for example, the preset size can be 2 / 3 of the original size) to obtain a target outline, so as to align the center point of each pattern with the center point of the target outline, and cut off the part of each pattern covered by the target outline to obtain a corresponding picture, and a total of M pictures are obtained. In other words, for each enteric-coated omeprazole capsule, it is rotated for shooting, and the edge outline of each enteric-coated omeprazole capsule in the obtained image can reflect the corresponding area on the surface of the enteric-coated omeprazole capsule. The outlines in different images can reflect different areas on the surface of the enteric-coated omeprazole capsule, and the M images can basically cover the surface of the enteric-coated omeprazole capsule through the areas near the edge outline of the enteric-coated omeprazole capsule in the images. Therefore, for this special shooting method, for the pattern of the enteric-coated omeprazole capsule in any one image, the subsequent neural network model only needs to analyze the edge area of the pattern, and there is no need to analyze the central area of the pattern. Therefore, the central area can be cut off in the above manner, that is, the pattern of each enteric-coated omeprazole capsule in each finally obtained picture is a pattern similar to a ring, which can also be called a quasi-circular pattern. In addition, each picture can contain the quasi-circular patterns of multiple enteric-coated omeprazole capsules included in the image. In other words, the picture is not a complete picture, but a set of quasi-circular patterns of multiple enteric-coated omeprazole capsules, that is, a set of quasi-circular patterns.

[0042] The electronic device can input the M pictures into the feature extraction sub-model to obtain the feature sequences corresponding to each of the M pictures extracted by the feature extraction sub-model, and a total of M feature sequences are obtained. That is, the feature sequences are obtained by convolution extraction, and the size of the convolution for each picture is the same. Therefore, the length of each feature sequence is also fixed.

[0043] The electronic device (or the electronic device through the intermediate processing sub-model) can classify and fuse the M feature sequences according to the matching degree to obtain N fused feature sequences, where N is an integer greater than 1, and the N fused feature sequences are the fused features of the M images.

[0044] For example, each of the M feature sequences contains L elements, that is, the length of each of the M feature sequences is L, the preset sequence length is X, L is an integer greater than 3, and X is an integer greater than 1 and less than L. On this basis,

[0045] The electronic device divides each of the M feature sequences into K feature subsequences according to a preset sequence length, where K = ceiling(L mod X), and ceiling() represents rounding up. A total of K * M feature subsequences are obtained. Among them, the size relationship between X and L needs to meet certain conditions. For example, X is basically 1 / 9 - 1 / 10 of L to ensure that the number of features in each feature subsequence is moderate, which is convenient for subsequent feature classification to extract the features of the defective part. The electronic device can determine the correlation (such as inner product, Euclidean distance, etc.) between every two of the K * M feature subsequences, and divide the feature subsequences with high correlation into the same feature sequence group, obtaining a total of N feature sequence groups.

[0046] Specifically, for the i-th feature subsequence among the K * M feature subsequences, where i is an integer ranging from 1 to K * M: If the i-th feature subsequence has been assigned to a feature sequence group, the electronic device does not process the i-th feature subsequence; or, if the i-th feature subsequence has not been assigned to a feature sequence group, the electronic device determines the correlation between the i-th feature subsequence and each of the (K * M) - 1 feature subsequences, obtaining (K * M) - 1 correlations. The (K * M) - 1 feature subsequences are the feature subsequences among the K * M feature subsequences except the i-th feature subsequence. The electronic device selects the highest correlation from the (K * M) - 1 correlations. The highest correlation is the correlation between the i-th feature subsequence and the j-th feature subsequence among the (K * M) - 1 feature subsequences, where j is an integer ranging from 1 to (K * M) - 1. If the j-th feature subsequence has been assigned to a feature sequence group, the i-th feature subsequence is assigned to the feature sequence group to which the j-th feature subsequence belongs. If the j-th feature subsequence has not been assigned to a feature sequence group, the i-th feature subsequence and the j-th feature subsequence are assigned to a new feature sequence group.

[0047] Exemplarily, assume that K*M = 10, that is, it includes feature subsequences 1 to 10. The electronic device calculates the correlations between feature subsequence 1 and feature subsequences 2 to 10 respectively, so as to determine that the correlation between feature subsequence 1 and feature subsequence 3 is the highest, and generates feature sequence group 1, which includes feature subsequence 1 and feature subsequence 3. After that, the electronic device calculates the correlations between feature subsequence 2 and feature subsequences 1, 3 to 10 respectively (or it can also only calculate the correlations between feature subsequence 2 and feature subsequences 3 to 10), so as to determine that the correlation between feature subsequence 2 and feature subsequence 3 is the highest, and divides feature subsequence 2 into feature sequence group 1. After that, the electronic device calculates the correlations between feature subsequence 4 and feature subsequences 1 to 3, and feature subsequences 5 to 10 respectively (or it can also only calculate the correlations between feature subsequence 4 and feature subsequences 3, 5 to 10), so as to determine that the correlation between feature subsequence 4 and feature subsequence 6 is the highest, and generates feature sequence group 2, which includes feature subsequence 4 and feature subsequence 6. After that, and so on.

[0048] It should be understood that by calculating the correlations, the feature sequences extracted from the same type of defect can be divided into the same group, and the feature sequences extracted without defects can be divided into other groups, which is convenient for subsequent processing. Of course, at this time, it is not yet determined which feature sequences are extracted from defects.

[0049] The electronic device splices the feature subsequences included in each feature sequence group among the N feature sequence groups to obtain a corresponding fused feature sequence, and a total of N fused feature sequences are obtained. The embodiments of the present application do not limit the splicing order.

[0050] S203, the electronic device processes the fused features of M images through a neural network model to obtain a processing result.

[0051] The processing result indicates whether there are defects on the surface of the omeprazole enteric-coated capsules. For example, the processing result may include M marked images, and the marks may be located at the surface defect positions of the omeprazole enteric-coated capsules in the images, that is, it indicates which position on the surface of the omeprazole enteric-coated capsules has defects.

[0052] For example, the electronic device inputs the N fused feature sequences into the feature processing sub-model to obtain the processing result output by the feature processing sub-model.

[0053] Specifically, the electronic device can select N classifiers with the top N high performance rankings from the classifiers of the feature processing sub-model according to the number of N fused feature sequences being N, that is, the N classifiers with the top N most neurons. The electronic device can sort the N fused feature sequences in ascending order of sequence length, add recognition elements to the top P high-ranked fused feature sequences to obtain P fused feature sequences with recognition elements added, and N - P fused feature sequences without recognition elements added, where P is an integer greater than or equal to 1 and less than N. For example, the electronic device can sort the N fused feature sequences in ascending order of sequence length, randomly extract P elements from the fused feature sequence with the lowest ranking, and add the P elements to the top P high-ranked fused feature sequences to obtain P fused feature sequences with recognition elements added, and the P elements are recognition elements. The purpose of this is to facilitate the subsequent analysis of elements by the classifier, and the slight differences between elements can assist the classifier in identifying whether the feature sequence is a feature sequence extracted from a defect. The electronic device sorts the N fused feature sequences in ascending order of sequence length (that is, the defect is usually relatively small, so the features are also relatively few, and a shorter feature sequence means a higher probability of being extracted from a defect and requires a classifier with better performance to process), and inputs the P fused feature sequences with recognition elements added and the N - P fused feature sequences without recognition elements added into the N classifiers of the feature processing sub-model one by one to obtain the processing results output by the feature processing sub-model. Among them, the N classifiers can output N analysis results one by one, and each analysis result can indicate whether a corresponding fused feature sequence is extracted from a defect, that is, identify the feature sequence extracted from a defect. Among them, when inputting the N classifiers, the N classifiers need to be input in descending order of classifier performance.

[0054] It should be understood that the output layer can mark the feature sequences extracted from the defect according to the N analysis results. Then the output layer can obtain the above-mentioned fusion process information from the above intermediate processing sub-model, execute the reverse order of the above fusion, restore to obtain M feature sequences with marks, and then perform deconvolution on the M feature sequences with marks to obtain M marked images, that is, the processing results.

[0055] In summary, considering that the omeprazole enteric-coated capsule is similar in shape to a column, M images can be obtained by photographing the omeprazole enteric-coated capsule during the rotation on the same axis. The electronic device can acquire these M images, extract the features of each of the M images through a neural network model, and then fuse the features of each of the M images to obtain the fused features of the M images. At this time, since the features are fused, the neural network model can perform an overall analysis on the M images, enabling the processing result output by the neural network model to accurately indicate whether there are defects on the surface of the omeprazole enteric-coated capsule, that is, realizing the surface defect detection of the omeprazole enteric-coated capsule with high robustness.

[0056] The above combination Figure 1 has described in detail the method for detecting surface defects of omeprazole enteric-coated capsules provided in the embodiments of the present application. The following will describe the device for detecting surface defects of omeprazole enteric-coated capsules for executing the above method.

[0057] The device is configured to: acquire M images of the omeprazole enteric-coated capsule, where the M images are multiple images taken during the rotation of the omeprazole enteric-coated capsule on the same axis, and M is an integer greater than 1; extract the features of each of the M images through a neural network model, and fuse the features of each of the M images to obtain the fused features of the M images; process the fused features of the M images through a neural network model to obtain a processing result, and the processing result indicates whether there are defects on the surface of the omeprazole enteric-coated capsule.

[0058] Figure 4 FIG. is a schematic structural diagram of an electronic device provided in an embodiment of the present application. Exemplarily, the electronic device may be a terminal device, or a chip (system) or other components or assemblies that can be disposed in a terminal device. As Figure 4 shown, the electronic device 400 may include a processor 401. Optionally, the electronic device 400 may further include a memory 402 and / or a transceiver 403. Among them, the processor 401 is coupled to the memory 402 and the transceiver 403, and may be connected through a communication bus, for example. In addition, the electronic device 400 may also be a chip, such as including a processor 401. At this time, the transceiver may be an input / output interface of the chip.

[0059] Next, in combination with Figure 4 a specific introduction will be made to each component of the electronic device 400:

[0060] Among them, the processor 401 is the control center of the electronic device 400, which can be a single processor or a collective term for multiple processing elements. For example, the processor 401 is one or more central processing units (CPUs), or can be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0061] Optionally, the processor 401 can execute various functions of the electronic device 400 by running or executing software programs stored in the memory 402 and calling data stored in the memory 402, such as executing the above-mentioned Figure 4 method for detecting surface defects of omeprazole enteric capsules shown.

[0062] In a specific implementation, as an embodiment, the processor 401 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 shown in

[0063] In a specific implementation, as an embodiment, the electronic device 400 may also include multiple processors. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, the processor can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer programs or instructions).

[0064] Among them, the memory 402 is used to store software programs for executing the solution of the present application and is controlled by the processor 401 for execution. The specific implementation method can refer to the above method embodiments and will not be elaborated here.

[0065] Optionally, the memory 402 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or any other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 402 may be integrated with the processor 401 or may exist independently and be coupled to the processor 401 through the interface circuit of the electronic device 400 ( Figure 4 not shown in the figure), and the embodiments of the present application do not make specific limitations on this.

[0066] The transceiver 403 is used for communication with other electronic devices. For example, when the electronic device 400 is a terminal device, the transceiver 403 may be used for communication with a network device or with another terminal device. For another example, when the electronic device 400 is a network device, the transceiver 403 may be used for communication with a terminal device or with another network device.

[0067] Optionally, the transceiver 403 may include a receiver and a transmitter ( Figure 4 not shown separately). Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0068] Optionally, the transceiver 403 may be integrated with the processor 401 or may exist independently and be coupled to the processor 401 through the interface circuit of the electronic device 400 ( Figure 4 not shown in the figure), and the embodiments of the present application do not make specific limitations on this.

[0069] It can be understood that Figure 4 the structure of the electronic device 400 shown in the figure does not constitute a limitation on the electronic device. The actual electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component layout.

[0070] In addition, the technical effects of the electronic device 400 may refer to the technical effects of the method described in the above method embodiments, and will not be elaborated here.

[0071] It should be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0072] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0073] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer program or instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer program or instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, or magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0074] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. Additionally, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood by referring to the context before and after.

[0075] In this application, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0076] It should be understood that in various embodiments of the present application, the magnitudes of the sequence numbers of the above processes do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0077] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled professionals may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0078] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0079] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0080] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0081] In addition, the functional units in each embodiment of this application can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0082] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0083] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claimed rights.

Claims

1. A method for detecting surface defects of omeprazole enteric-coated capsules, characterized in that: Applied to electronic equipment, the method comprises: The electronic device acquires M images of the omeprazole enteric-coated capsule, wherein the M images are multiple images taken during the process of the omeprazole enteric-coated capsule rotating on the same axis, and M is an integer greater than 1; The electronic device extracts features of each of the M images through a neural network model, and fuses the features of each of the M images to obtain fused features of the M images; The electronic device processes the fusion features of the M images through the neural network model to obtain a processing result, and the processing result indicates whether there are defects on the surface of the omeprazole enteric-coated capsule.

2. The method according to claim 1, characterized in that There are multiple omeprazole enteric-coated capsules, which are arranged along a first direction and clamped by corresponding clamping components. The first direction is the axis direction of each of the multiple omeprazole enteric-coated capsules. When the motor drives the multiple omeprazole enteric-coated capsules to rotate along the first direction, the shooting device shoots the multiple omeprazole enteric-coated capsules multiple times to obtain the corresponding M images.

3. The method according to claim 2, characterized in that When the motor drives the multiple omeprazole enteric-coated capsules to rotate 30° along the first direction, the shooting device takes an image of the multiple omeprazole enteric-coated capsules. When the motor drives the multiple omeprazole enteric-coated capsules to rotate 180° along the first direction, the rotation and shooting are stopped, that is, M=6.

4. The method according to any one of claims 1 to 3, characterized in that: The neural network model is a decoupled deployment model, which means that the neural network model includes a decoupled feature extraction sub-model and a feature processing sub-model. The electronic device extracts the features of each of the M images through the neural network model, and fuses the features of each of the M images to obtain the fused features of the M images, including: The electronic device preprocesses each of the M images to obtain a picture containing only the edge region of the omeprazole enteric-coated capsule, and obtains the M pictures in total; The electronic device inputs the M pictures into the feature extraction sub-model, obtains a feature sequence corresponding to each picture in the M pictures output by the feature extraction sub-model, and obtains a total of M feature sequences; The electronic device classifies and fuses the M feature sequences according to the matching degree to obtain N fused feature sequences, where N is an integer greater than 1, and the N fused feature sequences are the fused features of the M images.

5. The method according to claim 4, characterized in that The electronic device preprocesses each of the M images to obtain an image containing only the edge region of the omeprazole enteric-coated capsule, including: The electronic device determines the pixel points located at the edge of the omeprazole enteric-coated capsule in each image by performing binarization or grayscale processing on each of the M images; The electronic device extracts a pattern including only the portion of the omeprazole enteric-coated capsule from each image according to the pixel points located at the edge of the omeprazole enteric-coated capsule in each image, and obtains M patterns in total; The electronic device reduces the size of the outline of any one of the M patterns by a preset size to obtain a target outline; The electronic device aligns the center point of each pattern with the center point of the target outline, cuts off the portion of each pattern covered by the target outline, and obtains a corresponding picture, thereby obtaining the M pictures in total.

6. The method according to claim 4, characterized in that Each of the M feature sequences contains L elements, that is, the length of each of the M feature sequences is L, the preset sequence length is X, L is an integer greater than 3, X is an integer greater than 1 and less than L, and the electronic device classifies and fuses the M feature sequences according to the matching degree to obtain N fused feature sequences, including: The electronic device divides each of the M feature sequences into K feature subsequences according to the preset sequence length, where K=ceiling(LmodX), and ceiling() represents rounding up, to obtain K*M feature subsequences in total; The electronic device determines the correlation between every two feature subsequences in the K*M feature subsequences, and divides the feature subsequences with high correlation into the same feature sequence group, thereby obtaining N feature sequence groups in total; The electronic device splices the characteristic subsequences contained in each characteristic sequence group in the N characteristic sequence groups to obtain a corresponding fused characteristic sequence, thereby obtaining the N fused characteristic sequences in total.

7. The method according to claim 6, characterized in that The electronic device determines the correlation between every two feature subsequences in the K*M feature subsequences, and classifies the feature subsequences with high correlation into the same feature sequence group, including: For the i-th characteristic subsequence among the K*M characteristic subsequences, i is an integer ranging from 1 to K*M: If the i-th characteristic subsequence has been classified into a characteristic sequence group, the electronic device does not process the i-th characteristic subsequence; or, If the i-th characteristic subsequence is not classified into a characteristic sequence group, the electronic device determines the correlation between the i-th characteristic subsequence and each of (K*M)-1 characteristic subsequences to obtain (K*M)-1 correlations, and the (K*M)-1 characteristic subsequences are characteristic subsequences other than the i-th characteristic subsequence in the K*M characteristic subsequences; The electronic device selects a highest correlation from the (K*M)-1 correlations, where the highest correlation is a correlation between the i-th characteristic subsequence and the j-th characteristic subsequence in the (K*M)-1 characteristic subsequences, where j is an integer ranging from 1 to (K*M)-1; If the j-th feature subsequence has been classified into a feature sequence group, the i-th feature subsequence is classified into the feature sequence group to which the j-th feature subsequence belongs; if the j-th feature subsequence has not been classified into a feature sequence group, the i-th feature subsequence and the j-th feature subsequence are classified into a new feature sequence group.

8. The method according to claim 7, characterized in that The electronic device processes the fusion features of the M images through the neural network model to obtain a processing result, including: The electronic device inputs the N fused feature sequences into the feature processing sub-model to obtain the processing result output by the feature processing sub-model.

9. The method according to claim 8, characterized in that The electronic device inputs the N fused feature sequences into the feature processing sub-model to obtain the processing result output by the feature processing sub-model, including: The electronic device instructs the feature processing sub-model to select N classifiers with top N performance rankings from the classifiers of the feature processing sub-model according to the number of the N fusion feature sequences being N; The electronic device sorts the N fused feature sequences from short to long according to the sequence length, adds identification elements to the top P fused feature sequences, and obtains P fused feature sequences with added identification elements, and NP fused feature sequences without added identification elements, where P is an integer greater than or equal to 1 and less than N; The electronic device sorts the N fused feature sequences according to sequence length from short to long, and inputs the P fused feature sequences with added identification elements and the NP fused feature sequences without added identification elements into the N classifiers of the feature processing sub-model one by one to obtain the processing results output by the feature processing sub-model, wherein when inputting the N classifiers, the N classifiers need to be input in order from high to low according to the performance of the classifier.

10. The method according to claim 9, characterized in that The electronic device sorts the N fused feature sequences from short to long according to the sequence length, adds identification elements to the top P fused feature sequences with the highest ranking, and obtains P fused feature sequences with the added identification elements, including: The electronic device sorts the N fused feature sequences from short to long according to sequence length, randomly extracts P elements from the lowest-ranked fused feature sequence, and adds the P elements to the top P fused feature sequences, to obtain the P fused feature sequences with added identification elements, and the P elements are the identification elements.

Images