Quality testing methods and related equipment for capsule medicines

Abstract

The embodiment of the present invention provides a method for quality inspection of capsule medicines, including: obtaining a first depth image and a second depth image of a capsule to be inspected, the first depth image is acquired by a first depth camera set at a first designated position, and the second depth image is acquired by a second depth camera set at a second designated position; obtaining a lighting map of the capsule to be inspected, the lighting map is a lighting map of the capsule to be inspected on one horizontal side or on both horizontal sides; constructing a three-dimensional image of the capsule to be inspected based on the first depth image and the second depth image; and performing quality inspection on the capsule to be inspected based on the three-dimensional image and the lighting map to obtain a quality inspection result of the capsule to be inspected. The capsule can be constructed in three dimensions through the depth images collected at different positions to obtain a three-dimensional image of the capsule, and the quality of the capsule can be inspected in combination with the lighting map of the capsule, without the need for manual inspection, thereby improving the efficiency of capsule quality inspection.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and smart medical care, and in particular to a quality inspection method for capsule medicines and related equipment. Background Art

[0002] Capsules are a form of medicine that is formed by encapsulating drug ingredients with edible glue. After the capsule enters the human body, the edible glue melts to release the drug, which helps the human body fully absorb it. At the same time, the emergence of capsules solves the problem of poor taste when taking medicine, making the drug have a good absorption effect. During the production process of capsules, they need to be tested to ensure the quality of the capsules. The existing testing methods require manual testing, with a low degree of automation, and cannot be tested in batches, making the quality testing efficiency of capsules low. Summary of the invention

[0003] The embodiment of the present invention provides a method and related equipment for detecting the quality of capsule medicines. The method can construct a capsule in three dimensions through depth images collected at different positions to obtain a three-dimensional image of the capsule, and detect the quality of the capsule in combination with a light map of the capsule. Manual inspection is not required, thereby improving the efficiency of capsule quality inspection.

[0004] In a first aspect, an embodiment of the present invention provides a method for quality inspection of a capsule medicine, the method comprising:

[0005] Acquire a first depth image and a second depth image of the capsule to be detected, wherein the first depth image is acquired by a first depth camera set at a first specified position, and the second depth image is acquired by a second depth camera set at a second specified position;

[0006] Acquire a light map of the capsule to be detected, where the light map is a light map of the capsule to be detected on one horizontal side or on both horizontal sides;

[0007] constructing a three-dimensional image of the capsule to be detected according to the first depth image and the second depth image;

[0008] Based on the three-dimensional image and the illumination map, a quality inspection is performed on the capsule to be inspected to obtain a quality inspection result of the capsule to be inspected.

[0009] Optionally, constructing the three-dimensional image of the capsule to be detected according to the first depth image and the second depth image includes:

[0010] Extracting first depth information and first color information of the first depth image, and extracting second depth information and second color information of the second depth image;

[0011] performing background removal on the first depth image according to the first depth information and the first color information to obtain a first capsule foreground, and performing background removal on the second depth image according to the second depth information and the second color information to obtain a second capsule foreground;

[0012] The first capsule foreground and the second capsule foreground are subjected to three-dimensional construction processing to obtain a three-dimensional image of the capsule to be detected.

[0013] Optionally, the extracting the first depth information and the first color information of the first depth image, and extracting the second depth information and the second color information of the second depth image includes:

[0014] Performing target detection on the first depth image, and extracting a first candidate region image from the first depth image according to the obtained target detection result, wherein the first candidate region image includes the capsule to be detected;

[0015] Performing a scale transformation process on the first candidate region image to obtain a processed first candidate region image, wherein the processed first candidate region image has the same size as the first depth image;

[0016] Inputting the first candidate area image and the first depth image into an information extraction model to extract information, so as to obtain first depth information and first color information of the first depth image;

[0017] Performing target detection on the second depth image, and extracting a second candidate region image from the second depth image according to the obtained target detection result, wherein the second candidate region image includes the capsule to be detected;

[0018] Performing a scale transformation process on the second candidate region image to obtain a processed second candidate region image, wherein the processed second candidate region image has the same size as the second depth image;

[0019] The second candidate area image and the second depth image are input into the information extraction model to extract information, so as to obtain second depth information and second color information of the second depth image.

[0020] Optionally, performing three-dimensional construction processing on the first capsule foreground and the second capsule foreground to obtain a three-dimensional image of the capsule to be detected includes:

[0021] A three-dimensional construction process is performed according to the depth value of the first capsule foreground and the depth value of the second capsule foreground to obtain a three-dimensional image of the capsule to be detected.

[0022] Optionally, before the step of performing three-dimensional construction processing according to the depth value of the first capsule foreground and the depth value of the second capsule foreground to obtain the three-dimensional image of the capsule to be detected, the method includes:

[0023] The depth value of the first capsule foreground and the depth value of the second capsule foreground are repaired.

[0024] Optionally, the quality inspection result includes appearance quality and internal quality, and the quality inspection of the capsule to be inspected based on the three-dimensional image and the illumination map to obtain the quality inspection result of the capsule to be inspected includes:

[0025] The three-dimensional image and the illumination map are subjected to quality inspection by a preset quality inspection model to obtain the appearance quality and internal quality of the capsule to be inspected.

[0026] Optionally, the quality detection model includes a three-dimensional convolution module, a two-dimensional convolution module, a fusion module and a decoding module, and the quality detection of the three-dimensional image and the illumination map is performed by a preset quality detection model to obtain the appearance quality and internal quality of the capsule to be detected, including:

[0027] Inputting the three-dimensional image into the three-dimensional convolution module for processing to obtain a first intermediate feature of the three-dimensional image;

[0028] Inputting the illumination map into the two-dimensional convolution module for processing to obtain a second intermediate feature of the two-dimensional image;

[0029] Inputting the first intermediate feature and the second intermediate feature into the fusion module for feature fusion to obtain fusion features of the three-dimensional image and the illumination map;

[0030] The fusion feature is input into the decoding module for decoding processing to obtain the appearance quality and internal quality of the capsule to be detected.

[0031] In a second aspect, an embodiment of the present invention provides a device for detecting the quality of a capsule medicine, the device comprising:

[0032] A first acquisition module is used to acquire a first depth image and a second depth image of the capsule to be detected, wherein the first depth image is acquired by a first depth camera set at a first specified position, and the second depth image is acquired by a second depth camera set at a second specified position;

[0033] A second acquisition module is used to acquire a light map of the capsule to be detected, where the light map is a light map of the capsule to be detected on one horizontal side or on both horizontal sides;

[0034] A construction module, constructing a three-dimensional image of the capsule to be detected according to the first depth image and the second depth image;

[0035] The detection module is used to perform quality detection on the capsule to be detected based on the three-dimensional image and the illumination map to obtain a quality detection result of the capsule to be detected.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for quality inspection of capsule medicines provided in an embodiment of the present invention are implemented.

[0037] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for quality inspection of capsule medicines provided in an embodiment of the invention are implemented.

[0038] In an embodiment of the present invention, a first depth image and a second depth image of a capsule to be detected are obtained, wherein the first depth image is acquired by a first depth camera set at a first designated position, and the second depth image is acquired by a second depth camera set at a second designated position; an illumination map of the capsule to be detected is obtained, wherein the illumination map is an illumination map of the capsule to be detected on one horizontal side or on both horizontal sides; a three-dimensional image of the capsule to be detected is constructed based on the first depth image and the second depth image; and a quality inspection of the capsule to be detected is performed based on the three-dimensional image and the illumination map to obtain a quality inspection result of the capsule to be detected. The capsule can be constructed in three dimensions through depth images acquired at different positions to obtain a three-dimensional image of the capsule, and the quality of the capsule can be inspected in combination with the illumination map of the capsule, without the need for manual inspection, thereby improving the efficiency of capsule quality inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 It is a flow chart of a method for quality inspection of capsule medicines provided by an embodiment of the present invention;

[0041] Figure 2 It is a structural schematic diagram of a quality inspection device for capsule medicines provided by an embodiment of the present invention;

[0042] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for quality inspection of capsule medicines provided by an embodiment of the present invention. Figure 1 As shown, the quality inspection method of capsule medicine comprises the following steps:

[0045] 101. Obtain a first depth image and a second depth image of a capsule to be detected.

[0046] In an embodiment of the present invention, the first depth image is acquired by a first depth camera disposed at a first designated position, and the second depth image is acquired by a second depth camera disposed at a second designated position.

[0047] The above-mentioned depth camera can also be called a 3D camera. As a stereoscopic vision sensor and three-dimensional depth perception module, the 3D camera can obtain high-resolution, high-precision, low-latency depth and RGB video streams in real time, generate 3D images, and be used for real-time target recognition, motion capture or scene perception of three-dimensional images. The depth information is stable and reliable, and is not affected by ambient light. The depth point cloud and RGB pixels correspond one to one.

[0048] In an embodiment of the present invention, the first designated position and the second designated position may be on opposite sides of the capsule to be detected, and the first depth camera and the second depth camera are horizontally arranged relative to the capsule detection line so that the center line of the capsule to be detected in the horizontal direction is located at the horizontal center line of the field of view of the first depth camera and the second depth camera. The capsule detection line can be understood as a conveyor belt for transporting the capsule to be detected for detection.

[0049] The depth image of one side of the capsule to be detected is captured by the first depth camera, and the depth image of the other side of the capsule to be detected is captured by the second depth camera. This is equivalent to the first depth image including the image of one side of the capsule to be detected, and the second depth image including the image of the other side of the capsule to be detected.

[0050] The first depth image and the second depth image include not only the collected pixel information but also the depth information of each pixel.

[0051] 102. Obtain a lighting map of the capsule to be detected.

[0052] In the embodiment of the present invention, the above-mentioned illumination image is an illumination image of the capsule to be detected on one horizontal side or on both horizontal sides. The capsule to be detected can be illuminated by arranging a spotlight on the opposite side of the camera, so that the camera can collect the light transmission information of the capsule to be detected.

[0053] In the embodiment of the present invention, after obtaining the illumination map of the capsule to be detected, the illumination map can be segmented to separate the capsule to be detected from the light background. Specifically, an image segmentation network can be used to segment the illumination map of the capsule to be detected.

[0054] In a possible embodiment, the illumination map is the illumination map of the capsule to be detected on both sides horizontally, so that the light transmission information of the capsule to be detected on both sides can be collected. In the case where the illumination map is the illumination map of the capsule to be detected on both sides horizontally, the illumination map can be marked to distinguish the orientation of the illumination map.

[0055] When the illumination map is the illumination map of the capsule to be detected on both sides horizontally, there are two illumination maps, and the shooting time of the two illumination maps is staggered. This is because when collecting the illumination map of one side, it is avoided to be affected by the light source on the other side. For example, the left spotlight and the right spotlight are set at staggered positions on the capsule detection line. When the capsule to be detected is transported on the capsule detection line, it first passes through the left spotlight area, and the capsule to be detected is illuminated by the left light, and the illumination map of the capsule to be detected is collected through the right camera. Then it passes through the right spotlight area, and the illumination map of the capsule to be detected is collected through the left camera.

[0056] 103. Construct a three-dimensional image of the capsule to be detected according to the first depth image and the second depth image.

[0057] In the embodiment of the present invention, the first depth image contains depth information of one side of the capsule to be detected, so that a three-dimensional image of the side can be constructed based on the depth information of the side. The second depth image contains depth information of the other side of the capsule to be detected, so that a three-dimensional image of the other side can be constructed based on the depth information of the other side, and the three-dimensional images of the two sides can be aligned and spliced ​​to obtain a three-dimensional image of the capsule to be detected.

[0058] Optionally, in the step of constructing a three-dimensional image of the capsule to be detected based on the first depth image and the second depth image, the first depth information and the first color information of the first depth image can be extracted, and the second depth information and the second color information of the second depth image can be extracted; the background of the first depth image is removed according to the first depth information and the first color information to obtain a first capsule foreground, and the background of the second depth image is removed according to the second depth information and the second color information to obtain a second capsule foreground; the first capsule foreground and the second capsule foreground are three-dimensionally constructed to obtain a three-dimensional image of the capsule to be detected.

[0059] In an embodiment of the present invention, information extraction may be performed on the first depth image using a preset information extraction model to obtain first depth information and first color information; and information extraction may be performed on the second depth image using the preset information extraction model to obtain second depth information and second color information.

[0060] The above-mentioned information extraction layer model may include a backbone network, a first branch network and a second branch network. The backbone network is used to extract common features of color and depth, the first branch network is used to extract depth information from the common features, and the second branch network is used to extract color information from the common features. The backbone network may include multiple information extraction layers, the first branch network may also include multiple information extraction layers, and the second branch network may also include multiple information extraction layers. Furthermore, when the color of the capsule to be detected is significantly different from that of the environmental background and the color of the capsule detection line, the second branch network only includes one information extraction layer. The above-mentioned information extraction layer can be understood as a combination of a convolutional layer and a downsampling layer.

[0061] It should be noted that the color distribution of the capsule to be detected is a specific distribution, such as a combination of red and blue, while the color of the environmental background and the color of the capsule detection line may be different from the color distribution of the capsule to be detected, so that the color information of the capsule to be detected can be extracted through fewer information extraction layers. The information extraction model may be an extraction model based on a convolutional neural network, and the information extraction model includes multiple information extraction layers, and the information extraction layer may be a convolutional network layer.

[0062] The background of the first depth image is removed according to the first depth information and the first color information to obtain a first capsule foreground. Specifically, the first depth information can be used to remove near and far pixels, such as removing pixels that are close and far away, to obtain a mid-view image, which includes the capsule to be detected. The mid-view image is then removed according to the first color information of the capsule to be detected, thereby removing background pixels in the mid-view image and obtaining the first capsule foreground of the capsule to be detected.

[0063] Similarly, the background of the second depth image is removed according to the second depth information and the second color information to obtain the second capsule foreground. Specifically, the second depth information can be used to remove the near view and the far view, such as removing pixels that are close and far away, so as to obtain a mid-view image, which includes the capsule to be detected. The mid-view image is then removed according to the second color information of the capsule to be detected, thereby removing the background pixels in the mid-view image and obtaining the second capsule foreground of the capsule to be detected.

[0064] By performing background removal on the first depth image and the second depth image based on the trial information and the color information, the background removal can be made more accurate, and a more accurate capsule foreground can be obtained.

[0065] Three-dimensional construction processing can be performed using the depth information of the first capsule foreground and the depth information of the second capsule foreground to obtain a three-dimensional construction surface of the first capsule foreground and a three-dimensional construction surface of the second capsule foreground, and the three-dimensional construction surface of the first capsule foreground and the three-dimensional construction surface of the second capsule foreground are aligned in size, and then the three-dimensional construction surface of the first capsule foreground and the three-dimensional construction surface of the second capsule foreground after the size alignment are obtained to obtain a three-dimensional image of the capsule to be detected.

[0066] Optionally, in the step of extracting the first depth information and the first color information of the first depth image, target detection can be performed on the first depth image, and a first candidate area image can be extracted from the first depth image based on the target detection result, the first candidate area image including the capsule to be detected; the first candidate area image can be scaled to obtain a processed first candidate area image, and the processed first candidate area image has the same size as the first depth image; the first candidate area image and the first depth image are input into the information extraction model for information extraction to obtain the first depth information and the first color information of the first depth image.

[0067] In the embodiment of the present invention, the capsule to be detected is located at the center of the first depth image in most cases. Therefore, by cropping the center area of ​​the first depth image as the first candidate area image, and then expanding the cropped first candidate area image, it is equivalent to enlarging the center area of ​​the first depth image. By enlarging the center area, the detection performance can be improved, and more accurate first depth information and first color information can be obtained. For example, by performing a cropping and enlarging operation on the first depth image, image data of size 384*384 can be cropped from the center of the 512*512 first depth image, and then the image data can be scaled back to size 512*512.

[0068] Optionally, target detection is performed on the second depth image, and a second candidate area image is extracted from the second depth image based on the target detection result, the second candidate area image includes the capsule to be detected; the second candidate area image is scaled to obtain a processed second candidate area image, and the processed second candidate area image has the same size as the second depth image; the second candidate area image and the second depth image are input into an information extraction model for information extraction to obtain second depth information and second color information of the second depth image.

[0069] In the embodiment of the present invention, the capsule to be detected is located at the center of the second depth image in most cases. Therefore, by cropping the central area of ​​the second depth image as the candidate area image, and then expanding the cropped candidate area image, it is equivalent to enlarging the central area of ​​the second depth image. By enlarging the central area, the detection performance can be improved, and more accurate second depth information and second color information can be obtained. For example, by performing a cropping and enlarging operation on the first depth image, image data of size 384*384 can be cropped from the center of the 512*512 second depth image, and then the image data can be scaled back to size 512*512.

[0070] Optionally, in the step of performing three-dimensional construction processing on the first capsule foreground and the second capsule foreground to obtain a three-dimensional image of the capsule to be detected, three-dimensional construction processing can be performed according to the depth value of the first capsule foreground and the depth value of the second capsule foreground to obtain the three-dimensional image of the capsule to be detected.

[0071] Specifically, the first capsule foreground is mapped in three-dimensional space according to the depth value of the first capsule foreground to obtain a first construction surface of the first capsule foreground in the three-dimensional space, the second capsule foreground is mapped in three-dimensional space according to the depth value of the second capsule foreground to obtain a second construction surface of the second capsule foreground in the three-dimensional space, the first construction surface is size-aligned with the second construction surface so that the size of the first construction surface is the same as the size of the second construction surface, and then the first construction surface and the second construction surface after the size alignment are positionally aligned in the three-dimensional space to obtain a three-dimensional image of the capsule to be detected in the three-dimensional space.

[0072] Optionally, before performing three-dimensional construction processing according to the depth value of the first capsule foreground and the depth value of the second capsule foreground to obtain the three-dimensional image of the capsule to be detected, the depth value of the first capsule foreground and the depth value of the second capsule foreground may be repaired.

[0073] Specifically, the depth value of a single point abnormality in the depth value of the foreground of the first capsule can be repaired, and the depth value of a single point abnormality in the depth value of the foreground of the second capsule can be repaired. The above-mentioned single point abnormality refers to the depth value of a certain pixel being abnormal, but the depth values ​​of other surrounding pixels are normal. Only repairing the depth value of a single point abnormality can repair the error generated when the depth camera collects the image, thereby avoiding repairing the defects of the capsule to be detected itself.

[0074] 104. Based on the three-dimensional image and the illumination map, quality inspection is performed on the capsule to be inspected to obtain a quality inspection result of the capsule to be inspected.

[0075] In the embodiment of the present invention, the appearance of the capsule to be inspected can be inspected for quality through the three-dimensional image, and the interior of the capsule to be inspected can be inspected for quality through the illumination map.

[0076] Optionally, in order to facilitate the simultaneous quality inspection of the appearance and interior of the capsule to be inspected and obtain more accurate appearance quality inspection results and internal quality inspection results, the three-dimensional image and the illumination map can be quality inspected through a preset quality inspection model to obtain the appearance quality and internal quality of the capsule to be inspected.

[0077] Among them, the above-mentioned quality detection model includes a three-dimensional convolution module, a two-dimensional convolution module, a fusion module and a decoding module. The three-dimensional convolution module is used to extract the appearance features in the three-dimensional image, and the two-dimensional convolution module is used to extract the transmittance features of the illumination map.

[0078] In a possible embodiment, the illumination map is divided into illumination maps on both sides of the capsule to be detected, and each illumination map is distinguished by a mark, and is associated with the capsule in the three-dimensional image on the same side through the mark, and the left illumination map is associated with the left side of the capsule in the three-dimensional image, and the right illumination map is associated with the right side of the capsule in the three-dimensional image, so that the three-dimensional image and the illumination map are fused. When the three-dimensional convolution module extracts the appearance features, and the two-dimensional convolution module extracts the left light transmission features and the right light transmission features, the left light transmission features and the right light transmission features can be fused with the appearance features through the fusion module, and the fused features are obtained and decoded through the decoding module to obtain the appearance quality and internal quality of the capsule to be detected.

[0079] The above-mentioned three-dimensional convolution module can be constructed based on a three-dimensional convolutional neural network, the above-mentioned two-dimensional convolution module can be constructed based on a convolutional neural network, and the above-mentioned fusion module can include a linear transformation layer and a full convolution layer. The appearance features and the transmittance features are linearly transformed to the same size through the linear transformation layer, and the appearance features and the transmittance features are convolutionally fused through the full convolution layer.

[0080] The above decoding module may be a decoder based on the transform codec architecture.

[0081] During the training process, the quality detection model is trained with a data set, which includes three-dimensional images and illumination maps of sample capsules, as well as quality descriptions of sample capsules. For example, "there are cracks from the blue head to the white tail". The decoder can decode the quality semantics contained in the fusion features into quality descriptions. In this way, the quality problems of smaller capsules can be described in text, which is more intuitive than framing the location of the quality problem in the image for technical personnel to analyze.

[0082] In an embodiment of the present invention, a first depth image and a second depth image of a capsule to be detected are obtained, wherein the first depth image is acquired by a first depth camera set at a first designated position, and the second depth image is acquired by a second depth camera set at a second designated position; an illumination map of the capsule to be detected is obtained, wherein the illumination map is an illumination map of the capsule to be detected on one horizontal side or on both horizontal sides; a three-dimensional image of the capsule to be detected is constructed based on the first depth image and the second depth image; and a quality inspection of the capsule to be detected is performed based on the three-dimensional image and the illumination map to obtain a quality inspection result of the capsule to be detected. The capsule can be constructed in three dimensions through depth images acquired at different positions to obtain a three-dimensional image of the capsule, and the quality of the capsule can be inspected in combination with the illumination map of the capsule, without the need for manual inspection, thereby improving the efficiency of capsule quality inspection.

[0083] It should be noted that the quality inspection method for capsule medicines provided in the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, etc. that can perform quality inspection for capsule medicines.

[0084] Optional, see Figure 2 , Figure 2 Schematic diagram of a device for detecting the quality of a capsule drug provided by an embodiment of the present invention. Figure 2 As shown, the device comprises:

[0085] A first acquisition module 201 is used to acquire a first depth image and a second depth image of a capsule to be detected, wherein the first depth image is acquired by a first depth camera set at a first designated position, and the second depth image is acquired by a second depth camera set at a second designated position;

[0086] A second acquisition module 202 is used to acquire a light map of the capsule to be detected, where the light map is a light map of the capsule to be detected on one horizontal side or on both horizontal sides;

[0087] A construction module 203 constructs a three-dimensional image of the capsule to be detected according to the first depth image and the second depth image;

[0088] The detection module 204 is configured to perform a quality detection on the capsule to be detected based on the three-dimensional image and the illumination map to obtain a quality detection result of the capsule to be detected.

[0089] Optionally, building module 203 includes:

[0090] an extraction submodule, configured to extract first depth information and first color information of the first depth image, and to extract second depth information and second color information of the second depth image;

[0091] a removal submodule, configured to perform background removal on the first depth image according to the first depth information and the first color information to obtain a first capsule foreground, and to perform background removal on the second depth image according to the second depth information and the second color information to obtain a second capsule foreground;

[0092] The construction submodule is used to perform three-dimensional construction processing on the first capsule foreground and the second capsule foreground to obtain a three-dimensional image of the capsule to be detected.

[0093] Optionally, extract submodules, including:

[0094] a first detection unit, configured to perform target detection on the first depth image, and extract a first candidate region image from the first depth image according to an obtained target detection result, wherein the first candidate region image includes the capsule to be detected;

[0095] A first transformation unit, configured to perform a scale transformation process on the first candidate region image to obtain a processed first candidate region image, wherein the processed first candidate region image has the same size as the first depth image;

[0096] an extraction unit, configured to input the first candidate area image and the first depth image into an information extraction model to extract information, so as to obtain first depth information and first color information of the first depth image;

[0097] a second detection unit, configured to perform target detection on the second depth image, and extract a second candidate region image from the second depth image according to the obtained target detection result, wherein the second candidate region image includes the capsule to be detected;

[0098] A second transformation unit, configured to perform a scale transformation process on the second candidate region image to obtain a processed second candidate region image, wherein the processed second candidate region image has the same size as the second depth image;

[0099] The third extraction unit is used to input the second candidate area image and the second depth image into the information extraction model to extract information, so as to obtain second depth information and second color information of the second depth image.

[0100] Optionally, build submodules, including:

[0101] A construction unit is used to perform a three-dimensional construction process according to the depth value of the first capsule foreground and the depth value of the second capsule foreground to obtain a three-dimensional image of the capsule to be detected.

[0102] Optionally, the device comprises:

[0103] A repairing module is used to repair the depth value of the first capsule foreground and the depth value of the second capsule foreground.

[0104] Optionally, the quality inspection result includes appearance quality and internal quality, and the inspection module 204 includes:

[0105] The detection submodule is used to perform quality detection on the three-dimensional image and the illumination map through a preset quality detection model to obtain the appearance quality and internal quality of the capsule to be detected.

[0106] Optionally, the quality detection model includes a three-dimensional convolution module, a two-dimensional convolution module, a fusion module and a decoding module, and the detection submodule includes:

[0107] A first processing unit, configured to input the three-dimensional image into the three-dimensional convolution module for processing to obtain a first intermediate feature of the three-dimensional image;

[0108] a second processing unit, configured to input the illumination map into the two-dimensional convolution module for processing to obtain a second intermediate feature of the two-dimensional image;

[0109] a fusion unit, configured to input the first intermediate feature and the second intermediate feature into the fusion module for feature fusion, so as to obtain a fusion feature of the three-dimensional image and the illumination map;

[0110] The decoding unit is used to input the fusion feature into the decoding module for decoding processing to obtain the appearance quality and internal quality of the capsule to be detected.

[0111] It should be noted that the device for quality inspection of capsule medicines provided in the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, etc. that can perform quality inspection of capsule medicines.

[0112] The capsule medicine quality detection device provided in the embodiment of the present invention can realize each process realized by the capsule medicine quality detection method in the above method embodiment, and can achieve the same beneficial effect. To avoid repetition, it will not be described here.

[0113] See also Figure 3 , Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 3 As shown, it includes: a memory 302, a processor 301, and a computer program for the quality detection method of capsule medicine stored in the memory 302 and executable on the processor 301, wherein:

[0114] The processor 301 is used to call the computer program stored in the memory 302 and execute the following steps:

[0115] Acquire a first depth image and a second depth image of the capsule to be detected, wherein the first depth image is acquired by a first depth camera set at a first specified position, and the second depth image is acquired by a second depth camera set at a second specified position;

[0116] Acquire a light map of the capsule to be detected, where the light map is a light map of the capsule to be detected on one horizontal side or on both horizontal sides;

[0117] constructing a three-dimensional image of the capsule to be detected according to the first depth image and the second depth image;

[0118] Based on the three-dimensional image and the illumination map, a quality inspection is performed on the capsule to be inspected to obtain a quality inspection result of the capsule to be inspected.

[0119] Optionally, the step of constructing the three-dimensional image of the capsule to be detected according to the first depth image and the second depth image executed by the processor 301 includes:

[0120] Extracting first depth information and first color information of the first depth image, and extracting second depth information and second color information of the second depth image;

[0121] performing background removal on the first depth image according to the first depth information and the first color information to obtain a first capsule foreground, and performing background removal on the second depth image according to the second depth information and the second color information to obtain a second capsule foreground;

[0122] The first capsule foreground and the second capsule foreground are subjected to three-dimensional construction processing to obtain a three-dimensional image of the capsule to be detected.

[0123] Optionally, the extracting of the first depth information and the first color information of the first depth image, and the extracting of the second depth information and the second color information of the second depth image performed by the processor 301 include:

[0124] Performing target detection on the first depth image, and extracting a first candidate region image from the first depth image according to the obtained target detection result, wherein the first candidate region image includes the capsule to be detected;

[0125] Performing a scale transformation process on the first candidate region image to obtain a processed first candidate region image, wherein the processed first candidate region image has the same size as the first depth image;

[0126] Inputting the first candidate area image and the first depth image into an information extraction model to extract information, so as to obtain first depth information and first color information of the first depth image;

[0127] Performing target detection on the second depth image, and extracting a second candidate region image from the second depth image according to the obtained target detection result, wherein the second candidate region image includes the capsule to be detected;

[0128] Performing a scale transformation process on the second candidate region image to obtain a processed second candidate region image, wherein the processed second candidate region image has the same size as the second depth image;

[0129] The second candidate area image and the second depth image are input into the information extraction model to extract information, so as to obtain second depth information and second color information of the second depth image.

[0130] Optionally, the processor 301 performs three-dimensional construction processing on the first capsule foreground and the second capsule foreground to obtain a three-dimensional image of the capsule to be detected, including:

[0131] A three-dimensional construction process is performed according to the depth value of the first capsule foreground and the depth value of the second capsule foreground to obtain a three-dimensional image of the capsule to be detected.

[0132] Optionally, before the step of performing three-dimensional construction processing according to the depth value of the first capsule foreground and the depth value of the second capsule foreground to obtain the three-dimensional image of the capsule to be detected, performed by the processor 301, the method includes:

[0133] The depth value of the first capsule foreground and the depth value of the second capsule foreground are repaired.

[0134] Optionally, the quality inspection result performed by the processor 301 includes appearance quality and internal quality, and the quality inspection of the capsule to be inspected based on the three-dimensional image and the illumination map to obtain the quality inspection result of the capsule to be inspected includes:

[0135] The three-dimensional image and the illumination map are subjected to quality inspection by a preset quality inspection model to obtain the appearance quality and internal quality of the capsule to be inspected.

[0136] Optionally, the quality detection model executed by the processor 301 includes a three-dimensional convolution module, a two-dimensional convolution module, a fusion module, and a decoding module. The quality detection of the three-dimensional image and the illumination map by using a preset quality detection model to obtain the appearance quality and internal quality of the capsule to be detected includes:

[0137] Inputting the three-dimensional image into the three-dimensional convolution module for processing to obtain a first intermediate feature of the three-dimensional image;

[0138] Inputting the illumination map into the two-dimensional convolution module for processing to obtain a second intermediate feature of the two-dimensional image;

[0139] Inputting the first intermediate feature and the second intermediate feature into the fusion module for feature fusion to obtain fusion features of the three-dimensional image and the illumination map;

[0140] The fusion feature is input into the decoding module for decoding processing to obtain the appearance quality and internal quality of the capsule to be detected.

[0141] The electronic device provided in the embodiment of the present invention can implement each process implemented by the capsule medicine quality detection method in the above method embodiment, and can achieve the same beneficial effect. To avoid repetition, it will not be described here.

[0142] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the quality inspection method for capsule medicines or the quality inspection method for capsule medicines at an application end provided in the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0144] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for quality inspection of capsule medicines, characterized in that: The following steps are involved: Acquire a first depth image and a second depth image of the capsule to be detected, wherein the first depth image is acquired by a first depth camera set at a first specified position, and the second depth image is acquired by a second depth camera set at a second specified position; Acquire a light map of the capsule to be detected, where the light map is a light map of the capsule to be detected on one horizontal side or on both horizontal sides; constructing a three-dimensional image of the capsule to be detected according to the first depth image and the second depth image; The three-dimensional image and the illumination map are quality inspected by a preset quality inspection model to obtain the appearance quality and the internal quality of the capsule to be inspected. The quality inspection model includes a three-dimensional convolution module, a two-dimensional convolution module, a fusion module and a decoding module. Specifically, the three-dimensional image is input into the three-dimensional convolution module for processing to obtain a first intermediate feature of the three-dimensional image; the illumination map is input into the two-dimensional convolution module for processing to obtain a second intermediate feature of the illumination map; the first intermediate feature and the second intermediate feature are input into the fusion module for feature fusion to obtain a fusion feature of the three-dimensional image and the illumination map; the fusion feature is input into the decoding module for decoding processing to obtain the appearance quality and the internal quality of the capsule to be inspected.

2. The method according to claim 1, characterized in that The step of constructing a three-dimensional image of the capsule to be detected according to the first depth image and the second depth image includes: Extracting first depth information and first color information of the first depth image, and extracting second depth information and second color information of the second depth image; performing background removal on the first depth image according to the first depth information and the first color information to obtain a first capsule foreground, and performing background removal on the second depth image according to the second depth information and the second color information to obtain a second capsule foreground; The first capsule foreground and the second capsule foreground are subjected to three-dimensional construction processing to obtain a three-dimensional image of the capsule to be detected.

3. The method according to claim 2, characterized in that The extracting first depth information and first color information of the first depth image, and extracting second depth information and second color information of the second depth image, comprises: Performing target detection on the first depth image, and extracting a first candidate region image from the first depth image according to the obtained target detection result, wherein the first candidate region image includes the capsule to be detected; Performing a scale transformation process on the first candidate region image to obtain a processed first candidate region image, wherein the processed first candidate region image has the same size as the first depth image; Inputting the first candidate area image and the first depth image into an information extraction model to extract information, so as to obtain first depth information and first color information of the first depth image; Performing target detection on the second depth image, and extracting a second candidate region image from the second depth image according to the obtained target detection result, wherein the second candidate region image includes the capsule to be detected; Performing a scale transformation process on the second candidate region image to obtain a processed second candidate region image, wherein the processed second candidate region image has the same size as the second depth image; The second candidate area image and the second depth image are input into the information extraction model to extract information, so as to obtain second depth information and second color information of the second depth image.

4. The method according to claim 3, characterized in that The performing three-dimensional construction processing on the first capsule foreground and the second capsule foreground to obtain a three-dimensional image of the capsule to be detected includes: A three-dimensional construction process is performed according to the depth value of the first capsule foreground and the depth value of the second capsule foreground to obtain a three-dimensional image of the capsule to be detected.

5. The method according to claim 4, characterized in that Before the step of performing three-dimensional construction processing according to the depth value of the first capsule foreground and the depth value of the second capsule foreground to obtain the three-dimensional image of the capsule to be detected, the method includes: The depth value of the first capsule foreground and the depth value of the second capsule foreground are repaired.

6. A capsule medicine quality inspection device, characterized in that: The device comprises: A first acquisition module is used to acquire a first depth image and a second depth image of the capsule to be detected, wherein the first depth image is acquired by a first depth camera set at a first specified position, and the second depth image is acquired by a second depth camera set at a second specified position; A second acquisition module is used to acquire a light map of the capsule to be detected, where the light map is a light map of the capsule to be detected on one horizontal side or on both horizontal sides; A construction module, constructing a three-dimensional image of the capsule to be detected according to the first depth image and the second depth image; The detection module is used to perform quality detection on the three-dimensional image and the illumination map through a preset quality detection model to obtain the appearance quality and internal quality of the capsule to be detected. The quality detection model includes a three-dimensional convolution module, a two-dimensional convolution module, a fusion module and a decoding module. Specifically, the three-dimensional image is input into the three-dimensional convolution module for processing to obtain a first intermediate feature of the three-dimensional image; the illumination map is input into the two-dimensional convolution module for processing to obtain a second intermediate feature of the illumination map; the first intermediate feature and the second intermediate feature are input into the fusion module for feature fusion to obtain a fusion feature of the three-dimensional image and the illumination map; the fusion feature is input into the decoding module for decoding processing to obtain the appearance quality and internal quality of the capsule to be detected.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the quality inspection method for capsule medicines according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the quality inspection method for capsule medicines according to any one of claims 1 to 5 are implemented.

Images