Scanning system based on image processing and image processing method

Abstract

The invention provides a scanning system based on image processing, which comprises a box body, a rotating seat and a host, and is characterized in that the box body comprises a box wall, a top side and a bottom side; a first camera is arranged on the top side, and a plurality of second cameras are arranged on the box wall; the rotating seat is arranged on the bottom side and is used for placing medicines; the host is electrically connected with the first camera, the multiple second cameras and the rotating base and comprises a package type obtaining module, a control module, an image processing module and an image splicing module, and the package type obtaining module is used for obtaining the package type of medicine; the control module is used for controlling the first camera and / or the plurality of second cameras to shoot the medicine package of the medicine according to the package type to obtain a plurality of medicine package images; the image processing module is used for performing foreground extraction on the plurality of medicine packaging images to obtain a plurality of foreground images; and the image splicing module is used for splicing the plurality of foreground images to obtain a spliced image. In addition, the invention also provides an image processing method applied to the scanning system.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a scanning system based on image processing, and an image processing method. Background Art

[0002] With the development of the pharmaceutical industry, the requirements for drug management and information collection are becoming increasingly stringent. Traditional drug information collection mostly relies on manual operations, which is not only inefficient but also prone to errors. Especially for drugs with various shapes, such as cylindrical, cubic, irregular bags, etc., their image collection and information processing are a major challenge. Existing technologies cannot effectively extract clear images of all types of drugs, which affects the accuracy of subsequent image recognition and text recognition. Summary of the invention

[0003] The present application provides a scanning system and an image processing method based on image processing, which realizes the automation of drug packaging scanning and splicing by cooperating with a camera and a rotating seat and combining the packaging type of the drug packaging.

[0004] In a first aspect, an embodiment of the present application provides a scanning system based on image processing, the scanning system based on image processing comprises a box body, a rotating seat and a main body, the box body comprises a box wall, a top side located at the top of the box wall, and a bottom side arranged opposite to the top side; the top side is provided with a first camera, and the box wall is provided with a plurality of second cameras; the rotating seat is arranged on the bottom side corresponding to the first camera, and is used to place medicine; the main body is electrically connected to the first camera, the plurality of second cameras and the rotating seat, respectively, and comprises a packaging type acquisition module, a control module, an image processing module and an image stitching module, the packaging type acquisition module is used to acquire the packaging type of the medicine, the control module is used to control the first camera and / or the plurality of second cameras to shoot the medicine packaging of the medicine according to the packaging type to obtain a plurality of medicine packaging images; the image processing module is used to perform foreground extraction on the plurality of medicine packaging images to obtain a plurality of foreground images, each foreground image contains the medicine packaging; the image stitching module is used to stitch the plurality of foreground images to obtain a stitched image.

[0005] In a second aspect, an embodiment of the present application provides an image processing method, which is applied to a scanning system based on image processing, wherein the scanning system includes a box, a rotating base and a main unit, and the box is provided with a first camera and several second cameras. The image processing method includes: obtaining the packaging type of the drug; according to the packaging type, controlling the first camera and / or the several second cameras to shoot the drug packaging of the drug to obtain several drug packaging images; performing foreground extraction on the several drug packaging images to obtain several foreground images, each foreground image containing the drug packaging; and splicing the several foreground images to obtain a spliced ​​image.

[0006] The above-mentioned scanning system based on image processing and image processing method, by obtaining the packaging type of the drug input by the user to control the first camera and / or several second cameras to shoot the drug packaging to obtain several drug packaging images, after performing foreground extraction on the several drug packaging images to obtain several foreground images, the several foreground images are spliced ​​according to different packaging types of the drug packaging to obtain a spliced ​​image, thereby realizing automatic image acquisition of drugs with different drug packages, and while improving the integrity and clarity of the acquired images by performing foreground extraction through a background model and then performing image splicing, the image processing algorithm is optimized by providing image processing methods for different packaging types, thereby improving the accuracy of image recognition and text recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0008] Figure 1 A schematic diagram of a scanning system based on image processing provided in an embodiment of the present application.

[0009] Figure 2 A structural block diagram of the host provided in an embodiment of the present application.

[0010] Figure 3 This is a first flow chart of the image processing method provided in an embodiment of the present application.

[0011] Figure 4 A flowchart of sub-steps of step S102 provided in an embodiment of the present application.

[0012] Figure 5 This is a second flow chart of the image processing method provided in an embodiment of the present application.

[0013] Figure 6 This is a flowchart of an image processing method for a first packaging type provided in an embodiment of the present application.

[0014] Figure 7 This is a flowchart of the image processing method for the second packaging type provided in an embodiment of the present application.

[0015] Figure 8 This is a flowchart of the image processing method for the third packaging type provided in an embodiment of the present application.

[0016] Fig. 9This is a flowchart of the image processing method for the fourth packaging type provided in an embodiment of the present application.

[0017] Fig.10 A schematic diagram of the internal structure of a host computer used in an image processing-based scanning system provided in an embodiment of the present application.

[0018] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0020] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar planning objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate. In other words, the described embodiments are implemented according to an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations may also include other content. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to only those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] It should be noted that the descriptions involving "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0022] Please see Figure 1, the present application provides a scanning system 100 based on image processing. The scanning system 100 based on image processing includes a box body 1, a rotating seat 2 and a host 3. Among them, the box body 1 is a light-proof sealed box body. The box body 1 includes a box wall 11, a top side 12 located on the top of the box wall 11, and a bottom side 13 arranged opposite to the top side 12. Among them, the top side 12 is provided with a first camera 121. The box wall 11 is provided with a plurality of second cameras 110. The box body 1 is also provided with a light supply component (not shown) for providing a light source for the first camera 121 and the second camera 110. The rotating seat 2 is used to place medicines, and is an electrically controlled rotating seat, so that the host 3 drives the medicines to rotate. The rotating seat 2 is arranged on the bottom side 13 corresponding to the first camera 121, so that the field of view of the first camera 121 and each second camera 110 can cover the medicines to be placed on the rotating seat 2, thereby improving the clarity of the subsequent shooting of the medicine packaging. In the present application, the box wall 11 includes a front side 111 and a back side 112 facing away from the front side 111. A plurality of second cameras 110 are respectively disposed on the front side 111 and the back side 112.

[0023] In this embodiment, the host 3 is a computer device that can display and provide users with operations. The host 3 is electrically connected to the first camera 121, the plurality of second cameras 110 and the rotating seat 2 respectively. In other embodiments, the host 3 can be electrically connected through a USB extender, and simultaneously connected to the first camera 121, the plurality of second cameras 110 and the rotating seat 2 through the USB extender to achieve connection with each component in the box 1. The host 3 includes a packaging type acquisition module 31, a control module 32, an image processing module 33 and an image stitching module 34. Among them, the packaging type acquisition module 31 is used to obtain the packaging type of the drug. The control module 32 is used to control the first camera 121 and / or the plurality of second cameras 110 to shoot the drug packaging according to the packaging type to obtain a plurality of drug packaging images. The image processing module 33 is used to extract the foreground of a plurality of drug packaging images to obtain a plurality of foreground images. Among them, each foreground image contains the drug packaging. The image stitching module 34 is used to stitch a plurality of foreground images to obtain a stitched image.

[0024] In this embodiment, the packaging type acquisition module 31 includes but is not limited to input buttons, a touch screen that can display and be operated by the user, etc. The packaging type acquisition module 31 allows the user to input the packaging type of the drug, that is, the packaging type can be obtained through the user input. In addition, the first camera 121 and the plurality of second cameras 110 have recognition functions respectively. The packaging type acquisition module 31 is also respectively connected to the first camera 121 and the plurality of second cameras 110 for communication, so as to identify the drug through the first camera 121 and the plurality of second cameras 110 to obtain the packaging type of the drug.

[0025] In the above embodiment, the features of each component in the image processing-based scanning system 100 are described. The image processing-based scanning system 100 provided in the present application uses the host 3 to execute the image processing method to make the rotating base 2 and the camera in the image processing-based scanning system 100 cooperate to achieve scanning and splicing of drug packages of different packaging types. The image processing method will be specifically described below.

[0026] Please see Figure 3 The present application also provides an image processing method. The image processing method is applied to the scanning system 100 based on image processing, so that after the user inputs the packaging type of the medicine to be scanned through the scanning system 100, the control module 32 controls the combination of the rotating seat 2 and the camera to scan the medicines of different packaging types and obtain the corresponding spliced ​​images. The specific features of the scanning system 100 have been described in detail above and will not be repeated here. The image processing method includes steps S101-S104.

[0027] Step S101, obtaining the packaging type of the medicine.

[0028] In step S101, the packaging type of the drug includes but is not limited to cylindrical packaging, box packaging, stand-up bag packaging, lying bag packaging, spherical packaging, irregular packaging, etc., to meet the different needs of users for collecting drug information. The user inputs the packaging type of the drug placed on the rotating seat 2 through the packaging type acquisition module 31 of the host 3. In other embodiments, the packaging type acquisition module 31 can also be provided with corresponding input components that can control the input of each packaging type according to the packaging types of multiple preset drug packages, such as selection buttons, selection touch controls, etc., for the user to input the corresponding packaging type according to the drug placed on the rotating seat 2. In other embodiments, the drug can also be identified by the first camera and several second cameras to obtain the packaging type.

[0029] Step S102: According to the packaging type, the first camera and / or several second cameras are controlled to photograph the medicine packaging to obtain several medicine packaging images.

[0030] In step S102, since the location of drug information on different drug packages is not the same, after obtaining the input packaging type and obtaining the location of drug information on each drug package through multiple preset drug package types, in order to save scanning space and improve the clarity of scanning drug packages, when the first camera 121 and the plurality of second cameras 110 are fixedly set on the box wall 11, the control module 32 can control the rotating seat 2 to drive the drug to rotate in different rotation modes according to the drug packaging type, and control the camera to combine with the rotating seat 2 to shoot the drug package in different ways. Controlling the rotating seat 2 to drive the drug to rotate in different rotation modes according to the drug packaging type, and controlling the camera to combine with the rotating seat 2 to shoot the drug package in different ways will be described in detail below.

[0031] Please refer to Figure 4 According to the packaging type, controlling the first camera and / or several second cameras to photograph the drug packaging of the drug to obtain several drug packaging images includes steps S1021-S1022.

[0032] Step S1021, according to the packaging type, controlling the rotating base to rotate at a rotation angle corresponding to the packaging type.

[0033] Step S1022: when the rotating base rotates to drive the medicine to rotate, the first camera and / or a plurality of second cameras are controlled to shoot the medicine packaging to obtain a plurality of medicine packaging images.

[0034] In some feasible embodiments, after placing the medicine on the rotating disk, the image processing method further includes the following steps: controlling the first camera 121 and the plurality of second cameras 110 to shoot the medicine to obtain a medicine package recognition image; inputting the medicine package recognition image into the medicine package recognition model to obtain the corresponding packaging type of the medicine package image, so as to control the rotating seat 2 to rotate at a rotation angle corresponding to the packaging type according to the packaging type, and controlling the first camera 121 and / or the plurality of second cameras 110 to shoot the medicine package to obtain a plurality of medicine package images. The medicine package recognition type is obtained by inputting a plurality of preset medicine package recognition images into the image learning model for training, and is used to recognize the corresponding packaging type according to the medicine package image.

[0035] Step S103, performing foreground extraction on a plurality of drug packaging images to obtain a plurality of foreground images.

[0036] In step S103, each foreground image includes a drug package. Since the box wall 11 is used as the shooting background during shooting and there is no drug package image that needs to be spliced, the image processing module 33 needs to extract the foreground of several drug package images to remove the background parts such as the box wall 11, thereby improving the image quality. The present application can perform foreground extraction by a differential method, and after the image is processed by the differential method, perform morphological processing on the image to remove interference such as image noise. Among them, morphological processing includes but is not limited to corrosion algorithm, expansion algorithm, etc.

[0037] Please refer to Figure 4 , extracting foreground from a number of drug packaging images to obtain a number of foreground images includes the following steps: feeding a number of preset drug packaging images into an initial training model for training to obtain a background model; inputting a number of drug packaging images into the background model to obtain a number of foreground images. Among them, the background model is used to separate the background and foreground images of the drug packaging image to obtain a foreground image containing the drug packaging. In this embodiment, the initial training model is an initial neural network model. The background model is a neural network model obtained after deep learning of the initial neural network model. The initial neural network model can be an existing neural network model.

[0038] Step S104: stitching together a plurality of foreground images to obtain a stitched image.

[0039] Please see Figure 5 Before stitching a plurality of foreground images together to obtain a stitched image according to the packaging type, it is also necessary to obtain a stitched image size that matches each drug packaging image to improve the authenticity of the image stitching. Specifically, the image processing method further includes steps S201-S203.

[0040] Step S201 : calculating the cumulative distribution function of each foreground image in a preset horizontal direction and a preset vertical direction perpendicular to each other to obtain a horizontal boundary value and a vertical boundary value.

[0041] In step S201 , the preset horizontal direction and the preset vertical direction are in the same coordinate system and are perpendicular to each other.

[0042] The horizontal boundary value and the vertical boundary value can be calculated by the following expressions.

[0043]

[0044] Formula (1) is the Cumulative Distribution Function (CDF), which is an existing expression and can be used to determine the corresponding integral when certain feature points fall into a certain range of values. When embodied in a closed figure, it can be used to determine the corresponding area when certain feature points fall into a certain range of values. When using the difference method to extract the foreground of several drug packaging images, the essence is to divide the feature values ​​of the foreground part, that is, the drug packaging part, and the background part of the drug packaging image differently, so as to use CDF to quickly divide the feature values ​​of the drug packaging part from the drug packaging image into corresponding areas, that is, the region of interest (ROI). For example, the ROI area can be represented by xywh. Among them, x, y, w, h represent the horizontal coordinate, vertical coordinate, width along the x-axis direction from the upper left corner point in the ROI area, and height along the y-axis direction from the upper left corner point. CDF is essentially a curve that can be used to determine CDFx and CDFy. Among them, CDFx and CDFy are the horizontal boundary value and the vertical boundary value, respectively. When the values ​​of the horizontal coordinates corresponding to the vertical coordinates 0.05 and 0.95 are taken as the boundaries, CDFx can determine x1 and x2 of the ROI area, and CDFy can determine y1 and y2 of the ROI area. Accordingly, the ROI area can be expressed as ROI = (x1, y1, x2-x1, y2-y1).

[0045] Step S202: Obtain the region of interest of each foreground image according to the horizontal boundary value and the vertical boundary value.

[0046] Step S203, according to the region of interest of each foreground image, the size of the stitched image of each foreground image is calculated, so as to stitch several foreground images according to the packaging type and the size of the stitched image. The calculation of the size of the stitched image of each foreground image will be described in detail below.

[0047] In the above embodiment, the scanning system 100 controls the rotating seat 2 to rotate after the user inputs the packaging type of the medicine through the image processing method, and controls the first camera 121 and / or the second camera 110 to shoot the medicine on the rotating seat 2 to obtain the medicine packaging image, thereby realizing the subsequent splicing of the medicine packaging image. In the present application, several medicine packaging images are videos or several pictures. The image processing method provided in the present application can also control the rotating seat 2 to rotate in different rotation modes according to the different packaging types of the medicine, and control the first camera 121 and / or the second camera 110 to shoot in different ways to obtain the corresponding medicine packaging image. The following will elaborate on how to control the camera in combination with the rotating seat 2 to shoot the medicine packaging in different ways, and calculate the spliced ​​image size of different foreground images, thereby realizing the image processing method of different medicine packaging.

[0048] Please see Figure 6-Figure 9 , Figure 6-Figure 9 The image processing methods of drug packaging images with different packaging types are illustrated respectively. Figure 6 As shown, the image processing method for the first packaging type further includes steps S301-S304.

[0049] Step S301, controlling the rotating base to rotate from an initial position at a rotation angle of one circle, and controlling a plurality of second cameras to record synchronously with the rotating base to obtain a video.

[0050] In step S301, the video includes several video frames. The first packaging type is cylindrical packaging. The drugs in cylindrical packaging often have drug information printed on the column body. Since the column body has a certain curvature, in order to ensure that the drug information can be scanned, the control module 32 needs to control the second camera 110 to shoot the column body. In other embodiments, the drug information may be printed on one or more of the top, bottom and column body of the cylindrical package. At this time, the top and bottom of the drug can be controlled to flip relative to the rotating seat 2 by adding a component for moving the drug to the rotating seat 2, or a certain prompt information can be output to the user to prompt the user that the relative position of the drug on the rotating seat 2 needs to be adjusted, and the control module 32 controls the first camera 121 to shoot the top or bottom of the cylindrical package.

[0051] Step S302, when a plurality of video frames are input into a background model to obtain a plurality of foreground images, and a region of interest of each foreground image is obtained, a total stitching size of the stitched image is calculated according to the region of interest.

[0052] In step S302, the total stitching size includes width and height, which can be expressed as (ROI region height*π, ROI region height), where "ROI region height*π" represents the width of the stitched image, and "ROI region height" represents the height of the stitched image.

[0053] Step S303: Calculate the stitching image size of each foreground image according to the total stitching size.

[0054] In step S303, since the cylindrical packaged drug image corresponds to a video, which is actually a plurality of continuously shot pictures, when stitching the foreground images corresponding to the video into a stitched image, it is necessary to simultaneously consider the drug information existing in the image corresponding to each video frame. Specifically, the stitched image size of each foreground image can be calculated by the following expression.

[0055] ROI video =

[0056] (ROI x +ROI h / 2-stitched image width / FrameCount*2,stitched image width / FrameCount*4,ROI h )(Formula 2)

[0057] In formula (2), ROI x It is the horizontal coordinate of the upper left corner, (ROI x +ROI h / 2) is the horizontal coordinate of the middle point of the ROI area, and FrameCount is the number of video frames collected by the second camera 110 when the rotating base 2 rotates one circle.

[0058] Step S304 , stitching together the foreground images corresponding to a plurality of video frames in sequence to obtain a stitched image according to the total stitching size and the stitching image size of each foreground image.

[0059] In step S304, foreground images corresponding to several video frames are cyclically spliced ​​according to the shooting order of the video frames, and the splicing of the foreground images is completed by using a method of detecting SURF (Speeded Up Robust Features) feature points and calculating a homography matrix.

[0060] like Figure 7 As shown, the image processing method for the second packaging type further includes steps S401-S405.

[0061] Step S401, control the rotating base to rotate from the initial position to form a first angle with the front side, and control the second camera located on the front side to capture a number of first images every time the rotating base rotates the first angle, until the rotating base rotates to the initial position again, control the second camera located on the front side to stop capturing, and control the first camera to capture a second image.

[0062] In step S401, the second packaging type is a box-shaped package. Since the shape of the medicine in the box-shaped package is relatively regular, the drug information is often printed on one or more of the box body, the box top and the box bottom. Therefore, it is also necessary to control the first camera 121 to shoot and ensure that the potential drug information on the box top and the box bottom will not be missed. At the same time, since the shape of the medicine in the box-shaped package is relatively regular, when the medicine is placed on the rotating seat 2 at an initial angle so that the second camera 110 located at the front 111 can just shoot one side of the box body, the rotation angle of the rotating seat 2 can be controlled according to the shape of the box body, so that when the rotating seat 2 rotates a rotation angle, the rotating seat 2 is controlled to stop rotating so that one side of the box body can be shot by the second camera 110 located at the front 111. Optionally, the box-shaped package is a square box package. At this time, the first angle is 90°. When the rotating seat 2 is rotated to the initial position again, the box body part has been fully photographed. At this time, the first camera 121 is controlled to shoot the box top or box bottom of the box-shaped package to improve the integrity of the drug information scanning.

[0063] Step S402: When a plurality of first images and a second image are respectively input into a background model to obtain a plurality of first foreground images and a second foreground image, an average height of the plurality of first foreground images is calculated to obtain a first average height.

[0064] In step S402, the input of the first images and the second images into the background model to obtain the first foreground images and the second foreground images, and the acquisition of the stitched image size adapted to each drug package image are at least mentioned in the above step S103 and steps S201-S203, and are not described in detail here. Optionally, when the box-shaped package is a square box-shaped package, the number of the first foreground images is 4.

[0065] Step S403 : scaling the plurality of first foreground images according to the first average height, and splicing the plurality of first foreground images along a width direction perpendicular to the first average height to obtain a first spliced ​​image.

[0066] In step S403, the scaling process for the first foreground images is to scale the first foreground images in equal proportion according to the first average height. The first foreground images are stitched together. The image stitching module 34 stitches the first foreground images together according to the shooting order of the first images corresponding to the first foreground images.

[0067] Step S404: adjusting the image size of the second foreground image according to the plurality of first foreground images.

[0068] In step S404, the image size of the second foreground image is adjusted by determining the first foreground image adjacent to the second foreground image in the width direction and the height direction according to the shape of one side of the box body and the shape of the second foreground image, and adjusting the image size of the second foreground image according to the width of the first foreground image determined above. For example, when the box-shaped packaging is a square box-shaped packaging, the image size of the second foreground image can be adjusted according to the width of the first first foreground image and the second first foreground image.

[0069] Step S405 , expanding a stitching area in the first stitching image along the direction of the first average height, so that the adjusted second foreground image is stitched with the first foreground image in the stitching area to obtain a stitching image.

[0070] In step S405, the size of the stitching area is determined by the first stitching image and the second foreground image. For example, when the box-shaped package is a square box-shaped package, the stitching area can be expressed as (width of the first foreground image, 0, width of the adjusted second foreground image, height of the adjusted second foreground image).

[0071] like Figure 8 As shown, the image processing method for the third packaging type further includes steps S501-S503.

[0072] Step S501, controlling the rotating base to rotate from an initial position to form a second angle with the front side, and controlling a plurality of second cameras to capture a plurality of images each time the rotating base rotates the second angle.

[0073] In step S501, the third packaging type is a stand-up bag package. The stand-up bag-shaped drug package often has drug information printed on the bag body. The second angle can be set to 180°. When the drug in the stand-up bag package is initially placed on the rotating seat 2, one side of the bag body can be facing the second camera 110. Accordingly, the drug in the stand-up bag package can be photographed twice at the same time by the second camera 110 located at the front 111 and the back 112, so as to fully photograph the drug package.

[0074] Step S502, when a plurality of images are input into the background model to obtain a plurality of foreground images, the average height of each foreground image is calculated to obtain a second average height.

[0075] Step S503: scaling the plurality of foreground images according to the second average height, and splicing the plurality of foreground images along a width direction perpendicular to the second average height to obtain a spliced ​​image.

[0076] In steps S502-S503, the calculation of the second average height and the splicing of the foreground images may refer to the aforementioned steps S402-S403, which will not be described in detail here.

[0077] like Fig. 9 As shown, the image processing method for the fourth packaging type further includes packaging S601-S604.

[0078] Step S601, controlling the first camera to capture a first image, and outputting a prompt message to prompt the user to turn the medicine over.

[0079] In step S601, the fourth packaging type is a lying bag packaging. The difference between the lying bag packaging and the standing bag packaging is that the state of the medicine placed on the rotating seat 2 is different.

[0080] Step S602: When the user turns over the medicine, the first camera is controlled to take a picture again to obtain a second image.

[0081] Step S603: when the first image and the second image are input into the background model to obtain the first foreground image and the second foreground image, the average height of the first foreground image and the second foreground image is calculated to obtain a third average height.

[0082] Step S604: scaling the first foreground image and the second foreground image according to the third average height, and splicing the first foreground image and the second foreground image along a width direction perpendicular to the third average height to obtain a spliced ​​image.

[0083] In steps S603-S604, the calculation of the third average height and the splicing of the bag-shaped foreground image can refer to the aforementioned steps S402-S403, which will not be described in detail here.

[0084] In the above embodiment, by matching corresponding image processing methods to drugs of various drug packaging types, the processing of image information such as drug information on the drug packaging can be achieved without being restricted by the state of the drug placed on the rotating seat 2, the drug packaging type, etc., thereby achieving the goal that when the drug is placed in the scanning system 100 in a variety of placement methods, it can be scanned and the drug information on the drug packaging can be obtained.

[0085] It can be understood that since the rotating seat 2 can be controlled to rotate in different ways in the same box 1, and the first camera 121 and / or the plurality of second cameras 110 can be combined to shoot the drugs, the scanning and splicing of multiple different drug packaging images can be realized in the same box 1. Therefore, the host 3 can also be electrically connected to multiple boxes 1 at the same time through a USB extender, and the rotating seat 2, the first camera 121 and / or the plurality of second cameras 110 in each box 1 can be controlled according to the type of drug packaging to operate only corresponding to the image processing method of one type of drug packaging, so as to realize the large-scale scanning, processing and recognition of drug information on each drug package.

[0086] Please see Fig.10, which is a schematic diagram of the internal structure of a host applied to an image processing-based scanning system provided in an embodiment of the present application.

[0087] like Fig.10 As shown, the host 3 is a computer device for executing the image processing method. The host 3 includes a memory 301 and a processor 302. The processor 302 is used to run the computer program instructions in the memory 301 to implement the image processing method.

[0088] The memory 301 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 301 may be an internal storage unit of a computer device, such as a hard disk of a computer device. In other embodiments, the memory 301 may also be a storage device of an external computer device, such as a plug-in hard disk configured in the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 301 may also include both an internal storage unit of a computer device and an external storage device. The memory 301 may not only be used to store application software and various types of data installed in the computer device, such as the code of the image processing method, etc., but may also be used to temporarily store data that has been output or is to be output.

[0089] Furthermore, the host 3 also includes a bus 303. The bus 303 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0090] Furthermore, the host 3 may also include a display component 304. The display component 304 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an organic light-emitting diode (OLED) touch device, etc. The display component 304 may also be appropriately referred to as a display device or a display unit, which is used to display information processed in the host 3 and to display a visual user interface.

[0091] Furthermore, the host 3 may also include a communication component 305. The communication component 305 may optionally include a wired communication component and / or a wireless communication component (such as a Wi-Fi communication component, a Bluetooth communication component, etc.), which is generally used to establish a communication connection between the host 3 and other computer devices.

[0092] Fig.10 Only a host 3 having some components and implementing the image processing method is shown. It can be understood by those skilled in the art that Fig.10 The structure shown does not constitute a limitation on the host 3, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0093] In the above embodiments, all or part of the embodiments may be implemented by software, hardware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0094] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer device may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0095] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0096] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0098] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0099] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only storage medium (ROM, Read-Only Memory), random access storage medium (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0100] In the above embodiment, the first camera and / or several second cameras are controlled to shoot the drug packaging by obtaining the packaging type of the drug input by the user to obtain several drug packaging images. After the foreground of the several drug packaging images is extracted to obtain several foreground images, the several foreground images are spliced ​​according to different packaging types of the drug packaging to obtain a spliced ​​image, thereby realizing automatic image acquisition of drugs in different drug packages. While the integrity and clarity of the acquired images are improved by performing foreground extraction through a background model and then performing image stitching, the image processing algorithm is optimized by providing image processing methods for different packaging types, thereby improving the accuracy of image recognition and text recognition.

[0101] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0102] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0103] The above examples are only preferred embodiments of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. A scanning system based on image processing, characterized in that: The image processing-based scanning system comprises: The box body comprises a box wall, a top side located at the top of the box wall, and a bottom side arranged opposite to the top side; the top side is provided with a first camera, and the box wall is provided with a plurality of second cameras; a rotating seat, arranged on the bottom side corresponding to the first camera, for placing medicine; and A host, electrically connected to the first camera, a plurality of second cameras and a rotating base, respectively, comprises: A packaging type acquisition module, used to acquire the packaging type of the drug; a control module, configured to control the first camera and / or the plurality of second cameras to photograph the drug packaging of the drug according to the packaging type to obtain a plurality of drug packaging images; an image processing module, configured to perform foreground extraction on the plurality of drug package images to obtain a plurality of foreground images, each foreground image including the drug package; and The image stitching module is used to stitch the plurality of foreground images to obtain a stitched image.

2. The image processing-based scanning system according to claim 1, characterized in that: The packaging type is obtained by user input or by identifying the drug through the first camera and several second cameras; the box wall includes a front side and a back side facing away from the front side, and the several second cameras are respectively arranged on the front side and the back side.

3. An image processing method, applied to a scanning system based on image processing, the scanning system comprising a box, a rotating seat and a host, the box being provided with a first camera and a plurality of second cameras, characterized in that: The image processing method comprises: Get the type of packaging the medication comes in; According to the packaging type, controlling the first camera and / or the plurality of second cameras to photograph the drug packaging of the drug to obtain a plurality of drug packaging images; Performing foreground extraction on the plurality of drug package images to obtain a plurality of foreground images, each foreground image including the drug package; and The plurality of foreground images are stitched together to obtain a stitched image.

4. The image processing method according to claim 3, characterized in that: The packaging type is input by a user, or obtained by identifying the medicine by the first camera and a plurality of second cameras; According to the packaging type, controlling the first camera and / or the plurality of second cameras to photograph the drug packaging of the drug to obtain a plurality of drug packaging images includes: According to the packaging type, controlling the rotating base to rotate at a rotation angle corresponding to the packaging type; When the rotating seat rotates to drive the medicine to rotate, the first camera and / or the plurality of second cameras are controlled to shoot the medicine packaging to obtain the plurality of medicine packaging images.

5. The image processing method according to claim 3, characterized in that: Before splicing the plurality of foreground images to obtain a spliced ​​image, the image processing method further includes: Respectively calculating the cumulative distribution function of each foreground image in a preset horizontal direction and a preset vertical direction perpendicular to each other to obtain a horizontal boundary value and a vertical boundary value; Acquire a region of interest of each foreground image according to the horizontal boundary value and the vertical boundary value; According to the region of interest of each foreground image, the size of the stitched image of each foreground image is calculated, so as to stitch the plurality of foreground images according to the packaging type and the stitched image size.

6. The image processing method according to any one of claims 3 to 5, characterized in that: The box body includes a box wall, the box wall includes a front side and a back side facing away from the front side, the plurality of second cameras are respectively arranged on the front side and the back side; the plurality of drug packaging images are videos or a plurality of pictures.

7. The image processing method according to claim 6, characterized in that: The image processing method further comprises: Controlling the rotating base to rotate from an initial position at a rotation angle of one circle, and controlling the plurality of second cameras to record synchronously with the rotating base to obtain the video, wherein the video includes a plurality of video frames; When a plurality of video frames are input into the background model to obtain a plurality of foreground images, and a region of interest of each foreground image is obtained, a total stitching size of the stitching image is calculated according to the region of interest; Calculating the stitching image size of each foreground image according to the total stitching size; According to the total stitching size and the stitching image size of each foreground image, the foreground images corresponding to the plurality of video frames are stitched in sequence to obtain the stitching image.

8. The image processing method according to claim 6, characterized in that: The image processing method further comprises: Control the rotating base to rotate from an initial position to a first angle with the front side, and control the second camera located on the front side to capture a plurality of first images each time the rotating base rotates by the first angle, until the rotating base rotates to the initial position again, control the second camera located on the front side to stop capturing, and control the first camera to capture a second image; When a plurality of first images and a second image are respectively input into the background model to obtain a plurality of first foreground images and a second foreground image, an average height of the plurality of first foreground images is calculated to obtain a first average height; scaling the plurality of first foreground images according to the first average height, and splicing the plurality of first foreground images along a width direction perpendicular to the first average height to obtain a first spliced ​​image; adjusting the image size of the second foreground image according to the plurality of first foreground images; A stitching area is expanded in the first stitching image along the direction of the first average height, so that the adjusted second foreground image is stitched with the first foreground image in the stitching area to obtain the stitching image, and a size of the stitching area is determined by the first stitching image and the second foreground image.

9. The image processing method according to claim 6, characterized in that: The image processing method further comprises: Controlling the rotating base to rotate from an initial position to form a second angle with the front side, and controlling the plurality of second cameras to capture a plurality of images each time the rotating base rotates to the second angle; When a plurality of images are input into the background model to obtain a plurality of foreground images, the average height of each foreground image is calculated to obtain a second average height; The plurality of foreground images are scaled according to the second average height, and the plurality of foreground images are spliced ​​along a width direction perpendicular to the second average height to obtain the spliced ​​image.

10. The image processing method according to claim 6, characterized in that: The image processing method further comprises: Controlling the first camera to capture a first image, and outputting a prompt message to prompt a user to turn the medicine over; When the user turns over the medicine, the first camera is controlled to take a picture again to obtain a second image; When the first image and the second image are input into the background model to obtain the first foreground image and the second foreground image, the average height of the first foreground image and the second foreground image is calculated to obtain a third average height; The first foreground image and the second foreground image are scaled according to the third average height, and the first foreground image and the second foreground image are spliced ​​along a width direction perpendicular to the third average height to obtain the spliced ​​image.

Images