An ampoule bottle information recognition method and system based on machine vision

Abstract

The present invention belongs to the technical field of ampoule information recognition. The present invention discloses a method and system for ampoule information recognition based on machine vision, including: collecting image data of the ampoule, performing image filtering on the image data to obtain a contour-enhanced image; performing transformation correction on the contour-enhanced image to obtain a standard bottle body image, and performing label clipping on the standard bottle body image to obtain a label area image; extracting information from the standard bottle body image and the label area image based on the constructed information extraction model to obtain ampoule information; the present invention effectively improves the system's ability to quickly and automatically recognize ampoules, significantly improves the efficiency of the production line, and provides reliable data support for quality control in the production process.

Description

Technical Field

[0001] The present invention relates to the technical field of ampoule information recognition, and more specifically, the present invention relates to a method and system for ampoule information recognition based on machine vision. Background Art

[0002] An ampoule is a sealed container used for storing and transporting liquid medicine, which is widely used in the medical and pharmaceutical industries. To ensure the safety and traceability of drugs, ampoules usually need to be marked with key information such as drug name, production date, batch number, barcode or two-dimensional code, etc. These information are crucial for quality management and circulation management; however, in the actual production and use process, the information marked on the surface of the ampoule is often interfered by various factors, making traditional information recognition technology face many challenges; for example: the surface of the ampoule is usually made of glass or plastic material, which is easy to reflect light under strong light, resulting in overexposure or light spots in the image captured by the camera, making the marked information difficult to identify, and the surface of the ampoule is usually a curved cylindrical structure, and the marked text or pattern will be deformed due to the perspective effect of the curved surface, and traditional planar image processing methods are difficult to directly apply, and the misrecognition rate is relatively high; different light source conditions on the production line will lead to uneven brightness distribution in the image, affecting the clarity of the marked information, and the background on the production line is easy to merge with the edge contour of the ampoule, and traditional edge detection methods are difficult to distinguish the bottle body contour and background features, and are easily misidentified as effective features of the ampoule, reducing the accuracy of ampoule information recognition.

[0003] In view of this, the present invention proposes a method and system for ampoule information recognition based on machine vision to solve the above problems. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art and to achieve the above object, the present invention provides the following technical solution: A method for ampoule information recognition based on machine vision, comprising:

[0005] S1. Collect image data of the ampoule, perform image filtering on the image data to obtain a contour enhanced image;

[0006] S2. Perform transformation correction on the contour enhanced image to obtain a standard bottle body image, and perform label clipping on the standard bottle body image to obtain a label area image;

[0007] S3. Extract information from the standard bottle body image and the label area image based on the constructed information extraction model to obtain ampoule information.

[0008] Further, the method for collecting image data of the ampoule includes:

[0009] Use a high-resolution industrial camera with a global shutter configured. Install a photoelectric sensor on the conveyor belt to detect the position of the ampoule in real time. When the ampoule enters the imaging area, trigger the acquisition signal to capture an image of the ampoule and obtain the image data of the ampoule.

[0010] Further, the method of image filtering the image data includes:

[0011] Take the lower left corner of each image in the image data as the coordinate origin. For each additional pixel horizontally, the horizontal coordinate scale of the pixel is incremented by one. For each additional pixel vertically, the vertical coordinate scale of the pixel is incremented by one to obtain the coordinate label of each pixel. Use the RGB grayscale conversion algorithm to perform grayscale conversion on each image to obtain a pixel grayscale image, in which the grayscale value of each pixel is marked. Extract the frequency of the pixel grayscale image to obtain the low-frequency component. Based on the low-frequency component, subtract the corresponding low-frequency component from the grayscale value of the pixel to obtain the high-frequency component. Preset a bilateral filter and perform smoothing processing on the high-frequency component based on the preset bilateral filter to obtain the high-frequency smoothed grayscale.

[0012] Take the size of the smoothing parameter as the size of the low-frequency smoothing window, preset the low-frequency smoothing window. Based on the low-frequency smoothing window, take the pixel corresponding to the low-frequency component as the center, use the window length as the boundary, take the pixels corresponding to the boundary as the boundary pixels, and take all the pixels between the boundary pixels as the window pixels. The coordinate labels of all window pixels form a window coordinate set. Perform window smoothing on the low-frequency component based on the window coordinate set to obtain the low-frequency smoothed grayscale. Perform multi-scale fusion on the high-frequency smoothed grayscale and the low-frequency smoothed grayscale to obtain the fused grayscale. Use the maximum-minimum normalization algorithm to perform normalization reduction on the fused grayscale to obtain the normalized reduced grayscale. Perform detail enhancement on the normalized reduced grayscale to obtain the detail grayscale value. All the pixels after detail enhancement in the same image form a contour enhancement map.

[0013] Further, the formula for window smoothing the low-frequency component is:

[0014] ; where represents the low-frequency smoothed grayscale of the pixel at the coordinate label , represents the size of the window coordinate set, represents the window coordinate set, represents the low-frequency component of the pixel at the coordinate label , represents the horizontal axis value of the coordinate label in the window coordinate set, represents the vertical axis value of the coordinate label in the window coordinate set.

[0015] Further, the method of transformation and correction for the contour enhancement map includes:

[0016] Preset a horizontal gradient matrix and a vertical gradient matrix. For each pixel in the contour enhancement map, construct a pixel window. Use the pixel value of each pixel in the pixel window as the matrix element value to construct a pixel matrix for each pixel. Fill the elements without corresponding pixels in the pixel matrix with zeros. Perform amplitude transformation on the pixels in the contour enhancement map based on the pixel matrix to obtain pixel amplitudes. Preset a high amplitude threshold and a low amplitude threshold. Consider the pixels with pixel amplitudes greater than the high amplitude threshold as strong edge pixels, the pixels with pixel amplitudes less than the low amplitude threshold as discarded pixels, and the pixels with pixel amplitudes greater than or equal to the low amplitude threshold and less than or equal to the high amplitude threshold as weak edge pixels. For weak edge pixels, perform association determination on the weak edge pixels in a four-neighborhood manner. Mark the weak edge pixels that do not contain strong edge pixels in the four-neighborhood as discarded pixels. Discard all the pixels marked as discarded pixels in the contour enhancement map to obtain an initial contour map. Perform contour tracking on the initial contour map to obtain an initial bottle body map. Perform image stretching on the initial bottle body map to obtain a standard bottle body map.

[0017] Further, the method of performing contour tracking on the initial contour map includes:

[0018] Use the strong edge pixels in the initial contour map as the exploration starting points. Initialize A exploration bodies, where the value of A is equal to the number of strong edge pixels. Assign the exploration bodies to each exploration starting point. Initialize the path set of each exploration body as the exploration starting point. Construct an exploration area for each exploration starting point with an eight-neighborhood. Use a random selection algorithm to select a pixel point in the exploration area as the starting selection point. Perform subsequent pixel selection in a clockwise direction starting from the starting selection point. When an exploration body reaches a weak edge pixel or a strong edge pixel, use the pixel position where the exploration body arrives as the point to be selected. Record the pixel position and pixel amplitude of the point to be selected. Consider the point to be selected with the largest pixel amplitude as the new exploration starting point, and include the pixel position of the new exploration starting point in the path set. When the exploration body returns to the pixel position corresponding to the first element in the path set, the exploration body stops moving. When all exploration bodies stop moving, perform pixel connection in the initial contour map based on the path set of each exploration body to obtain a pixel connection map. Use the pixel connection map with the most strong edge pixels as the initial bottle body map.

[0019] Further, the method of performing image stretching on the initial bottle body map includes:

[0020] A standard coordinate system is preset, and the lower left corner pixel of the initial bottle body image is used as the standard origin of the standard coordinate system. For each pixel added horizontally, the horizontal coordinate scale of the pixel is increased by one, and for each pixel added vertically, the vertical coordinate scale of the pixel is increased by one. The initial bottle body image is mapped to the standard coordinate system to obtain the mapping coordinates of each pixel; the points on the edge of the image in the initial bottle body image are used as corner points, and the mapping coordinates of the corner points are used as corner point coordinates. Based on the constructed affine transformation model, the coordinates of the corner points are affine transformed to obtain affine coordinates, and based on the affine coordinates, the corner point coordinates are pulled up to the corresponding affine coordinates in the standard coordinate system to obtain a standard bottle body image.

[0021] Furthermore, the method of cutting the label on the standard bottle body image includes:

[0022] Based on the standard coordinate system, the strong edge pixels in the standard bottle body image are transformed into polar coordinates. The formula for polar coordinate transformation is: ;in, Represents the vertical distance of the mapping coordinates of strong edge pixels from the standard origin, Represents the pixel angle between the mapping coordinates of strong edge pixels, the line connecting the standard origin and the horizontal axis in the standard coordinate system, Represents the horizontal coordinate of the mapping coordinates corresponding to the strong edge pixels, The vertical coordinate represents the mapping coordinates corresponding to the strong edge pixel; each vertical distance and pixel angle constitutes a pixel space, the preset space marking table is empty, each pixel space is generated as the space to be marked, and the space marking table is queried one by one. When the space marking table does not contain the space to be marked, the space to be marked is recorded in the space marking table, and the voting value is initialized to one. When the space marking table contains the space to be marked, the voting value of the space to be marked is increased by one; the implicit equation method of Cartesian coordinates is used to convert the first D pixel spaces with the highest voting values ​​into coordinate curves, and the closed space formed by the coordinate curve is used as the clipping area to clip the standard bottle body image to obtain the label area map.

[0023] Furthermore, the information extraction model is constructed in the following manner:

[0024] The CNN model is used as the initial model of the information extraction model, and group B training data is collected in advance. The training data includes historical standard bottle body images, historical label area images and historical ampoule bottle information. The training data is used as the training sample set, and the CNN model is trained using the training sample set. The historical standard bottle body images, historical label area images and historical ampoule bottle information are used as input data of the information extraction model, and the extracted ampoule bottle information is used as output data of the information extraction model. Minimizing the error between the actual historical ampoule bottle information and the ampoule bottle information extracted by the information extraction model is used as the training goal, and the recall rate function is used as the loss function of the information extraction model. When the loss function converges, the training is stopped to obtain the information extraction model.

[0025] An ampoule information recognition system based on machine vision, comprising:

[0026] Data acquisition and processing module: Collect image data of the ampoule, perform image filtering on the image data, and obtain a contour-enhanced image;

[0027] Image correction and cropping module: including an image correction unit and an image cropping unit. The image correction unit performs transformation correction on the contour-enhanced image to obtain a standard bottle body image, and the image cropping unit performs label cropping on the standard bottle body image to obtain a label area image;

[0028] Information extraction module: Based on the constructed information extraction model, extract information from the standard bottle body image and the label area image to obtain ampoule information.

[0029] Technical effects and advantages of the ampoule information recognition method and system based on machine vision of the present invention:

[0030] In the present invention, by performing image filtering on the image data, large-area brightness fluctuations in the image are effectively eliminated, while key details and edge information are retained, further enhancing the contrast and details of the image; by performing transformation correction on the contour-enhanced image, the edge contour of the ampoule can be accurately extracted, ensuring the integrity of the contour, eliminating deformations caused by the placement angle or light changes in the image, and ensuring the standardization of the bottle body image; by performing label cropping on the standard bottle body image, the closed cropping space of the label area is accurately defined, and the label area is efficiently cropped, which can ensure the precise matching of the label area with the edge of the bottle body image, providing a reliable basis for subsequent information extraction; by using the information extraction model, efficient retrieval and recognition of label information are realized, ensuring the accuracy and reliability of the extraction. Description of the Drawings

[0031] Figure 1 Schematic diagram of an ampoule information recognition method based on machine vision of the present invention;

[0032] Figure 2 Schematic diagram of an ampoule information recognition system based on machine vision of the present invention. Detailed Embodiments

[0033] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0034] Embodiment 1

[0035] Please refer to Figure 1 As shown, a method for identifying ampoule information based on machine vision in this embodiment includes:

[0036] S1. Collect image data of the ampoule, perform image filtering on the image data, and obtain a contour-enhanced image;

[0037] S2. Perform transformation and correction on the contour-enhanced image to obtain a standard bottle body image, and perform label cropping on the standard bottle body image to obtain a label area image;

[0038] S3. Extract information from the standard bottle body image and the label area image based on the constructed information extraction model to obtain ampoule information;

[0039] The methods for collecting image data of the ampoule include:

[0040] Use a high-resolution industrial camera and configure a global shutter. Install a photoelectric sensor on the conveyor belt to detect the position of the ampoule in real time. When the ampoule enters the imaging area, trigger a collection signal to perform image collection on the ampoule to obtain image data of the ampoule; The high-resolution industrial camera captures more than 100 frames per second of images to avoid image distortion caused by the high-speed movement of the conveyor belt.

[0041] The methods for performing image filtering on the image data include:

[0042] Take the lower left corner of each image in the image data as the coordinate origin. For each additional pixel horizontally, the horizontal coordinate scale of the pixel is incremented by one. For each additional pixel vertically, the vertical coordinate scale of the pixel is incremented by one to obtain the coordinate mark of each pixel; Use the RGB grayscale conversion algorithm to perform grayscale conversion on each image to obtain a pixel grayscale image, in which the grayscale value of each pixel is marked. Perform frequency extraction on the pixel grayscale image. The formula for performing frequency extraction is: ; where represents the low-frequency component of the pixel at the coordinate mark , represents the grayscale value of the pixel at the coordinate mark , represents pi, represents the smoothing parameter, which is used to control the low-frequency smoothing intensity of the pixel. Usually, the value ranges from 1% to 5% of the width of the image. In this embodiment, the value of the smoothing parameter is preferably 2%, represents the horizontal axis value of the coordinate mark, represents the vertical axis value of the coordinate mark; Based on the low-frequency component, subtract the corresponding low-frequency component from the grayscale value of the pixel to obtain the high-frequency component;

[0043] Preset bilateral filter, smooth the high-frequency components based on the preset bilateral filter to obtain high-frequency smoothed grayscale; protect the image edges by smoothing the high-frequency components to avoid destroying details; use the size of the smoothing parameter as the size of the low-frequency smoothing window, preset the low-frequency smoothing window, based on the low-frequency smoothing window, use the pixels corresponding to the low-frequency components as the center, use the window length as the boundary, use the pixels corresponding to the boundary as the boundary pixels, use all the pixels between the boundary pixels as the window pixels, and the coordinate marks of all the window pixels form the window coordinate set. Smooth the low-frequency components based on the window coordinate set, and the formula for window smoothing is:

[0044] ; where represents the low-frequency smoothed grayscale of the pixel at the coordinate mark , represents the size of the window coordinate set, represents the window coordinate set, represents the low-frequency component of the pixel at the coordinate mark , represents the horizontal axis value of the coordinate mark in the window coordinate set, represents the vertical axis value of the coordinate mark in the window coordinate set; eliminate large-area brightness fluctuations in the image by window smoothing the low-frequency components; perform multi-scale fusion on the high-frequency smoothed grayscale and the low-frequency smoothed grayscale, and the formula for multi-scale fusion is: ; where represents the fused grayscale of the pixel at the coordinate mark , represents the fusion weight, which is used to allocate the weight ratio of the high-frequency smoothed grayscale and the low-frequency smoothed grayscale for fusion, represents the high-frequency smoothed grayscale of the pixel at the coordinate mark , represents the low-frequency smoothed grayscale of the pixel at the coordinate mark ;

[0045] Use the maximum-minimum normalization algorithm to normalize and restore the fused grayscale to obtain the normalized and restored grayscale; perform detail enhancement on the normalized and restored grayscale, and the formula for detail enhancement is: ; where represents the detail grayscale value of the pixel at the coordinate mark , represents the detail adjustment coefficient, which is used to control the adjustment intensity of the image brightness, represents the control constant, which is used to control the range of adjusted pixels; all the pixels after detail enhancement in the same image form the contour enhancement map;

[0046] The detail adjustment coefficient is adjusted by those skilled in the art. In low-contrast images, a smaller detail adjustment coefficient (e.g., 0.5 to 0.8) is selected to enhance the details of the image. In high-contrast images, a larger detail adjustment coefficient (such as 1.2 to 1.5) is selected to avoid over-enhancement.

[0047] The methods for transforming and correcting the contour enhancement map include:

[0048] Preset a horizontal gradient matrix and a vertical gradient matrix, and construct a pixel window centered on each pixel in the contour enhancement map. Use the pixel value of each pixel in the pixel window as the matrix element value to construct a pixel matrix for each pixel. The elements without corresponding pixels in the pixel matrix are filled with zeros; perform amplitude conversion on the pixels in the contour enhancement map based on the pixel matrix. The formula for amplitude conversion is: ; where represents the pixel amplitude, represents the horizontal gradient matrix, represents the vertical gradient matrix, represents the pixel matrix; preset a high amplitude threshold and a low amplitude threshold. Consider the pixels with pixel amplitudes greater than the high amplitude threshold as strong edge pixels, the pixels with pixel amplitudes less than the low amplitude threshold as discarded pixels, and the pixels with pixel amplitudes greater than or equal to the low amplitude threshold and less than or equal to the high amplitude threshold as weak edge pixels; for the weak edge pixels, perform an association determination on the weak edge pixels in the four-neighborhood. Discard the weak edge pixels that do not contain strong edge pixels in the four-neighborhood, and discard all the pixels marked as discarded pixels in the contour enhancement map to obtain the initial contour map;

[0049] The horizontal gradient matrix uses the horizontal convolution kernel in the sobel operator, and the vertical gradient matrix uses the vertical convolution kernel in the sobel operator. In this embodiment, it is preferably with a value of 3. The high amplitude threshold and the low amplitude threshold are set by those skilled in the art based on the actual situation; the four-neighborhood refers to the four pixels in the upper, lower, left, and right directions directly associated with a certain pixel centered on that pixel;

[0050] The initial contour map is contour traced, and the strong edge pixels in the initial contour map are used as the exploration starting point. A exploration bodies are initialized, and the value of A is equal to the number of strong edge pixels. The exploration bodies are assigned to each exploration starting point, and the path set of each exploration body is initialized as the exploration starting point. The exploration area of ​​each exploration starting point is constructed with an eight-neighborhood, and a pixel point in the exploration area is selected as the starting selection point by a random selection algorithm. Subsequent pixel selection is performed in a clockwise direction from the starting selection point. When the exploration body reaches a weak edge pixel or a strong edge pixel, the pixel position reached by the exploration body is used as the point to be selected, and the pixel position and pixel amplitude of the point to be selected are recorded. The point to be selected with the largest pixel amplitude is used as the new exploration starting point, and the pixel position of the new exploration starting point is counted into the path set. When the exploration body returns to the pixel position corresponding to the first element in the path set, the exploration body stops moving. When all exploration bodies stop moving, pixels are connected in the initial contour map based on the path set of each exploration body to obtain a pixel connection map, and the pixel connection map containing the most strong edge pixels is used as the initial bottle body map;

[0051] The eight neighborhoods are the pixels in the eight directions of up, down, left, right, upper left, lower left, upper right, and lower right, centered on the exploration starting point and directly connected to the exploration starting point;

[0052] The initial bottle body image is stretched, a standard coordinate system is preset, and the lower left corner pixel of the initial bottle body image is used as the standard origin of the standard coordinate system. For each pixel added horizontally, the horizontal coordinate scale of the pixel is increased by one, and for each pixel added vertically, the vertical coordinate scale of the pixel is increased by one. The initial bottle body image is mapped to the standard coordinate system to obtain the mapping coordinates of each pixel; the points on the edge of the image in the initial bottle body image are used as corner points, and the mapping coordinates of the corner points are used as corner point coordinates. Based on the constructed affine transformation model, the coordinates of the corner points are affine transformed to obtain affine coordinates, and the corner point coordinates are pulled to the corresponding affine coordinates in the standard coordinate system based on the affine coordinates to obtain a standard bottle body image; by stretching the initial bottle body image, the deformation caused by the placement angle or light change in the image is eliminated, the standardization of the bottle body image is ensured, the clarity of the image is improved, and the accuracy of subsequent information extraction from the image is effectively improved.

[0053] The affine transformation model is constructed in the following ways:

[0054] Collect the historical bottle body template image and the historical initial bottle body image. The historical initial bottle body image is the same type of image as the initial bottle body image. Use the pixels at the edge of the bottle body in the historical initial bottle body image as the transformation points, and the pixels corresponding to the transformation points in the historical bottle body template image as the mapping points. Based on the standard coordinate axes pair, use the bottom-left pixel of the historical bottle body template image as the origin of the standard coordinate axes to coordinate the transformation points and obtain the transformation point coordinates. Use the bottom-left pixel of the historical initial bottle body image as the origin of the standard coordinate axes to coordinate the mapping points and obtain the mapping point coordinates. Use the transformation point coordinates and the mapping point coordinates as the input of the affine transformation model, and the predicted affine coordinates as the output of the affine transformation model. Use the mapping point coordinates as the prediction target of the affine transformation model to train the model and construct the affine transformation model. The affine transformation model is a linear regression model, and the target loss function of the affine transformation model is the mean square error function. Use minimizing the value of the target loss function as the training target to obtain the affine transformation model with the minimum value of the target loss function. The historical bottle body template image is the image in which the ampoule bottle is placed most upright and the edge of the bottle body is the clearest.

[0055] The methods for label cropping of the standard bottle body image include:

[0056] Perform polar coordinate transformation on the strong edge pixels in the standard bottle body image based on the standard coordinate system. The formula for performing polar coordinate transformation is: ; where represents the vertical distance from the mapping coordinates of the strong edge pixel to the standard origin, represents the pixel angle between the line connecting the mapping coordinates of the strong edge pixel and the standard origin and the horizontal axis in the standard coordinate system, represents the abscissa of the mapping coordinates corresponding to the strong edge pixel, represents the ordinate of the mapping coordinates corresponding to the strong edge pixel; each vertical distance and pixel angle form a pixel space. The preset space marking table is empty. Each generated pixel space is used as a space to be marked. Query the space marking table one by one. When the space marking table does not contain the space to be marked, record the space to be marked in the space marking table and initialize the voting value to one. When the space marking table contains the space to be marked, increment the voting value of the space to be marked by one. Use the implicit equation method of Cartesian coordinates to convert the top D pixel spaces with the highest voting values into coordinate curves, and use the closed space formed by the coordinate curves as the cropping area to crop the standard bottle body image to obtain the label area image.

[0057] The construction method of the information extraction model includes:

[0058] Taking the CNN model as the initial model of the information extraction model, pre-collecting B groups of training data, where the training data includes historical standard vial images, historical label region images, and historical ampoule information. Using the training data as the training sample set, training the CNN model with the training sample set, taking the historical standard vial images, historical label region images, and historical ampoule information as the input data of the information extraction model, and taking the extracted ampoule information as the output data of the information extraction model; taking minimizing the error between the actual historical ampoule information and the ampoule information extracted by the information extraction model as the training objective, and taking the recall rate function as the loss function of the information extraction model. When the loss function converges, stop training to obtain the information extraction model;

[0059] Based on the trained information extraction model, using the initial vial image as the input of the information extraction model for feature extraction to obtain vial feature information.

[0060] In this embodiment, by performing image filtering on the image data, large-area brightness fluctuations in the image are effectively eliminated, while key details and edge information are retained, further enhancing the contrast and details of the image; by performing transformation correction on the contour enhancement map, the edge contour of the ampoule can be accurately extracted, ensuring the integrity of the contour, eliminating deformations in the image caused by the placement angle or light changes, and ensuring the standardization of the vial image; by performing label cropping on the standard vial image, the closed cropping space of the label region is precisely defined, and the label region is efficiently cropped, which can ensure the precise matching of the label region with the edge of the vial image, providing a reliable basis for subsequent information extraction; by using the information extraction model to achieve efficient retrieval and recognition of label information, the accuracy and reliability of the extracted ampoule information are guaranteed.

[0061] Embodiment 2

[0062] Please refer to Figure 2 As shown, for the parts not described in detail in this embodiment, refer to the description content of Embodiment 1. A machine vision-based ampoule information recognition system is provided, including:

[0063] Data acquisition and processing module: Collecting the image data of the ampoule, performing image filtering on the image data to obtain a contour enhancement map;

[0064] Image correction and cropping module: Including an image correction unit and an image cropping unit. The image correction unit performs transformation correction on the contour enhancement map to obtain a standard vial image, and the image cropping unit performs label cropping on the standard vial image to obtain a label region image;

[0065] Information extraction module: Based on the constructed information extraction model, performing information extraction on the standard vial image and the label region image to obtain ampoule information;

[0066] Each module is connected by wired and / or wireless means to achieve data transmission between modules.

[0067] Embodiment 3

[0068] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the operation mode of the above-provided method for identifying ampoule information based on machine vision.

[0069] Since the electronic device introduced in this embodiment is the electronic device used to implement a method for identifying ampoule information based on machine vision in an embodiment of the present application, based on the method for identifying ampoule information based on machine vision introduced in an embodiment of the present application, those skilled in the art can understand the specific implementation manner and various variations of the electronic device in this embodiment. Therefore, the details of how the electronic device implements the method in an embodiment of the present application will not be described in detail here. As long as those skilled in the art implement the electronic device used in a method for identifying ampoule information based on machine vision in an embodiment of the present application, it falls within the scope of protection of the present application.

[0070] The above formulas are all dimensionless and take their numerical values for calculation. The formula is obtained by collecting a large amount of data for software simulation to obtain a formula that is closest to the actual situation. The preset parameters and threshold selection in the formula are set by those skilled in the art according to the actual situation.

[0071] The above are only the preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for ordinary technical users in the technical field, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the protection scope of the present invention.

Claims

1. A method for identifying ampoule information based on machine vision, characterized in that: include: S1, collecting image data of the ampoule bottle, performing image filtering on the image data, and obtaining a contour enhancement image; S2, transform and correct the contour enhancement image to obtain a standard bottle body image, and perform label cropping on the standard bottle body image to obtain a label area image; S3, extracting information from the standard bottle body image and label area image based on the constructed information extraction model to obtain ampoule bottle information; The method of performing image filtering on the image data includes: Take the lower left corner of each image in the image data as the coordinate origin, add one to the horizontal coordinate scale of the pixel for each horizontal pixel increase, and add one to the vertical coordinate scale of the pixel for each vertical pixel increase, and obtain the coordinate mark of each pixel; use the RGB grayscale conversion algorithm to convert each image into grayscale to obtain a pixel grayscale map, in which the grayscale value of each pixel is marked, and perform frequency extraction on the pixel grayscale map to obtain the low-frequency component; based on the low-frequency component, subtract the corresponding low-frequency component from the grayscale value of the pixel to obtain the high-frequency component; preset a bilateral filter, and smooth the high-frequency component based on the preset bilateral filter to obtain a high-frequency smoothed grayscale; use the size of the smoothing parameter as the low-frequency smoothing window size, preset a low-frequency smoothing window, based on the low-frequency smoothing window, take the pixel corresponding to the low-frequency component as the center, take the window length as the boundary, take the pixel corresponding to the boundary as the boundary pixel, take all the pixels between the boundary pixels as the window pixels, and the coordinate marks of all window pixels constitute a window coordinate set. Based on the window coordinate set, perform window smoothing on the low-frequency component to obtain a low-frequency smooth grayscale; perform multi-scale fusion on the high-frequency smooth grayscale and the low-frequency smooth grayscale to obtain a fused grayscale; use the maximum and minimum normalization algorithm to normalize and restore the fused grayscale to obtain a normalized restored grayscale; perform detail enhancement on the normalized restored grayscale to obtain a detail grayscale value, and all pixels in the same image that have undergone detail enhancement constitute a contour enhancement map; The formula for window smoothing the low-frequency component is: Among them, I smooth (x, y) represents the low-frequency smooth grayscale of the pixel at the coordinate mark (x, y), Siz represents the size of the window coordinate set, Dic represents the window coordinate set, I low (x ′ ,y ′ ) represents the coordinates marked as (x ′ ,y ′ ), x represents the horizontal axis value of the coordinate mark, y represents the vertical axis value of the coordinate mark, x ′ Represents the horizontal axis value of the coordinate mark in the window coordinate set, y ′ Represents the vertical axis value of the coordinate marker within the window coordinate set.

2. The ampoule bottle information recognition method based on machine vision according to claim 1, characterized in that: The method of collecting the image data of the ampoule bottle includes: A high-resolution industrial camera with a global shutter is used, and a photoelectric sensor is installed on the conveyor belt to detect the position of the ampoule in real time. When the ampoule enters the camera area, a collection signal is triggered to collect the image of the ampoule and obtain the image data of the ampoule.

3. The ampoule bottle information recognition method based on machine vision according to claim 2, characterized in that: The method of transforming and correcting the contour enhancement image includes: A horizontal gradient matrix and a vertical gradient matrix are preset, an N×N pixel window is constructed with each pixel in the contour enhancement image as the center, the pixel value of each pixel in the pixel window is used as the matrix element value, and an N×N pixel matrix is ​​constructed for each pixel, and the elements without corresponding pixels in the pixel matrix are filled with zero; the pixels in the contour enhancement image are amplitude-converted based on the pixel matrix to obtain the pixel amplitude; a high amplitude threshold and a low amplitude threshold are preset, pixels with a pixel amplitude greater than the high amplitude threshold are regarded as strong edge pixels, pixels with a pixel amplitude less than the low amplitude threshold are regarded as discarded pixels, and pixels with a pixel amplitude greater than or equal to the low amplitude threshold and less than or equal to the high amplitude threshold are regarded as weak edge pixels; for weak edge pixels, four neighborhoods are used to perform association judgment on the weak edge pixels, weak edge pixels that do not contain strong edge pixels in the four neighborhoods are marked as discarded pixels, and all pixels marked as discarded pixels in the contour enhancement image are discarded to obtain an initial contour image; contour tracking is performed on the initial contour image to obtain an initial bottle body image; image stretching is performed on the initial bottle body image to obtain a standard bottle body image.

4. The ampoule bottle information recognition method based on machine vision according to claim 3, characterized in that: The method of performing contour tracking on the initial contour map includes: The strong edge pixels in the initial contour map are used as the exploration starting points, A exploration bodies are initialized, and the value of A is equal to the number of strong edge pixels. The exploration bodies are assigned to each exploration starting point, and the path set of each exploration body is initialized as the exploration starting point. The exploration area of ​​each exploration starting point is constructed with an eight-neighborhood. A pixel point in the exploration area is selected as the starting selection point by a random selection algorithm. Subsequent pixel selection is performed in a clockwise direction from the starting selection point. When the exploration body reaches a weak edge pixel or a strong edge pixel, the pixel position reached by the exploration body is used as the point to be selected, and the pixel position and pixel amplitude of the point to be selected are recorded. The point to be selected with the largest pixel amplitude is used as the new exploration starting point, and the pixel position of the new exploration starting point is counted into the path set. When the exploration body returns to the pixel position corresponding to the first element in the path set, the exploration body stops moving. When all exploration bodies stop moving, pixels are connected in the initial contour map based on the path set of each exploration body to obtain a pixel connection map, and the pixel connection map containing the most strong edge pixels is used as the initial bottle body map.

5. The ampoule bottle information recognition method based on machine vision according to claim 4, characterized in that: The method of stretching the initial bottle body image includes: A standard coordinate system is preset, and the lower left corner pixel of the initial bottle body image is used as the standard origin of the standard coordinate system. For each pixel added horizontally, the horizontal coordinate scale of the pixel is increased by one, and for each pixel added vertically, the vertical coordinate scale of the pixel is increased by one. The initial bottle body image is mapped to the standard coordinate system to obtain the mapping coordinates of each pixel; the points on the edge of the image in the initial bottle body image are used as corner points, and the mapping coordinates of the corner points are used as corner point coordinates. Based on the constructed affine transformation model, the coordinates of the corner points are affine transformed to obtain affine coordinates, and based on the affine coordinates, the corner point coordinates are pulled up to the corresponding affine coordinates in the standard coordinate system to obtain a standard bottle body image.

6. The ampoule bottle information recognition method based on machine vision according to claim 5, characterized in that: The method of cutting the label on the standard bottle body image includes: Based on the standard coordinate system, the strong edge pixels in the standard bottle body image are transformed into polar coordinates, and the formula for polar coordinate transformation is: ρ = hor × cosθ + ver × cosθ; wherein ρ represents the vertical distance of the mapping coordinates of the strong edge pixel from the standard origin, θ represents the pixel angle between the mapping coordinates of the strong edge pixel, the line connecting the standard origin and the horizontal axis in the standard coordinate system, ver represents the horizontal coordinate of the mapping coordinates corresponding to the strong edge pixel, and hor represents the vertical coordinate of the mapping coordinates corresponding to the strong edge pixel; each vertical distance and pixel angle constitutes a pixel space, and the preset space marking table is empty. Each time a pixel space is generated as a space to be marked, the space marking table is queried one by one. When the space marking table does not contain the space to be marked, the space to be marked is recorded in the space marking table and the voting value is initialized to one. When the space marking table contains the space to be marked, the voting value of the space to be marked is increased by one; the implicit equation method of Cartesian coordinates is used to convert the first D pixel spaces with the highest voting values ​​into coordinate curves, and the closed space formed by the coordinate curve is used as the clipping area to clip the standard bottle body image to obtain the label area map.

7. The ampoule bottle information recognition method based on machine vision according to claim 6, characterized in that: The information extraction model is constructed in the following manner: The CNN model is used as the initial model of the information extraction model, and group B training data is collected in advance. The training data includes historical standard bottle body images, historical label area images and historical ampoule bottle information. The training data is used as the training sample set, and the CNN model is trained using the training sample set. The historical standard bottle body images, historical label area images and historical ampoule bottle information are used as input data of the information extraction model, and the extracted ampoule bottle information is used as output data of the information extraction model. Minimizing the error between the actual historical ampoule bottle information and the ampoule bottle information extracted by the information extraction model is used as the training goal, and the recall rate function is used as the loss function of the information extraction model. When the loss function converges, the training is stopped to obtain the information extraction model.

8. An ampoule bottle information recognition system based on machine vision, which is used to implement the ampoule bottle information recognition method based on machine vision as claimed in any one of claims 1 to 7, characterized in that: include: Data acquisition and processing module: collects image data of the ampoule bottle, performs image filtering on the image data, and obtains a contour enhancement image; Image correction and cropping module: including an image correction unit and an image cropping unit. The image correction unit transforms and corrects the contour enhancement image to obtain a standard bottle body image. The image cropping unit performs label cropping on the standard bottle body image to obtain a label area image. Information extraction module: Based on the constructed information extraction model, information is extracted from the standard bottle body image and label area image to obtain ampoule bottle information.

Images