An automated cargo identification system for an arm robot

Abstract

The present application relates to the technical field of image recognition, and particularly relates to an automatic cargo recognition system of an ARM robot, which comprises: a collection module for determining a barcode area; a processing module for determining an X and Y direction of the barcode and a reference length of a stripe; judging the completeness of the barcode; for a complete barcode, determining an X reference width based on an initial X direction width corresponding to a reference straight line passing through a Y direction center point and parallel to the X direction, and determining a target stripe width and interval based on the X reference width; for an incomplete barcode, selecting a temporary X direction width based on multiple reference straight lines in the X direction, determining the X reference width based on the temporary X direction width, determining a standard straight line based on a multiple deviation of the temporary X direction width relative to the X reference width in the X direction, and obtaining the target stripe width and interval by segmenting the standard straight line; and an identification module for reconstructing the barcode and reading cargo information based on the reference length of the stripe, the target stripe width and the interval. The present application can improve the accuracy of barcode information recognition.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to an automated cargo recognition system for an ARM robot. Background Technology

[0002] Automated cargo identification and sorting are core components for improving operational efficiency in logistics warehousing and intelligent manufacturing. Barcodes, due to their high information density, low cost, and ease of generation, have become the mainstream carrier for cargo information storage and transmission. With the integration of embedded technology and robotics, Adjustable Robot Modules (ARM), with their low power consumption and high computing power edge computing characteristics, are gradually becoming the core hardware platform for automated cargo identification systems. They can support robots in performing integrated operations of cargo positioning, image acquisition, and data processing in complex scenarios.

[0003] In related technologies, the automated cargo identification system carried by ARM robots mostly relies on visual recognition technology to read barcode information. Specifically, it is implemented by: capturing barcode images through a camera, extracting key parameters such as bar width and spacing after image preprocessing, and then converting them into cargo information through a decoding algorithm.

[0004] The methods described above require identification of the entire barcode area. However, in automated cargo identification, packaging wrinkles, uneven surfaces, or deformation caused by compression during transportation can lead to problems such as localized distortion and edge breakage in the barcode area, making it impossible to obtain the complete barcode area. Methods such as image geometric correction can only handle slight linear distortions and cannot compensate for damaged conditions such as wrinkles, making it difficult to meet the accuracy requirements of automated operations. Summary of the Invention

[0005] This invention provides an automated cargo recognition system for ARM robots to solve the problem that in the prior art, wrinkles in cargo packaging, uneven surfaces, or compression deformation during transportation can cause local distortion and edge breakage in the barcode area. Methods such as image geometric correction can only handle slight linear distortion and cannot compensate for damage such as wrinkles.

[0006] The automated cargo identification system for an ARM robot of the present invention adopts the following technical solution:

[0007] One embodiment of the present invention provides an automated cargo identification system for an ARM robot, the system comprising the following modules:

[0008] The data acquisition module is used to acquire barcode images and identify barcode regions.

[0009] The processing module is used to determine the Y direction of the barcode length extension and the X direction perpendicular to it, extract the stripes, remove stripes in abnormal directions, and determine the reference length of the stripes.

[0010] The processing module is also used to detect the integrity of the barcode;

[0011] The processing module is further configured to, for a complete barcode, determine an initial stripe width and an initial stripe interval based on a first reference straight line parallel to the X direction and passing through the center point of the Y direction, denoted as the initial X direction width; determine an X reference width based on the initial X direction width; and determine a target stripe width and a target stripe interval based on the initial X direction width and the X reference width.

[0012] The processing module is further configured to, for incomplete barcodes, draw multiple second reference straight lines parallel to the X direction at preset intervals, select a temporary X-direction width based on the initial stripe width corresponding to the second reference straight line and the average angle between the stripe width and the stripe edge, and determine the X reference width based on the temporary X-direction width; traverse the stripes along the X direction on the second reference straight lines, and determine standard straight lines based on the multiple deviation of the temporary X-direction width of the stripe relative to the X reference width; set the stopping threshold of the standard straight lines based on the multiple cumulative deviation before the current stripe; and determine the target stripe width and target stripe interval based on the X reference width and the initial X-direction width corresponding to each of the standard straight lines.

[0013] The identification module is used to reconstruct the barcode and read cargo information based on the stripe reference length, the target stripe width, and the target stripe interval.

[0014] For example, the processing module achieves the determination of the Y direction of the barcode length extension and the X direction perpendicular to it and the extraction of stripes by executing the following method, including: filtering candidate straight lines with a length greater than a first preset threshold in the barcode area by a straight line detection method and identifying the direction of the candidate straight lines; setting the cluster number to 1, performing clustering operation on the direction of the candidate straight lines, determining the direction corresponding to the cluster center as the Y direction, and recording the direction perpendicular to the Y direction as the X direction; and extracting all stripes in the barcode area by a threshold segmentation algorithm.

[0015] For example, the processing module implements the removal of abnormal directional stripes and determination of stripe reference length by performing the following method: extracting two X-stripe edges in the X direction, calculating the shortest distance between each pixel point of the X-stripe edge and taking the average value, which is recorded as the stripe length; extracting target straight lines in the Y direction whose length is greater than the first preset threshold, calculating the average angle between each target straight line and the Y direction, which is recorded as the directional deviation between the stripe direction and the Y direction; removing abnormal directional stripes based on the directional deviation, and determining the average value of the stripe length of the remaining stripes after removing abnormal directional stripes as the stripe reference length.

[0016] For example, the processing module achieves the detection of barcode integrity by performing the following method: extracting all straight lines in the barcode area by a straight line detection method, and filtering out edge straight lines whose angle with the Y direction is less than a second preset threshold, and calculating the length variance of each edge straight line; if the length variance is less than a third preset threshold, the barcode is determined to be complete, otherwise the barcode is determined to be incomplete.

[0017] For example, the processing module implements the following method for determining the initial stripe width and initial stripe interval based on a first reference line parallel to the X direction and passing through the center point of the Y direction for a complete barcode, denoted as the initial X direction width, including: drawing the first reference line parallel to the X direction and passing through the center point of the Y direction in the barcode area; determining the length of the line segment intersecting each stripe with the first reference line as the initial stripe width; determining the length of the line segment formed by the first reference line between adjacent stripes as the initial stripe interval; and denoting the stripe width and the stripe interval as the initial X direction width.

[0018] For example, the processing module implements the determination of the X reference width based on the initial X direction width by executing the following method, including: merging the initial X direction widths with a width difference less than a fourth preset threshold into the same width set; for each initial X direction width, calculating the sum of the remainders of other initial X direction widths divided by the initial X direction width, and recording it as the remainder factor; determining the reference width probability based on the number of elements in the width set where the initial X direction width is located and the remainder factor, and determining the initial X direction width with the highest reference width probability as the X reference width.

[0019] For example, the processing module implements the determination of the target stripe width and target stripe interval based on the initial X-direction width and the X-reference width by executing the following method, including: determining a preset multiple set; traversing the preset multiples in the preset multiple set, and determining a candidate X-direction width based on the preset multiples and the X-reference width; determining the multiple deviation between the initial X-direction width and the X-reference width under the candidate X-direction width; and determining the candidate X-direction width with the smallest multiple deviation as the target stripe width or the target stripe interval corresponding to the initial X-direction width.

[0020] For example, the processing module implements the following method to draw multiple second reference lines parallel to the X direction at preset intervals, and select a temporary X-direction width based on the initial stripe width corresponding to the second reference lines and the average angle between the second reference lines and the stripe edges, including: drawing multiple second reference lines parallel to the X direction at preset intervals in the barcode area; obtaining two edge intersection points of the second reference lines and each stripe edge, and recording the length of the line segment of the second reference lines between the two edge intersection points of the same stripe as the initial stripe width of the corresponding stripe; calculating the average angle formed by the second reference lines at the two edge intersection points of the same stripe, and recording it as the average edge angle of the corresponding stripe; determining the initial stripe width of the stripe corresponding to the average edge angle whose absolute value of the difference between the angle and the right angle is less than a fifth preset threshold as the temporary X-direction width.

[0021] For example, the processing module implements the following method to traverse stripes along the X direction on the second reference line and determine a standard line based on the multiple deviation of the temporary X-direction width of the stripe relative to the X reference width, including: traversing each stripe in the barcode area along the X direction, extracting the temporary X-direction width corresponding to the current stripe on each of the second reference lines; determining a preset multiple set, determining candidate X-direction widths based on each preset multiple in the preset multiple set and the temporary X-direction width corresponding to the current stripe; determining the minimum multiple deviation of the temporary X-direction width from the X reference width under each of the candidate X-direction widths; and determining the second reference line corresponding to the temporary X-direction width with the smallest minimum multiple deviation as the current standard line.

[0022] For example, the processing module implements the stopping threshold of the standard straight line based on the multiple cumulative deviation before the current stripe by executing the following method, including: in the direction in which the current standard straight line traverses the stripe, determining the sum of the multiple deviations of a preset number of temporary X-direction widths and X-reference widths before the current stripe under the candidate X-direction width, and recording it as the multiple cumulative deviation; when the multiple cumulative deviation is greater than or equal to a sixth preset threshold, taking the intersection point of the left edge of the current standard straight line and the current stripe as the endpoint, and stopping the use of the current standard straight line.

[0023] The beneficial effects of the technical solution of the present invention are as follows:

[0024] In the automated cargo recognition system for an ARM robot provided by this invention, the processing module first determines the Y-direction, X-direction, and stripe reference length of the barcode. Then, it detects the integrity of the barcode. For complete barcodes, the X reference width is determined based on the initial X-direction width corresponding to a reference straight line passing through the center point of the Y-direction and parallel to the X-direction. The target stripe width and spacing are then determined based on the X reference width. For incomplete barcodes, a temporary X-direction width is selected based on multiple X-direction reference straight lines. The X reference width is determined based on this temporary X-direction width. A standard straight line is determined along the X-direction based on the multiple deviation of the temporary X-direction width from the X reference width. The target stripe width and spacing are obtained segmented using the standard straight line, avoiding the influence of local wrinkles on the stripe width and spacing. By processing complete and incomplete barcodes according to different scenarios, the limitations of existing technologies that rely on complete barcode areas or can only correct minor distortions are avoided. Therefore, even when barcodes are wrinkled or deformed due to cargo packaging, the system can still accurately identify barcode information, meeting the requirements of automated operations for recognition stability. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This invention provides a structural block diagram of an automated cargo identification system for an ARM robot. Detailed Implementation

[0027] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an automated cargo identification system for an ARM robot according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0029] The following description, in conjunction with the accompanying drawings, details the specific solution of an automated cargo identification system for an ARM robot provided by the present invention.

[0030] Please see Figure 1 The diagram illustrates a structural block diagram of an automated cargo identification system for an ARM robot according to an embodiment of the present invention. The system includes the following modules:

[0031] The acquisition module 110 is used to acquire barcode images and identify barcode areas;

[0032] The processing module 120 is used to determine the Y direction of the barcode length extension and the X direction perpendicular to it, extract the stripes, remove stripes in abnormal directions, and determine the reference length of the stripes.

[0033] The processing module 120 is also used to detect the integrity of the barcode;

[0034] The processing module 120 is also used to determine the initial stripe width and initial stripe interval for a complete barcode based on a first reference straight line parallel to the X direction and passing through the center point of the Y direction, denoted as the initial X direction width; determine the X reference width based on the initial X direction width; and determine the target stripe width and target stripe interval based on the initial X direction width and the X reference width.

[0035] The processing module 120 is further configured to, for incomplete barcodes, draw multiple second reference lines parallel to the X direction at preset intervals, select a temporary X-direction width based on the initial stripe width corresponding to the second reference line and the average angle between the stripe and the stripe edge, and determine the X reference width based on the temporary X-direction width; traverse the stripes along the X direction on the second reference lines, and determine standard lines based on the multiple deviation of the temporary X-direction width of the stripe relative to the X reference width; set the stopping threshold of the standard lines based on the multiple cumulative deviation before the current stripe; and determine the target stripe width and target stripe interval based on the X reference width and the initial X-direction width corresponding to each standard line.

[0036] The identification module 130 is used to reconstruct the barcode and read the cargo information based on the stripe reference length, target stripe width and target stripe interval.

[0037] The functions of each module of the aforementioned automated cargo recognition system for an ARM robot are described in detail below in one embodiment:

[0038] The aforementioned acquisition module 110 is used to acquire barcode images and identify barcode areas.

[0039] In this embodiment of the application, the barcode image is image data containing a goods barcode acquired by an imaging device; the barcode image needs to clearly show the black stripes and white intervals of the barcode used to carry goods information.

[0040] In this embodiment of the application, the barcode area is a rectangular area extracted from the captured barcode image that contains only the complete barcode (excluding interfering elements such as the background of the goods and packaging patterns).

[0041] Specifically, the acquisition module in this embodiment acquires visual information through an ARM robot. Relying on the low-power edge computing power of the ARM architecture and coupled with scene-appropriate visual hardware, it accurately acquires information such as images, dimensions, and positions of goods / environment, balancing real-time performance and battery life. This supports subsequent functions such as recognition, positioning, and operation. Its actual operation is as follows: The ARM robot uses Simultaneous Localization and Mapping (SLAM), QR code navigation, or Automated Guided Vehicle (AGV) magnetic tracks. The AGV (Automated Guided Vehicle) moves to the cargo area and uses infrared / laser sensors to confirm whether the cargo has entered the recognition range. When the cargo distance and posture meet the shooting requirements (e.g., the cargo enters the effective recognition distance of 0.5-1.5 meters and the barcode faces the camera), a trigger signal is sent to the ARM main control chip to start the vision acquisition process. The main control chip starts the camera to capture the barcode image, transmits the image data to the ARM local storage, and performs preprocessing such as denoising (Gaussian filtering), geometric correction (eliminating lens distortion), and grayscale conversion on the original image to optimize image quality and highlight the contrast of the black and white stripes of the barcode. Using semantic segmentation technology, a dedicated area containing only the barcode is extracted from the preprocessed barcode image. After verifying the validity of the stripes within the area by edge detection, the coordinates of the barcode area and the grayscale image data are transmitted to the processing module to provide accurate input for subsequent stripe parameter calculation.

[0042] A barcode is composed of regularly spaced black stripes, with the stripes aligned in the same direction. The cargo information it carries is expressed in the length, width, and spacing of the black stripes. Therefore, after the barcode image is acquired by the acquisition module and the barcode area is identified, this embodiment of the application uses a processing module to determine the stripe length, width, and spacing of the barcode as follows:

[0043] The aforementioned processing module 120 is used to determine the Y direction of the barcode length extension and the X direction perpendicular to it, extract the stripes, remove stripes in abnormal directions, and determine the reference length of the stripes.

[0044] In this embodiment of the application, the above-mentioned stripes refer to the regularly arranged black rectangular areas in the barcode, and their width, length and arrangement intervals are the core carriers of cargo information; the above-mentioned Y direction refers to the direction in which the stripe length in the barcode extends; the above-mentioned X direction is perpendicular to the Y direction and is the direction in which the stripe width extends.

[0045] For example, the above-mentioned determination of the Y direction of the barcode length extension and the X direction perpendicular to it and extraction of stripes can be achieved as follows: a straight line detection method is used to filter out candidate straight lines in the barcode area whose length is greater than a first preset threshold, and the direction of the candidate straight lines is identified; the number of clusters is set to 1, and a clustering operation is performed on the direction of the candidate straight lines, the direction corresponding to the cluster center is determined as the Y direction, and the direction perpendicular to the Y direction is recorded as the X direction; all stripes in the barcode area are extracted by a threshold segmentation algorithm.

[0046] In one specific implementation of this application, the process of determining the barcode direction and extracting stripes is as follows: All straight lines within the barcode area are identified using a straight line detection method (such as the Hough transform algorithm); candidate straight lines with a length greater than 30 pixels are selected (here, the first preset threshold is set to 30 pixels, which can be adjusted according to actual needs. The purpose of setting the first preset threshold is to eliminate short straight lines formed by the stripe width and background noise lines, retaining only long straight lines at the stripe edges; this needs to be determined comprehensively based on the physical structure characteristics of the barcode, image acquisition accuracy, and subsequent algorithm requirements), and the direction of the candidate straight lines is recorded. (Select the angle between the straight line and the horizontal direction); use the K-Means clustering algorithm, set the number of clusters K to 1, and cluster the direction angles of the candidate straight lines. The direction corresponding to the cluster center in the clustering result is recorded as the barcode Y direction, and the direction perpendicular to the Y direction is defined as the barcode X direction; use Otsu's method to binarize the grayscale image of the barcode area, automatically calculate the optimal grayscale threshold, and convert the grayscale image into a black and white binary image (black corresponds to stripes, and white corresponds to stripe intervals). Extract all black connected regions from the black and white binary image, which are the stripes, and record the pixel coordinate range of each stripe in the image.

[0047] In this embodiment, the above-mentioned abnormal directional stripes are stripes whose stripe direction deviates too much from the Y direction of the overall barcode. They are mostly stripe distortion caused by wrinkles in the goods and need to be removed to avoid affecting the accuracy of length calculation.

[0048] In this embodiment of the application, the above-mentioned stripe reference length refers to the average length of the remaining valid stripes after removing stripes in abnormal directions, which is used to reflect the actual length of the barcode.

[0049] Barcodes of different specifications have different stripe lengths, so stripe length is also an important element for information identification. However, barcodes affixed to goods may have distorted stripes due to factors such as unevenness of the goods surface. In this case, the stripe length cannot be directly represented by the length of the stripe edge. For this scenario, the embodiments of this application determine the actual stripe length (that is, the above-mentioned stripe reference length) based on the distance between the two edges of the stripe in the longitudinal direction.

[0050] For example, the above determination of the stripe reference length can be achieved as follows: extract the two X stripe edges in the X direction, calculate the shortest distance between each pixel point of the X stripe edge and take the average value, which is recorded as the stripe length; extract the target straight line in the Y direction whose length is greater than a first preset threshold, calculate the average angle between each target straight line and the Y direction, which is recorded as the direction deviation between the stripe direction and the Y direction; remove abnormal direction stripes based on the direction deviation, and determine the average stripe length of the remaining stripes after removing abnormal direction stripes as the stripe reference length.

[0051] In one specific implementation of this application embodiment, the above-mentioned removal of abnormal directional stripes and determination of stripe reference length is specifically implemented as follows: for each extracted stripe, identify its two edges in the X direction, denoted as edge XA and edge XB respectively; traverse each pixel on edge XA and calculate the shortest distance from each pixel to edge XB; take the average of the shortest distances of all pixels (this average distance is the length of a single stripe), and record the length value of each stripe (that is, the stripe length mentioned above); for each stripe, detect its edge straight lines again, and filter out target straight lines with a length greater than 30 pixels (maintaining the set value of the first preset threshold mentioned above). Consistency is ensured to guarantee the validity of the edge straight lines); the angle between each target straight line and the barcode's Y-direction is obtained, and the average of all angles is recorded as the directional deviation between the stripe direction and the Y-direction; among all the directional deviations between the stripes and the Y-direction, stripes with abnormal directions are removed according to a rule (collect all stripe directional deviation values, calculate the mean μ and standard deviation of the directional deviation values; stripes with directional deviation values ​​exceeding the range of μ±3 are judged as abnormal directional stripes and removed from the stripe list); for the remaining valid stripes after removing abnormal directional stripes, their stripe lengths are collected, and the average of all valid stripe lengths is calculated, and this average length is recorded as the stripe reference length.

[0052] After determining the direction and length of the stripes in the barcode area, information recognition is required. However, in practice, wrinkles or other factors during packaging can result in incomplete barcodes. Therefore, before information recognition, a barcode integrity check is necessary. This integrity check process is implemented by the processing module as follows:

[0053] The aforementioned processing module 120 is also used to detect the integrity of the barcode.

[0054] For example, the above process for detecting the integrity of a barcode can be implemented as follows: extract all straight lines in the barcode area using a straight line detection method, and select edge lines whose angle with the Y direction is less than a second preset threshold, and calculate the length variance of each edge line; if the length variance is less than a third preset threshold, the barcode is determined to be complete, otherwise the barcode is determined to be incomplete.

[0055] In one specific implementation of this application embodiment, the above-mentioned process of detecting barcode integrity is specifically implemented as follows: extracting all straight lines in the barcode area using the Hough transform line detection algorithm, and selecting straight lines with an angle of less than 30° with the Y direction as the edge lines of the stripes in the Y direction (here, the second preset threshold is set to 30°, which can be adjusted according to actual needs; its value selection should ensure that it neither over-filters effective edges nor effectively eliminates the interference of most packaging wrinkles, background patterns, stains, and other non-barcode area edges on integrity detection); obtaining the pixel length of each edge line in the image, and calculating the variance of all pixel lengths; the smaller the calculated length variance, the closer the lengths of each edge line are, that is, the edge lengths in the Y direction are consistent, without obvious breaks, and the barcode is regular; if the value of the above length variance is less than 3 (here, the third preset threshold is set to 3, which can be adjusted according to actual needs; specifically, it can be set according to the empirical value of the dispersion of the edge length of the complete barcode), then the current barcode area is determined to be complete; otherwise, the current barcode area is determined to be incomplete.

[0056] If the barcode area is determined to be complete through the above process, the automated cargo identification system provided in this application embodiment can directly identify the cargo information it carries based on the captured image information through the above processing module, as detailed below:

[0057] The processing module 120 is also used to determine the initial stripe width and initial stripe interval for a complete barcode based on a first reference straight line parallel to the X direction and passing through the center point of the Y direction, denoted as the initial X direction width; determine the X reference width based on the initial X direction width; and determine the target stripe width and target stripe interval based on the initial X direction width and the X reference width.

[0058] In this embodiment of the application, the initial stripe width refers to the pixel length of the intersection area between the first reference line and the black stripe of the barcode; the initial stripe interval refers to the pixel length of the intersection area between the first reference line and the white interval (blank area between stripes) of the barcode; the initial X-direction width is a general term for the initial stripe width and the initial stripe interval, which is a set of parameters reflecting the alternation pattern of stripes and spaces based on the first reference line.

[0059] Specifically, the process of determining the initial X-direction width described above can be implemented as follows: draw a first reference straight line parallel to the X-direction and passing through the center point of the Y-direction in the barcode area; determine the pixel length of the line segment intersecting each stripe with the first reference straight line as the initial stripe width of the corresponding stripe; determine the pixel length of the line segment formed by the first reference straight line between adjacent stripes as the initial stripe interval of the corresponding adjacent stripes; and record the stripe width and stripe interval as the initial X-direction width.

[0060] In the embodiments of this application, the above-mentioned X reference width is a fixed reference value that defines the stripe width and stripe interval of the barcode; the stripe width and stripe interval of the barcode must be an integer multiple of the X reference width; for example, in the EAN-13 code, the stripe width and stripe interval are 1, 2, 3, and 4 times its X reference width.

[0061] For example, the above process of determining the X reference width based on the initial X direction width can be implemented as follows: merge the initial X direction widths with a width difference less than the fourth preset threshold into the same width set; for each initial X direction width, calculate the sum of the remainders of other initial X direction widths divided by the initial X direction width, and record it as the remainder factor; determine the reference width probability based on the number of elements in the width set where the initial X direction width is located and the remainder factor, and determine the initial X direction width with the highest reference width probability as the X reference width.

[0062] In one specific implementation of this application embodiment, the above-mentioned determination of the X reference width based on the initial X direction width is implemented as follows: traversing the initial X direction widths determined based on the first reference line, initial X direction widths with a width difference of less than 3 pixels (here, the fourth preset threshold is set to 3 pixels, which can be adjusted according to actual needs; its purpose is to offset small systematic errors caused by factors such as shooting noise) are grouped into the same set, resulting in multiple width sets. For example, initial X direction widths with a width of 2 pixels and a width of 3 pixels (width difference of 1 pixel, less than 3 pixels) are grouped into the same width set; since the design rule for the X direction width is a multiple of the X reference width, for any initial X direction width, the larger the other initial X direction widths whose width is an integer multiple of it, the greater the probability that it is the X reference width. Specifically, this can be quantified by the above-mentioned remainder factor. Taking the initial X direction width as the current candidate X reference width as an example, the remainder factor is calculated by the following formula:

[0063]

[0064] Where, is the remainder factor, which is the sum of the remainders of all initial X-direction widths and candidate reference widths; is the i-th initial X-direction width; is the current candidate X-reference width; n is the total number of initial X-direction widths determined based on the first reference line; is the remainder operation; the smaller the remainder factor calculated by this formula, the closer the initial X-direction width is to an integer multiple of the current candidate reference width, that is, the more the current candidate reference width conforms to the barcode encoding logic, and the greater the probability that it is the X-reference width.

[0065] Furthermore, since the barcode encoding logic uses the X-reference width as the smallest unit (module), and defines the width of each stripe and the interval between adjacent stripes by combining the number of modules, the more initial X-direction widths are contained in the width set containing the current initial X-direction width, the greater the probability that it is the X-reference width. Therefore, the probability of the reference width can be determined by combining the number of elements in the width set containing the current initial X-direction width and the aforementioned remainder factor as follows:

[0066]

[0067] in, This is the probability that the current initial width in the X direction is the reference width. The larger the value, the greater the probability that the current initial width in the X direction is the reference width. This represents the number of elements in the width set containing the current initial X-direction width. The larger the value, the greater the probability that the current initial X-direction width is the X reference width. This is the remainder factor corresponding to the current initial X-direction width. The smaller the value, the greater the probability that the current initial X-direction width is the X reference width. The goal of adding 1 to the denominator is to avoid the denominator being 0.

[0068] After determining the reference width probabilities of all initial X-direction widths using the above method, this embodiment of the application uses the initial X-direction width with the highest reference width probability as the X reference width. And based on the X reference width To identify the width and spacing of barcodes.

[0069] For example, the above determination of the target stripe width and target stripe interval based on the initial X-direction width and the X-reference width can be achieved as follows: determine a preset set of multiples; traverse the preset multiples in the preset set of multiples, and determine the candidate X-direction width based on the preset multiples and the X-reference width; determine the multiple deviation between the initial X-direction width and the X-reference width under the candidate X-direction width; and determine the candidate X-direction width with the smallest multiple deviation as the target stripe width or target stripe interval corresponding to the initial X-direction width.

[0070] In one specific implementation of this application embodiment, the determination of the target stripe width and target stripe interval is achieved as follows: A preset set of multiples is determined according to the barcode encoding rules. Taking EAN-13 code as an example, its encoding rules design the bar and space dimensions to be 1-4 times the X reference width, so its corresponding preset set of multiples is [1,2,3,4]. For each initial X-direction width, the multiple deviation between it and the X reference width under the candidate X-direction width is calculated using the following formula:

[0071]

[0072] in, The initial X-direction width of the i-th element is taken as a preset multiple. The multiple deviation at time, where, The value needs to be obtained by traversing the preset set of multiples. Taking EAN-13 code as an example, it needs to be calculated sequentially. The corresponding multiple deviations for values ​​of 1, 2, 3, and 4 ; The width in the i-th initial X direction; The value of the current preset multiple; X is the baseline width; The initial X-direction width is a preset multiple. The corresponding candidate X-direction width; the smaller the multiple deviation calculated by the above formula, the greater the corresponding candidate X-direction width. The greater the probability of the target stripe width or target stripe interval corresponding to the initial X-direction width.

[0073] In this embodiment, multiple multiple deviations are obtained by traversing a preset set of multiples using the above formula. Afterwards, When taking the minimum value, the corresponding Let the width in the i-th initial X direction be denoted as . The true multiple, and compare this true multiple with the reference width. The product of as The indicated stripe width (i.e., the target stripe width) or stripe interval (i.e., the target stripe interval).

[0074] When a barcode area is determined to be incomplete, the current barcode may have local wrinkles or other areas. A straight line in the X direction may pass through these wrinkles or other local deviation areas. Therefore, it is not possible to directly use a single straight line in the X direction to determine the width of all stripes in the X direction and the distance between adjacent stripes in the X direction. In this case, the automated cargo identification system provided in this application embodiment can draw multiple straight lines in the X direction using the aforementioned processing module, and select appropriate line segments based on the stripe width distribution to obtain the stripe width in the X direction and the distance between adjacent stripes in the X direction. The specific explanation is as follows:

[0075] The processing module 120 is further configured to, for incomplete barcodes, draw multiple second reference lines parallel to the X direction at preset intervals, select a temporary X-direction width based on the initial stripe width corresponding to the second reference line and the average angle between the stripe width and the stripe edge, and determine the X reference width based on the temporary X-direction width; traverse the stripes along the X direction on the second reference lines, and determine standard lines based on the multiple deviation of the temporary X-direction width of the stripes relative to the X reference width; set the stopping threshold of the standard lines based on the multiple cumulative deviation of the current stripe; and determine the target stripe width and target stripe interval based on the X reference width and the initial X-direction width corresponding to each standard line.

[0076] In this embodiment, the aforementioned second reference line is a series of horizontal lines that are parallel to the X direction of the barcode and are evenly distributed longitudinally within the barcode area at preset intervals. It is the core carrier of the segmented sampling bar size of the incomplete barcode and is used to avoid sampling deviations caused by local wrinkles.

[0077] In this embodiment, the temporary X-direction width is the effective width data selected from the initial stripe width obtained by sampling based on the second reference line, and is used to determine the X reference width in subsequent steps.

[0078] For example, the above-mentioned method of drawing multiple second reference lines parallel to the X direction at preset intervals and selecting a temporary X-direction width based on the initial stripe width corresponding to the second reference lines and the average angle between them and the stripe edges can be implemented as follows: draw multiple second reference lines parallel to the X direction at preset intervals in the barcode area; obtain the two edge intersection points of the second reference lines and each stripe edge, and record the length of the line segment of the second reference lines between the two edge intersection points of the same stripe as the initial stripe width of the corresponding stripe; calculate the average angle formed by the second reference lines at the two edge intersection points of the same stripe, and record it as the average edge angle of the corresponding stripe; determine the initial stripe width of the stripe corresponding to the average edge angle whose absolute value of the difference between the angle and the right angle is less than a fifth preset threshold as the temporary X-direction width.

[0079] In one specific implementation of this application embodiment, the determination of the temporary X-direction width is achieved as follows: A preset interval of 10 pixels is set (this can be customized according to actual scenario requirements, ensuring that the generated second reference line covers the entire vertical area of ​​the barcode). A second reference line parallel to the X-direction and traversing the entire X-direction range of the barcode is generated every 10 pixels in the Y-direction, forming a set of second reference lines. Each second reference line is traversed, and the two intersection points between the second reference line and the stripe edge are recorded as edge intersection points. The distance between the two edge intersection points of the same stripe and the second reference line is calculated and recorded as the initial stripe width of the corresponding stripe. The included angle corresponding to the two edge intersection points of the second reference line at the same stripe is recorded as the included angle of the corresponding stripe, and the average of the two included angles is calculated and recorded as the average included angle. Calculate the mean of the included angles of each edge. The absolute value of the difference from the right angle, since the angle formed by the intersection of the stripe edge and the second reference line is usually a right angle, is the mean value of the edge angle. The smaller the absolute value of the difference from the right angle, the more realistic the initial stripe width of the corresponding stripe; the average of the edge angles whose absolute value of the difference from the right angle is less than 3 (here, the fifth preset threshold mentioned above is set to 3, which can be adjusted according to actual scene factors such as barcode printing accuracy, shooting device resolution, and flatness of the goods surface). The corresponding initial stripe width is determined as the temporary X-direction width. All the retained temporary X-direction widths are sorted and organized according to their respective second reference lines. Each second reference line corresponds to a set of temporary X-direction widths.

[0080] After filtering out the temporary X-direction width using the above method, the X-reference width can be determined based on the filtered temporary X-direction width. It should be noted that the implementation principle of determining the X-reference width here is the same as the implementation principle of determining the X-reference width when the barcode is complete, as described above, and will not be repeated here.

[0081] In this embodiment of the application, the aforementioned standard straight line refers to the reference straight line from which valid empty data is extracted in segments selected from multiple second reference straight lines.

[0082] For example, the above-mentioned method of traversing stripes along the X direction on the second reference line and determining the standard line based on the multiple deviation of the temporary X-direction width of the stripe relative to the X reference width can be achieved as follows: traversing each stripe in the barcode area along the X direction, extracting the temporary X-direction width corresponding to the current stripe on each second reference line; determining a preset multiple set, and determining candidate X-direction widths based on each preset multiple in the preset multiple set and the temporary X-direction width corresponding to the current stripe; determining the minimum multiple deviation between the temporary X-direction width and the X reference width under each candidate X-direction width; and determining the second reference line corresponding to the temporary X-direction width with the smallest minimum multiple deviation as the current standard line.

[0083] The stopping threshold of the above standard straight line is set based on the cumulative deviation of the current stripe as follows: In the direction of the current standard straight line traversing the stripe, the sum of the multiple deviations of the current standard straight line from the X-direction width and the X-reference width under the candidate X-direction width of the preset number of temporary X-direction widths before the current stripe is determined and recorded as the multiple cumulative deviation; when the multiple cumulative deviation is greater than or equal to the sixth preset threshold, the intersection of the left edge of the current standard straight line and the current stripe is taken as the endpoint and the current standard straight line is stopped.

[0084] In one specific implementation of this application embodiment, the determination of the standard straight line is achieved by: traversing the stripes in the barcode area along the barcode width extension direction, and extracting the temporary X-direction width corresponding to each second reference straight line for each stripe; for each extracted temporary X-direction width, traversing a preset multiple set (the setting basis is the same as above, and will not be repeated here), and calculating the multiple deviation of the temporary X-direction width relative to the X reference width. Taking the first temporary X-direction width corresponding to the j-th second reference straight line as an example, the multiple deviation is calculated using the following formula:

[0085]

[0086] in, The deviation of the first temporary X-direction width (the temporary X-direction width corresponding to the first stripe along the width extension direction) of the j-th second reference line from the X reference width at each preset multiple; The width of the first temporary X-direction corresponding to the j-th second reference line; To calculate the preset multiplier, taking EAN-13 code as an example, the calculations must be performed sequentially. The corresponding multiple deviations for values ​​of 1, 2, 3, and 4 ; X is the baseline width; This is the candidate X-direction width corresponding to the temporary X-direction width at the current preset multiple.

[0087] After determining the multiple deviations relative to the X-reference width for each second reference line using the above formula, the minimum multiple deviation for each second reference line is obtained, and the second reference line with the smallest minimum multiple deviation is determined as the current standard line. Following a left-to-right order (the direction of barcode width extension), for each temporary X-direction width corresponding to the current standard line, the first two (the preset number is set to 2, which can be adjusted according to actual scenario requirements) temporary X-direction widths relative to the X-reference width are calculated. The cumulative deviation by multiples is as follows:

[0088]

[0089] in, The current temporary X-direction width is the width of the previous two temporary X-directions relative to the X-reference width on the current standard line. The cumulative deviation of multiples; The minimum multiple deviation of the first temporary X-direction width relative to the X-reference width ∆X, which corresponds to the current temporary X-direction width on the current standard straight line; This is the minimum multiple deviation of the second temporary X-direction width relative to the X-reference width ∆X, which corresponds to the current temporary X-direction width on the current standard line. The smaller the cumulative multiple deviation calculated by this formula, the higher the matching degree between the current standard line's width sampling data and the X-reference width, and the more suitable the current standard line is to continue as the standard line. It should be noted that the cumulative deviation corresponding to the first temporary X-direction width is its own minimum multiple deviation, while the cumulative deviation corresponding to the second temporary X-direction width is the sum of the first temporary X-direction width and its own minimum multiple deviation.

[0090] The cumulative deviation of each temporary X-direction width corresponding to the standard straight line relative to the X reference width ∆X is calculated using the above method. When the cumulative deviation is less than the preset stop threshold (i.e., the sixth preset threshold mentioned above, which can be set by combining factors such as X reference width, barcode quality, and scene characteristics), the current standard straight line is used. When the cumulative deviation is greater than or equal to the preset stop threshold, the current standard straight line is stopped, and the intersection of the current standard straight line and the left edge of the current stripe is taken as its endpoint to form a standard straight line segment. Starting from the current stripe, a new second reference straight line is re-determined as the standard straight line based on the above method. The cumulative deviation is calculated and it is determined whether to segment until all stripes of the barcode are covered.

[0091] In this embodiment, the above process ultimately yields multiple continuous standard straight line segments. Each standard straight line segment corresponds to a wrinkle-free local area in the barcode. All standard straight line segments can cover all stripe widths and stripe intervals in the barcode. The stripe width and stripe interval corresponding to each standard straight line segment are determined as the initial X-direction width. Similarly, based on the corresponding method in the above complete barcode processing steps, the target stripe width and target stripe interval are determined based on the X reference width and the initial X-direction width corresponding to each standard straight line.

[0092] After determining the barcode's stripe reference length, target stripe width, and target stripe interval through the above process, this embodiment of the application obtains the goods information corresponding to the barcode through the above identification module as follows:

[0093] The identification module 130 is used to reconstruct the barcode and read the cargo information based on the stripe reference length, target stripe width and target stripe interval.

[0094] In this embodiment of the application, after reconstructing the barcode and reading the corresponding goods information, the goods information can be compared and verified with the background database. After confirming the goods information, the ARM chip sends instructions to the execution mechanism (such as a robotic arm or AGV drive module) to complete the sorting, handling or warehousing actions. After the action is completed, the execution result is fed back through the sensor. The ARM robot either moves to the next goods area or pauses to wait for a new task.

[0095] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An automated cargo identification system for an ARM robot, characterized in that, The system includes: The data acquisition module is used to acquire barcode images and identify barcode regions. The processing module is used to determine the Y direction of the barcode length extension and the X direction perpendicular to it, extract the stripes, remove stripes in abnormal directions, and determine the reference length of the stripes. The processing module is also used to detect the integrity of the barcode; The processing module is further configured to, for a complete barcode, determine an initial stripe width and an initial stripe interval based on a first reference straight line parallel to the X direction and passing through the center point of the Y direction, denoted as the initial X direction width; determine an X reference width based on the initial X direction width; and determine a target stripe width and a target stripe interval based on the initial X direction width and the X reference width. The processing module is further configured to, for incomplete barcodes, draw multiple second reference straight lines parallel to the X direction at preset intervals, select a temporary X-direction width based on the initial stripe width corresponding to the second reference straight line and the average angle between the stripe width and the stripe edge, and determine the X reference width based on the temporary X-direction width; traverse the stripes along the X direction on the second reference straight lines, and determine standard straight lines based on the multiple deviation of the temporary X-direction width of the stripe relative to the X reference width; set the stopping threshold of the standard straight lines based on the multiple cumulative deviation before the current stripe; and determine the target stripe width and target stripe interval based on the X reference width and the initial X-direction width corresponding to each of the standard straight lines. The identification module is used to reconstruct the barcode and read cargo information based on the stripe reference length, the target stripe width, and the target stripe interval; The processing module implements the following method for determining the initial stripe width and initial stripe interval for a complete barcode based on a first reference straight line parallel to the X direction and passing through the center point of the Y direction, denoted as the initial X direction width: Draw the first reference straight line in the barcode area that is parallel to the X direction and passes through the center point of the Y direction; The length of the line segment intersecting each stripe with the first reference line is determined as the initial stripe width, the length of the line segment formed by the first reference line between adjacent stripes is determined as the initial stripe interval, and the stripe width and the stripe interval are denoted as the initial X-direction width. The processing module implements the following method to create multiple second reference lines parallel to the X direction at preset intervals, and selects a temporary X-direction width based on the initial stripe width corresponding to the second reference line and the average angle between the stripe width and the stripe edge: Multiple second reference straight lines parallel to the X direction are drawn at preset intervals in the barcode area; Obtain the two edge intersection points of the second reference line with the edge of each stripe, and record the length of the line segment of the second reference line between the two edge intersection points of the same stripe as the initial stripe width of the corresponding stripe; Calculate the mean of the angle formed by the second reference line at the intersection of the two edges of the same stripe, and record it as the mean of the edge angle of the corresponding stripe; The initial stripe width of the stripe corresponding to the average edge angle where the absolute value of the difference between the right angle and the right angle is less than a fifth preset threshold is determined as the temporary X-direction width; The aforementioned X reference width is a fixed reference value that defines the stripe width and stripe spacing of the barcode; the stripe width and stripe spacing of the barcode must be an integer multiple of this X reference width.

2. The automated cargo identification system for ARM robots according to claim 1, characterized in that, The processing module determines the Y-direction of the barcode length extension and the X-direction perpendicular to it, and extracts the stripes by executing the following method: Candidate straight lines with a length greater than a first preset threshold are selected from the barcode area using a straight line detection method, and the direction of the candidate straight lines is identified. Set the number of clusters to 1, perform clustering operation on the candidate straight line direction, determine the direction corresponding to the cluster center as the Y direction, and record the direction perpendicular to the Y direction as the X direction; All stripes in the barcode region are extracted using a threshold segmentation algorithm.

3. The automated cargo identification system for ARM robots according to claim 2, characterized in that, The processing module removes abnormal directional stripes and determines the stripe reference length by executing the following methods: Extract the two X-stripe edges in the X direction, calculate the shortest distance between each pixel point of the X-stripe edge and take the average value, which is recorded as the stripe length; Extract target straight lines whose stripe length in the Y direction is greater than the first preset threshold, calculate the average angle between each target straight line and the Y direction, and record it as the directional deviation between the stripe direction and the Y direction; Abnormal directional stripes are removed based on the directional deviation, and the average length of the remaining stripes after removing abnormal directional stripes is determined as the stripe reference length.

4. The automated cargo identification system for an ARM robot according to claim 1, characterized in that, The processing module achieves the detection of barcode integrity by executing the following methods: All straight lines in the barcode area are extracted using a straight line detection method, and edge straight lines with an angle less than a second preset threshold with the Y direction are selected. The length variance of each edge straight line is then calculated. If the length variance is less than a third preset threshold, the barcode is determined to be complete; otherwise, the barcode is determined to be incomplete.

5. The automated cargo identification system for an ARM robot according to claim 1, characterized in that, The processing module implements the determination of the X reference width based on the initial X direction width by executing the following method: The initial X-direction widths with a width difference less than the fourth preset threshold are merged into the same width set; For each of the initial X-direction widths, calculate the sum of the remainders of the other initial X-direction widths divided by the initial X-direction width, and denot it as the remainder factor; The reference width probability is determined based on the number of elements in the width set containing the initial X-direction width and the remainder factor, and the initial X-direction width with the highest reference width probability is determined as the X reference width.

6. The automated cargo identification system for an ARM robot according to claim 5, characterized in that, The processing module implements the determination of the target stripe width and target stripe interval based on the initial X-direction width and X-reference width by executing the following method, including: Determine the preset set of multiples; Traverse the preset multiples in the preset multiple set, and determine the candidate X-direction width based on the preset multiples and the X reference width; Determine the multiple deviation between the initial X-direction width and the X-reference width under the candidate X-direction width; The candidate X-direction width with the smallest multiple deviation is determined as the target stripe width or the target stripe interval corresponding to the initial X-direction width.

7. The automated cargo identification system for an ARM robot according to claim 1, characterized in that, The processing module implements the following method to traverse the stripes along the X direction on the second reference line, and determines the standard line based on the multiple deviation of the temporary X-direction width of the stripes relative to the X reference width, including: Traverse each stripe in the barcode region along the X direction and extract the temporary X-direction width of the current stripe on each of the second reference lines; A preset set of multiples is determined, and a candidate X-direction width is determined based on each preset multiple in the preset set and the temporary X-direction width corresponding to the current stripe; Determine the minimum multiple deviation between the temporary X-direction width and the X-reference width under each of the candidate X-direction widths; The second reference line corresponding to the temporary X-direction width with the smallest minimum multiple deviation is determined as the current standard line.

8. The automated cargo identification system for an ARM robot according to claim 7, characterized in that, The processing module sets the stop threshold of the standard straight line based on the cumulative deviation of the previous multiple before the current stripe by executing the following method: In the direction of the current standard straight line traversing the stripes, determine the sum of the multiple deviations of the temporary X-direction width and the X-reference width of the previous preset number of the current stripes under the candidate X-direction width, and denot it as the multiple cumulative deviation; When the cumulative deviation of the multiple is greater than or equal to the sixth preset threshold, the intersection of the current standard line and the left edge of the current stripe is taken as the endpoint, and the current standard line is stopped from being used.

Images