A DPM Metal QR Code Decoding Method Based on Light Source Direction Sensitivity Differences

Abstract

This application discloses a DPM metal QR code decoding method based on the difference in light source direction sensitivity. The method includes: acquiring multiple light source images by sequentially activating N groups of light sources from different orientations, and dividing the code area into candidate code point regions; calculating the direction sensitivity index based on the grayscale response of pixels in the candidate code point regions under different light sources, and determining the diversion threshold through histogram statistics, dividing the candidate code point regions into depth-dominant and reflectivity-dominant types; generating code point confidence scores for each type; generating code point confidence scores for reflectivity-dominant candidate code point regions based on reflectivity contrast; arranging the code point confidence scores of all candidate code point regions according to spatial position to form a code point confidence map, and inputting it into a neural network decoding model to obtain the code character recognition result. This invention can improve the accuracy and stability of DPM QR code decoding on complex metal surfaces.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a DPM metal QR code decoding method based on the difference in light source direction sensitivity. Background Technology

[0002] Direct Part Marking (DPM) QR codes are widely used in the production, circulation, and traceability of metal workpieces. For metal surface QR codes such as rammed pin codes and casting codes, the code dots are usually formed directly on the workpiece surface, and their recognition results are easily affected by the combined influence of the code dot geometry and surface reflectivity. In actual production processes, due to factors such as rammed pin wear, localized mold blockage, and material hardness fluctuations, different code dots often differ in depth, edge morphology, and surface roughness. Simultaneously, the workpiece surface may also have uneven oxidation, oil residue, and mold release agent adhesion, leading to changes in local reflectivity. The combined effect of these factors can easily cause unstable imaging differences between the code dots and the background.

[0003] Existing DPM metal QR code recognition methods are mostly based on single-image grayscale comparison, fixed threshold segmentation, or uniformly using photometric stereoscopic methods to reconstruct the surface morphology before judgment. The former is quite sensitive to changes in lighting conditions and surface conditions, and is prone to missed or false detections when the code points are shallow or the background reflection is uneven. Although the latter can extract surface undulation features using multi-directional illumination information, in some shallow code points or areas with significant reflectivity differences, grayscale changes are not mainly caused by geometric depth. If they are still uniformly processed according to morphology information, it is easy to cause depth estimation distortion, which in turn leads to code point boundary offset, decreased confidence, and overall decoding failure. Especially in complex situations such as shallow indentations, weak protrusions, local high reflectivity, and pollution coverage on metal surfaces, existing methods lack targeted processing for different types of code point signals, making it difficult to balance decoding accuracy and stability. Summary of the Invention

[0004] This application provides a DPM metal QR code decoding method based on the difference in light source direction sensitivity, so as to at least solve some of the technical problems existing in the related technologies described above.

[0005] According to a first aspect of the embodiments of this application, a DPM metal QR code decoding method based on the difference in light source direction sensitivity is provided, including: N light sources from different directions are turned on sequentially to acquire N light source images, and candidate code point regions are divided within their code areas, where N>1; The gray values ​​of pixels within the candidate code point region are taken in order of light source number to form a light source response sequence. The directional sensitivity index is obtained by averaging the ratio of the gray standard deviation to the gray mean within the region. Histogram statistics are performed on the directional sensitivity indices of all candidate code point regions in the current code region. The diversion threshold is obtained from the position where the inter-class variance is maximized. Those with directional sensitivity indices not less than the diversion threshold are judged as depth-dominant, and those with diversion indices less than the threshold are judged as reflectivity-dominant. For depth-dominant candidate code point regions, photometric stereo processing is performed based on the N light source images to obtain the depth difference and depth estimation signal-to-noise ratio, thereby generating the code point confidence of the region. For reflectance-dominant candidate code point regions, the average gray value under the N light source images is used as the reflectance approximation. The background reflectance approximation is obtained in the same way for the surrounding non-code point neighborhood. The ratio of the difference between the two and the background reflectance approximation is used as the reflectance contrast. The code point confidence of the region is generated based on its absolute value. The code point confidence scores of all candidate code point regions are arranged according to their spatial location to form a code point confidence map, which is then fed into a neural network decoding model to obtain the codeword recognition result.

[0006] As an optional approach, the histogram statistics of the directional sensitivity index of all candidate code point regions in the current code region include: determining the number of bins in the histogram by taking the square root of the total number of candidate code point regions in the current code region; and when the histogram is unimodal and the diversion threshold cannot be obtained based on the position where the inter-class variance is maximized, uniformly processing all candidate code point regions in the current code region according to the depth-dominant type.

[0007] As an optional approach, photometric stereo processing is performed on the N light source images to obtain the depth difference and depth estimation signal-to-noise ratio, including: solving the surface normal vector of each pixel in the region based on the direction vector of each light source and the grayscale response of each pixel in the depth-dominant candidate code point region and its surrounding non-code point neighborhood in the N light source images; integrating the normal vector field formed by the surface normal vectors to obtain the relative depth distribution of the region relative to its surrounding non-code point neighborhood; taking the absolute value of the difference between the relative depth at the center of the region and the average relative depth of the surrounding non-code point neighborhood as the depth difference; and taking the reciprocal of the ratio of the fluctuation amplitude of the relative depth estimate in the region to the depth difference and a small constant as the depth estimation signal-to-noise ratio.

[0008] As an optional approach, the peripheral non-code point neighborhood is configured as a rectangular neighborhood surrounding the outer edge of the reflectivity-dominant candidate code point region, the width of which is selected according to a set ratio of the feature size of the candidate code point region, and does not overlap with adjacent candidate code point regions.

[0009] As an optional approach, for depth-dominant candidate code point regions, generating code point confidence scores for those regions includes: inputting the depth difference and the depth estimation signal-to-noise ratio into a preset monotonically non-decreasing mapping function, and using the output of the monotonically non-decreasing mapping function as the code point confidence scores corresponding to the depth-dominant candidate code point regions.

[0010] As an optional approach, for reflectivity-dominant candidate code point regions, generating code point confidence scores for those regions includes: inputting the absolute value of the reflectivity contrast into a preset monotonically non-decreasing mapping function, and using the output of the monotonically non-decreasing mapping function as the code point confidence score corresponding to the reflectivity-dominant candidate code point region.

[0011] As an optional approach, the neural network decoding model includes a convolutional feature extraction module and a fully connected classification module cascaded in sequence; the convolutional feature extraction module includes at least one convolutional layer and a pooling layer that works in conjunction with it, and the input of the convolutional feature extraction module is the code point confidence map; the fully connected classification module includes at least one fully connected layer, the output layer size of the fully connected classification module corresponds to the codeword dictionary size, and the output of the fully connected classification module is the codeword recognition result.

[0012] As an optional approach, the neural network decoding model is obtained through offline training of training sample pairs with codeword annotations, wherein the input of each training sample pair is the generated code point confidence map, and the corresponding annotation is the codeword.

[0013] As an optional approach, if the proportion of candidate code point regions with values ​​near a preset lower limit in the code point confidence map exceeds a preset ratio, the system switches to the next set of parameters in the preset light source combination parameter library, re-acquires N light source images, and again performs the following steps: dividing the candidate code point regions, obtaining the direction sensitivity index, obtaining the diversion threshold, generating the code point confidence of depth-dominant and reflectivity-dominant candidate code point regions, constructing the code point confidence map, and feeding the reconstructed code point confidence map into the neural network decoding model to update the codeword recognition results.

[0014] According to a second aspect of the embodiments of this application, a DPM metal QR code decoding system based on the difference in light source direction sensitivity is also provided, comprising: The image acquisition and region segmentation module is used to sequentially activate N groups of light sources from different azimuths to acquire N light source images, and to segment candidate code point regions within their code areas, where N>1; The orientation sensitivity index calculation module is used to take the gray values ​​of the pixels in the candidate code point area in the order of the light source number to form a light source response sequence, and take the average of the ratio of the gray standard deviation to the gray mean in the area to obtain the orientation sensitivity index. The dominant type determination module is used to perform histogram statistics on the directional sensitivity index of all candidate code point regions in the current code region, and obtain the diversion threshold from the position where the inter-class variance is maximized; those with directional sensitivity index not less than the diversion threshold are judged as depth dominant, and those with reflectivity dominant are judged as reflectivity dominant. The depth-dominant confidence generation module is used to perform photometric stereo processing on the N light source images to obtain the depth difference and depth estimation signal-to-noise ratio for the depth-dominant candidate code point region, and generate the code point confidence of the region accordingly. The reflectance-dominant confidence generation module is used to approximate the reflectance of a reflectance-dominant candidate code point region by using its grayscale mean value under the N light source images as the reflectance approximation, and to obtain the background reflectance approximation of the surrounding non-code point neighborhood in the same way. The ratio of the difference between the two and the background reflectance approximation is used as the reflectance contrast, and the absolute value of the contrast is used to generate the code point confidence of the region. The decoding module is used to arrange the code point confidence scores of all candidate code point regions according to their spatial positions to form a code point confidence map, which is then fed into the neural network decoding model to obtain the codeword recognition result.

[0015] This application determines the directional sensitivity of candidate code point regions to different lighting orientations, then adaptively splits them into depth-dominant and reflectivity-dominant types for separate processing. Furthermore, the two types of determination results are unified into a code point confidence map before being fed into a neural network decoding model. This allows for the selection of more suitable determination criteria based on the actual causes of different code points, effectively suppressing the interference of reflectivity changes on depth estimation in shallow code point scenarios, mitigating misjudgments caused by factors such as uneven surface oxidation, oil residue, impactor wear, and localized mold blockage, thereby improving code point positioning accuracy and code character recognition stability. Simultaneously, the splitting threshold is adaptively determined based on the current statistical distribution of the code area, and its applicability can be expanded by combining a light source combination switching mechanism. This invention has strong adaptability to different batches of workpieces, different surface conditions, and complex lighting conditions, and can improve the reliability of metal DPM QR code decoding.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Furthermore, no embodiment in this disclosure is required to achieve all the effects described above. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0018] Figure 1 This is a schematic diagram of a DPM metal QR code decoding method based on the difference in light source direction sensitivity provided in an embodiment of this disclosure.

[0019] Figure 2 This is a schematic diagram of the dominant type determination process provided in the embodiments of this disclosure.

[0020] Figure 3 This is a schematic diagram of the process for generating confidence scores for deep-dominant candidate code point regions, as provided in an embodiment of this disclosure.

[0021] Figure 4 This is a schematic diagram of the process for generating confidence scores for candidate code point regions based on reflectance-dominant candidate code point regions, as provided in an embodiment of this disclosure.

[0022] Figure 5 A schematic diagram of a neural network model structure is provided for embodiments of this disclosure.

[0023] Figure 6 This is a schematic diagram of the structure of a DPM metal QR code decoding system based on the difference in light source direction sensitivity, provided in an embodiment of this disclosure.

[0024] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] This embodiment applies the method to the optical recognition of direct part identification (DPM) code dots on the surface of metal workpieces. The code systems involved include impact-type codes formed by impact pins and relief-cast codes formed by casting molds. The code dots being measured are distributed in a regular array on the metal surface, and the feature size of a single code dot is in the sub-millimeter to millimeter range. In actual production, due to factors such as impact pin wear, local blockage of the casting mold, and fluctuations in material hardness, different code dots exhibit point-to-point differences in geometric properties such as depth, edge angle, and bottom roughness. Uneven oxidation of the workpiece surface, residual mold release agent, and light oil stains further exacerbate the spatial inhomogeneity at the reflectivity level. These two types of factors together constitute the main sources of interference in this scenario.

[0027] This method is based on an area-array industrial camera with a set of independently controllable light source modules that maintain a fixed imaging geometry with the workpiece, and a computing unit with image processing and neural network inference capabilities. In one example, the computing unit integrates a neural network processor (NPU) to handle the inference computation for code point recognition. The light source modules are spatially arranged at different azimuth and elevation angles above or to the side of the code area being measured. The computing unit independently switches on and off each set of light sources according to a preset time sequence. Throughout the acquisition and processing, the workpiece and the camera remain relatively stationary to ensure comparable imaging geometry between multiple images.

[0028] The implementation process of the method described in this application will be described in detail below with reference to specific embodiments. It should be noted that this embodiment is only used to explain this application and is not intended to limit the scope of protection of this application. Conventional adjustments or substitutions of each step by those skilled in the art without departing from the concept of this application should be included in the scope of protection of this application.

[0029] Please see Figure 1 , Figure 1 This is a flowchart of a DPM metal QR code decoding method based on the difference in light source direction sensitivity according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes steps S1-S6: In step S1, N sets of light sources from different directions are turned on in sequence to acquire N light source images, and candidate code point regions are divided within their code areas, where N>1.

[0030] The computing unit sequentially activates N groups of light sources from different azimuths according to a preset time sequence. Each activation of a group triggers an exposure by the industrial camera, resulting in an image sequence. ,in This corresponds to a single grayscale image obtained under illumination from the nth group of light sources. As an example, N can be configured as four groups, with the four groups of light sources arranged at four different azimuth angles above the code area being measured. The azimuth intervals between them are approximately uniform, and the directions of the light sources participating in the photometric stereo processing are linearly independent. In actual implementation, N can also be three, five, or more groups. The specific number of groups can be reasonably selected by those skilled in the art based on the surface undulation characteristics of the workpiece and the code area coverage requirements.

[0031] To avoid the brightness difference between the light sources causing an overall bias in subsequent processing, each light source is balanced in brightness according to the calibration curve on the diffuse reflection reference board before leaving the factory or during the on-site deployment stage, so that the average gray level of each light source image on the reference board is at a similar level; for slight saturation caused by local highlight reflection, the camera exposure time can be slightly adjusted in conjunction with the light source switching.

[0032] Code region localization is performed on the obtained image sequence to obtain the rectangular range of the code region in the image pixel coordinate system. Within this rectangular range, candidate code point regions are divided according to the row and column pitch of the target code system, denoted as... Here, k is the candidate code point number, and K is the total number of candidate code points in the current code region. Code region localization and candidate code point region division can be achieved using conventional methods such as template matching, edge detection combined with geometric constraints.

[0033] In step S2, the gray values ​​of the pixels in the candidate code point area are taken in order of light source number to form a light source response sequence, and the directional sensitivity index is obtained by averaging the ratio of the gray standard deviation to the gray mean in the area.

[0034] For any pixel position within the code area The grayscale values ​​corresponding to the given location are taken from N light source images in order of light source number, and then concatenated to obtain the light source response sequence. ,in This represents the grayscale value of the pixel under illumination from the nth group of light sources. At the physical level, this sequence describes the grayscale response profile of a single pixel to illumination from different directions: for regions with significant surface undulations, the numerical differences between elements in the sequence are obvious due to the large differences in the angle between the surface normal and the directions of each light source; for regions with weak geometric undulations where grayscale changes mainly stem from differences in reflectivity, the differences between elements in the sequence are relatively gradual because reflectivity is less dependent on the direction of the light source under the Lambert approximation.

[0035] Based on the pixel-level light source response sequence, for each candidate code point region Calculate a direction sensitivity index Its definition is: In the formula The total number of pixels in the region. The mean of the light source response sequence is denoted as . The standard deviation of the light source response sequence is calculated using the following formulas: in To prevent small constants with zero denominators, their values ​​are taken on the order of one-thousandth of the grayscale dynamic range. They only serve to stabilize the values ​​and are not used as adjustable quantities for subsequent judgments. They are fixed during operation and do not participate in updates. The average value of the corresponding pixel-level grayscale variation coefficient within the candidate code point region is used to characterize the overall sensitivity of the candidate code point region to the illumination direction.

[0036] The value distribution corresponds to two states of the candidate code point region. For code points with significant geometric fluctuations, the surface normal changes continuously with position, and the brightness response of the same pixel varies greatly under illumination from different directions. The ratio of the standard deviation to the mean of each pixel within the region is generally high. The values ​​tend to be relatively large; for code points with weak depth that mainly rely on differences in reflectivity to form contrast with the background, the grayscale difference of the same pixel is small under different light sources, and the ratio of the standard deviation to the mean of each pixel in the region is generally low. The value is relatively small.

[0037] In step S3, histogram statistics are performed on the directional sensitivity index of all candidate code point regions in the current code region, and the diversion threshold is obtained from the position where the inter-class variance is maximized; those with directional sensitivity index not less than the diversion threshold are judged as depth-dominant, and those with diversion index less than the threshold are judged as reflectivity-dominant.

[0038] Please see Figure 2 , Figure 2 A schematic diagram of the dominant type determination process provided in an embodiment of this disclosure is shown. For example... Figure 2 As shown, in step S201, histogram statistics are performed on the directional sensitivity index of all candidate code point regions in the current code region.

[0039] right Histogram statistics are performed within its own value range. The number of bins used in the statistics is determined by taking the square root of the total number of candidate code points K in the current code region, i.e., taking... The number of bins is used to automatically adjust the statistical granularity as the total number of candidate code points changes: when the number of candidate code points is small, the number of bins decreases accordingly to avoid overly sparse samples within a single bin; when the number of candidate code points is large, the number of bins increases accordingly to preserve the details of the distribution pattern. As an example, when the total number of candidate code points is in the hundreds, the number of bins is in the tens, which is compatible with the common DPM code size.

[0040] In step S202, the splitting threshold is calculated based on maximizing the inter-class variance. The Otsu thresholding method is then applied to the resulting histogram to determine the splitting threshold. The code points are divided into two subclasses to maximize the inter-class variance between them. For any candidate segmentation position t, all candidate code point regions are divided into two classes, with t as the boundary: one class has a directional sensitivity index lower than t, and the other class has a directional sensitivity index higher than t. The proportions of the two classes are denoted as follows: and The mean values ​​within each class are respectively and Then the variance between the two classes is: Traverse t within the range of values ​​covered by the histogram, and take the value that makes t = t. The position where the maximum value is reached is used as the diversion threshold. The Otsu method transforms the target of the image from a conventional grayscale image to the numerical distribution of directional sensitivity indicators. It performs a holistic binary division of the code point signal type in the current code region, enabling the shunting threshold to adaptively adjust according to the surface condition changes of the workpiece batch. The range of values ​​is determined by the current code region. The numerical range is determined without any manual upper or lower limits; it is recalculated based on the statistical distribution of the current code region each time it is processed for a new code region, and it is not continued or reused across code regions.

[0041] Optionally, when the total number of candidate code points is small or the overall surface state of the current code region is relatively uniform, resulting in When the distribution exhibits a unimodal nature and the split threshold cannot be stably obtained by maximizing inter-class variance, all candidate code point regions in the current code area are uniformly treated as depth-dominant according to a preset backoff rule. Unimodal identification can be achieved using peak-valley detection of histograms or bimodal indices (e.g., the ratio of peak spacing to average peak width), and the threshold parameters are obtained through small-sample calibration in the offline stage. The backoff process prevents forced segmentation when the distribution itself does not have a bipartite structure, thereby avoiding situations where inaccurate initial split judgment affects subsequent branch selection, or low-quality thresholds misclassify most normal code points as reflectivity-dominant, ensuring that processing can continue in a conventional photometric stereo method even when the distribution degrades.

[0042] In step S203, if the direction sensitivity index is not less than the diversion threshold, it is judged as depth-dominant type; otherwise, it is judged as reflectivity-dominant type.

[0043] For each candidate code point region Its direction sensitivity index With shunt threshold Comparison: If ,determination If it is depth-driven, enter the geometric depth determination branch; if ,determination If it is a reflectance-dominant type, it enters the reflectance contrast determination branch, and the two branches are developed independently according to their respective determination criteria.

[0044] In step S4, for depth-dominant candidate code point regions, photometric stereo processing is performed based on the N light source images to obtain the depth difference and depth estimation signal-to-noise ratio, thereby generating the code point confidence score for that region.

[0045] Please see Figure 3 , Figure 3 A schematic diagram of the confidence generation process for a deep-dominant candidate codepoint region provided in an embodiment of this disclosure is shown. Figure 3 As shown, in step S301, the photometric stereo solution of the surface normal vector is obtained.

[0046] For each depth-dominant candidate code point region, based on the known direction vectors of each group of light sources... The grayscale response of this region and its surrounding non-code point neighboring pixels under N light source images. A system of linear equations is constructed based on the Lambert reflection approximation: In the formula This represents the reflectivity at that pixel location. Let N be the unit surface normal vector to be determined. When N is not less than three sets and the light source directions are linearly independent, the above system of equations can be solved simultaneously for each pixel position using least squares. and When N takes four sets, the linear equation system is an overdetermined system, and the solution adopts a pseudo-inverse form. The residual terms can be used together for subsequent auxiliary estimation of signal-to-noise ratio. The data is pre-obtained during the hardware deployment phase through reflective ball calibration or geometric measurement, and remains fixed after deployment, not participating in runtime updates. Solving for the normal vector in photometric stereo vision is a mature method in this field and will not be elaborated upon here. Optionally, pixels exhibiting obvious specular reflection or in self-shadow are first removed according to preset upper and lower grayscale thresholds before participating in the solution, to avoid extreme responses degrading the condition number of the linear equation system.

[0047] In step S302, the integration of the normal vector field and the acquisition of the relative depth distribution are performed. The selection method for the peripheral non-code point neighborhood is described below. Based on the surface normal vectors of the candidate code point region and its peripheral non-code point neighboring pixels, the surface height is determined. direction and The gradient components in the direction are obtained, and the gradient field corresponding to the candidate code point region and its surrounding non-code point neighborhood is integrated using the normal field integration method to obtain the relative depth distribution of the candidate code point region and its surrounding non-code point neighborhood.

[0048] In some embodiments, the normal field integral adopts the frequency domain-based Frankot-Chellappa integral method. Its core is to map the gradient field to the frequency domain through fast Fourier transform, and to solve for the height by projecting the non-integrable gradient field onto the integrable gradient field space. The algorithm has low complexity and numerical stability, which is convenient for deployment in embedded computing units. Thus, the relative depth distribution is output with the gradient field as input.

[0049] In step S303, the depth difference and depth estimation signal-to-noise ratio are calculated. Based on the relative depth distribution, the relative depth value at the center of the candidate code point region and the average relative depth in the surrounding non-code point neighborhood are extracted respectively. The absolute value of the difference between the two is taken as the depth difference for that region. This reflects the degree of protrusion or depression of the code point relative to the background. Simultaneously, it statistically analyzes the fluctuation of the relative depth estimate within a region relative to its own mean, for example, taking the standard deviation of the relative depth estimate as a fluctuation measure, and then comparing this fluctuation measure with... The reciprocal of the ratio is used as the depth estimation signal-to-noise ratio for that region. The greater the relative fluctuation of the depth difference, the lower the signal-to-noise ratio; conversely, the smaller the depth difference, the higher the signal-to-noise ratio. To prevent small constants with a denominator of zero.

[0050] Optionally, when the residuals of the linear equation system are given simultaneously with the pseudo-inverse solution, the residual term can be used as a supplementary input for the fluctuation metric. Perform a more detailed estimate.

[0051] In step S304, the mapping of depth-dominant confidence is generated. The depth difference is... Signal-to-noise ratio with depth estimation Simultaneously, the input is a preset monotonically non-decreasing mapping function, which is also used as the independent variable. The output is used as the confidence level of the code point corresponding to the candidate code point region. The value range is limited to 0 to 1. One implementation involves first normalizing the depth difference to its statistical scale within the current code region, then multiplying it with the depth estimation signal-to-noise ratio (SNR) using a predetermined weight, and finally mapping it to the 0-1 interval using a saturation transformation in the form of a logistic function. The predetermined weight refers to the relative contribution coefficients of the normalized depth difference and the depth estimation SNR in the overall index. These weights are obtained through calibration using an offline sample set with codeword annotations. The calibration criterion is to minimize the codeword recognition error rate on the sample set.

[0052] Once calibrated, the weights are fixed and used during runtime, and are not updated online. When there are significant changes in the workpiece material or surface treatment process, the weights can be recalibrated offline and refreshed in batches. Output The closer the value is to 1, the higher the probability that a code point exists in the candidate code point region; conversely, the closer the value is to 0, the lower the probability that a code point exists. It does not carry a marker indicating whether its source is depth-based or reflectance-based.

[0053] In step S5, for the reflectance-dominant candidate code point region, the average gray value under the N light source images is used as the reflectance approximation. The background reflectance approximation is obtained in the same way for the surrounding non-code point neighborhood. The ratio of the difference between the two and the background reflectance approximation is used as the reflectance contrast. The code point confidence of the region is generated based on its absolute value.

[0054] Please see Figure 4 , Figure 4A schematic diagram of the reflectivity-dominated candidate code point region code point confidence generation process provided in an embodiment of this disclosure is shown. Figure 4 As shown, in step S401, the approximate reflectance of the candidate code point region is extracted.

[0055] Candidate code point region First, the grayscale values ​​of each pixel in the candidate code point region are averaged across N images of the light source in the light source dimension. Then, the average values ​​are averaged across the region in the spatial dimension to obtain an approximate reflectance of the region. Under the premise that the brightness of each group of light sources is balanced as described in S1 and illuminates the same code area, the average illuminance at different positions within the same code area is approximately equal. Therefore, This can be used to characterize the relative reflectivity level of the candidate code point region. This process assumes that the reflection mainly satisfies Lambertian reflection, and pixels with obvious local specular highlights are first removed before being included in the mean statistics.

[0056] In step S402, the selection of the peripheral non-code point neighborhood is approximated by the background reflectivity. In the candidate code point region... The non-code point neighborhood is selected from the immediate periphery of the candidate code point region according to a preset shape. In some embodiments, the neighborhood is annular, with the inner diameter of the ring being approximately equal to the radius of the circumcircle of the candidate code point region, and the outer diameter of the ring being determined according to a set ratio of the feature size of the candidate code point region, for example, a multiple of the feature size. In other embodiments, the neighborhood is a rectangular ring, with the inner frame of the rectangle being approximately equal to the radius of the circumcircle of the candidate code point region, and the outer frame being enlarged according to a set ratio. The ratio parameter is calibrated offline based on the code system geometry and the spacing between adjacent code points. During neighborhood selection, overlapping portions with adjacent candidate code point regions are automatically excluded, ensuring that the neighborhood contains only background pixels. For the selected non-code point neighborhood, the approximate background reflectance is obtained in the same manner as described above. .

[0057] In step S403, the reflectance contrast is calculated. The reflectance contrast is obtained by dividing the difference between the approximate reflectance of the candidate code point region and its approximate background reflectance by the sum of the approximate background reflectance and a small constant. : in As mentioned above, it only serves to stabilize the numerical values. The larger the absolute value, the more significant the difference in reflectivity between the candidate code point region and its background, and the higher the probability of the existence of a code point. Its sign reflects whether the candidate code point region is brighter or darker than the background, corresponding to different physical causes such as fresh metal fracture, slight oxidation, and residual mold release agent. For the code point existence determination itself, only the absolute value is taken. The absolute value participates in subsequent mappings, regardless of its sign; while the sign information can be recorded as an optional feature in the auxiliary channel of the confidence map, which can be used as an optional input for the subsequent neural network decoding model.

[0058] In step S404, a reflectance-dominant confidence score is generated. The absolute value of the reflectance contrast is then used. Input the preset monotonically non-decreasing mapping function The output is used as the confidence level of the code point corresponding to the candidate code point region. The value range is limited to 0 to 1. It can be implemented using saturation functions, such as logic functions or piecewise linear saturation functions; its parameters are similar to... The parameters are jointly calibrated on the same offline sample set with codeword annotations. The calibration criterion is also to minimize the codeword recognition error rate. After calibration, the parameters are used permanently at runtime and do not participate in online updates. These are generated separately above. Both are consistent in numerical semantics and are used to characterize the probability of a code point existing in a candidate code point region. A value close to 1 corresponds to a high probability of the existence of a code point, and a value close to 0 corresponds to a high probability of the absence of a code point. The two can be arranged according to their spatial positions in the subsequent process without the need for additional normalization or alignment operations.

[0059] In step S6, the code point confidence scores of all candidate code point regions are arranged according to their spatial positions to form a code point confidence map, which is then fed into the neural network decoding model to obtain the codeword recognition result.

[0060] Based on the row and column indices of the candidate code point regions within the code area, A single-channel two-dimensional matrix arranged in R rows and C columns is denoted as a code point confidence map. , where R and C correspond to the number of rows and columns of the code area, respectively, and are determined a priori by the code system. Each position corresponds to the confidence level of a candidate code point region. The confidence levels of depth-dominant and reflectance-dominant candidate code point regions are written in the same row and column, without introducing additional channels or flags for differentiation. When using the aforementioned reflectance contrast symbol as an auxiliary feature, it is possible to... An additional channel of the same size is superimposed on top to carry symbolic information, forming a dual-channel input.

[0061] Please see Figure 5 , Figure 5 A schematic diagram of the neural network model structure provided in an embodiment of this disclosure is shown. For example... Figure 5As shown, the neural network decoding model adopts a structure of sequentially cascading convolutional feature extraction module and fully connected classification module, with the output of the previous module serving as the input of the next module. The convolutional feature extraction module consists of several convolutional layers and pooling layers cascaded alternately, configured as a two-level structure in one example. The first level consists of a two-dimensional convolutional layer, a non-linear activation function, and a max pooling layer connected in series: the kernel size of the two-dimensional convolutional layer can be configured to 3×3, the number of channels can be configured to 16, the stride can be configured to 1, and zero padding is used to maintain spatial resolution; the non-linear activation function uses a modified linear unit (ReLU) and acts on each element of the convolution output; the window size of the max pooling layer can be configured to 2×2, the stride can be configured to 2, and the spatial resolution is downsampled by a factor of two.

[0062] The second stage consists of a 2D convolutional layer, a ReLU activation function, and a max-pooling layer connected in series. The number of channels in the 2D convolutional layer is doubled compared to the first stage, for example, configured as 32. The kernel size and stride remain the same as the first stage. The convolutional feature extraction module uses code point confidence maps. The input is a multi-channel feature tensor, and the output is a multi-channel feature tensor.

[0063] The fully connected classification module consists of a flattening operation, a first fully connected layer, a ReLU activation function, and a second fully connected layer, sequentially connected in series. The flattening operation expands the multi-channel feature tensor output by the convolutional feature extraction module into a one-dimensional vector along rows, columns, and channels. The number of neurons in the first fully connected layer can be configured to 128, mapping the flattened vector to a fixed-dimensional latent feature representation, followed by a non-linear transformation using the ReLU activation function. The second fully connected layer serves as the output layer, with the number of neurons corresponding to the size of the codeword dictionary defined by the current code system. Its output is transformed by a normalized exponential function (Softmax) to obtain the probability distribution of each codeword category, and the category with the highest probability is taken as the codeword recognition result for the current code region.

[0064] In some embodiments, the neural network decoding model is obtained through offline training. The input to the training sample pairs is the code point confidence map generated on the actual workpiece according to the aforementioned process, and the corresponding annotation is the actual code character on the surface of the sample workpiece. During the sample acquisition stage, workpieces from different batches, with different degrees of pin wear, and different oxidation states are covered to ensure that the sample set fully represents both types of candidate code point regions. The training loss adopts the cross-entropy loss function, and the optimization method can adopt stochastic gradient descent or adaptive moment estimation (Adam). Hyperparameters such as learning rate and batch size are conventional choices in this field. To improve the robustness of the model to position jitter and confidence noise, data augmentation processing such as small-range translation and mild noise injection can be performed on the code point confidence map during training. These data augmentation methods are common training techniques. After training, the model parameters are fixed and deployed to the NPU side in the computing unit. During the inference stage, only forward computation is performed, and the parameters are not updated.

[0065] During runtime, after the above process is completed for each new code region, a code point confidence map is generated, and the calculation unit will... As a single forward input to the NPU-side neural network decoding model, the model outputs the codeword recognition result after performing one forward inference, which serves as the final output of the entire processing flow. In the NPU-side implementation with multi-core parallel capabilities, different channels of the convolutional feature extraction module can be allocated to different cores for parallel execution to improve the throughput of a single inference.

[0066] In some embodiments, when the overall code point confidence in the code point confidence map is low, for example... If the proportion of candidate code point regions with values ​​near the preset lower limit exceeds a preset ratio, the calculation unit determines that the current light source combination parameters are not well adapted to the current workpiece surface state, triggering a switch of the light source combination parameter library: calling the next set of parameters from the preset light source combination parameter library, reconfiguring the on / off timing or intensity settings of N sets of light sources according to the set of parameters, re-acquiring N light source images, and re-executing the aforementioned process until the code character recognition result is obtained; if the confidence requirement is still not met after the light source combination parameter library has been traversed, an anomaly report is triggered.

[0067] The parameters in the light source combination parameter library are calibrated offline, and the number of groups can be configured to a maximum of four to achieve a balance between fault tolerance coverage and on-site cycle time. There are differences between the parameters in terms of light source orientation combination and intensity ratio, covering different typical states caused by workpiece materials, surface treatment and external lighting interference.

[0068] Therefore, in DPM metal QR code decoding under typical challenging scenarios such as shallow code points due to impact pin wear and shallow code points in areas of uneven oxidation on casting surfaces, this invention adaptively classifies candidate code point regions into depth-dominant or reflectivity-dominant types based on their directional sensitivity differences, and establishes a unified code point confidence metric using geometric depth difference and reflectivity contrast criteria, respectively. This effectively suppresses the contamination of depth estimation by reflectivity signals on shallow code points, and alleviates code point boundary position offset and its resulting decoding failures. The shunting threshold is adaptively determined based on the current code area index distribution, making the overall process highly adaptable to changes in the surface condition of different batches of workpieces, and can be further expanded in the working range by combining it with the switching mechanism of the light source combination parameter library.

[0069] Please see Figure 6 , Figure 6 This is a schematic diagram of a DPM metal QR code decoding system based on the difference in light source direction sensitivity provided in an embodiment of this application. Figure 6 As shown, the system includes: The image acquisition and region segmentation module 601 is used to sequentially activate N groups of light sources from different azimuths to acquire N light source images, and to segment candidate code point regions within their code areas, where N>1; The orientation sensitivity index calculation module 602 is used to take the gray values ​​of the pixels in the candidate code point area in the order of the light source number to form a light source response sequence, and take the average of the ratio of the gray standard deviation to the gray mean in the area to obtain the orientation sensitivity index. The dominant type determination module 603 is used to perform histogram statistics on the directional sensitivity index of all candidate code point regions in the current code region, and obtain the diversion threshold from the position where the inter-class variance is maximized; those with directional sensitivity index not less than the diversion threshold are judged as depth dominant, and those with reflectivity dominant are judged as reflectivity dominant. The depth-dominant confidence generation module 604 is used to perform photometric stereo processing on the depth-dominant candidate code point region based on the N light source images to obtain the depth difference and depth estimation signal-to-noise ratio, and generate the code point confidence of the region accordingly. The reflectance-dominant confidence generation module 605 is used to take the average gray value of the candidate code point region under the N light source images as the reflectance approximation, and to obtain the background reflectance approximation of the surrounding non-code point neighborhood in the same way. The ratio of the difference between the two and the background reflectance approximation is the reflectance contrast, and the absolute value of the contrast is used to generate the code point confidence of the region. The decoding module 606 is used to arrange the code point confidence scores of all candidate code point regions according to their spatial positions to form a code point confidence map, which is then fed into the neural network decoding model to obtain the codeword recognition result.

[0070] Each processing unit and / or module in the embodiments of this application can be implemented by an analog circuit that implements the functions described in the embodiments of this application, or by software that executes the functions described in the embodiments of this application.

[0071] Please see Figure 7 It shows a schematic diagram of the structure of an electronic device according to an embodiment of this application, which can be used to implement... Figure 1 The method in the illustrated embodiment. (As shown) Figure 7 As shown, the electronic device may include: The system includes at least one processor 701, at least one network interface 704, a user interface 703, a memory 705, and at least one communication bus 702. The communication bus 702 is used to enable connection and communication between the components. The user interface 703 may include buttons, and optionally include a standard wired or wireless interface. The network interface 704 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0072] The processor 701 may include one or more processing cores and connect to various parts within the electronic device via various interfaces and lines. It implements various functions and data processing of the electronic device by running or executing instructions, programs, code sets, or instruction sets stored in the memory 705, and by accessing data in the memory 705. Optionally, the processor 701 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 701 may also integrate one or more combinations of CPU, GPU, and modem.

[0073] The memory 705 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 705 includes a non-transitory computer-readable medium for storing instructions, programs, code, code sets, or instruction sets. The memory 705 may be divided into a program storage area and a data storage area, wherein the program storage area can be used to store instructions for implementing an operating system and instructions for implementing the foregoing method embodiments; the data storage area can be used to store data related to the relevant method embodiments. The memory 705 may also be at least one storage device located remotely from the processor 701. Figure 7 As shown, the memory 705, which serves as a computer storage medium, may contain an operating system, a network communication module, a user interface module, and program instructions.

[0074] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by processor 701, the functions defined in the methods of this application are performed.

[0075] Another embodiment of this application provides a storage medium storing computer program instructions thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application.

[0076] In the above embodiments, the descriptions of each embodiment have different focuses. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. The above descriptions are merely preferred embodiments of this application and explanations of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by specific combinations of the above technical features, but should also cover other technical solutions formed by arbitrary combinations of the above technical features or their equivalent features without departing from the inventive concept.

Claims

1. A DPM metal QR code decoding method based on differences in light source direction sensitivity, characterized in that, include: N sets of light sources from different directions are turned on in sequence to acquire N light source images, and candidate code point regions are divided within their code areas, where N>1; The gray values ​​of pixels within the candidate code point region are taken in order of light source number to form a light source response sequence. The directional sensitivity index is obtained by averaging the ratio of the gray standard deviation to the gray mean within the region. Histogram statistics are performed on the directional sensitivity index of all candidate code point regions in the current code area, and the diversion threshold is obtained from the position where the inter-class variance is maximized. Those with a directional sensitivity index not less than the diversion threshold are judged as depth-dominant, and those with a non-directional sensitivity index less than the threshold are judged as reflectivity-dominant. For depth-dominant candidate code point regions, photometric stereo processing is performed based on the N light source images to obtain the depth difference and depth estimation signal-to-noise ratio, thereby generating the region code point confidence. For reflectance-dominant candidate code point regions, the average gray value under the N light source images is used as the reflectance approximation. The background reflectance approximation is obtained in the same way for the surrounding non-code point neighborhood. The ratio of the difference between the two and the background reflectance approximation is used as the reflectance contrast. The code point confidence of the region is generated based on its absolute value. The code point confidence scores of all candidate code point regions are arranged according to their spatial location to form a code point confidence map, which is then fed into a neural network decoding model to obtain the codeword recognition result.

2. The DPM metal QR code decoding method based on the difference in light source direction sensitivity according to claim 1, characterized in that, The histogram statistics of the directional sensitivity index of all candidate code point regions in the current code region include: determining the number of bins of the histogram by taking the square root of the total number of candidate code point regions in the current code region; and when the histogram is unimodal and the diversion threshold cannot be obtained based on the position where the inter-class variance is maximized, all candidate code point regions in the current code region are uniformly processed according to the depth-dominant type.

3. The DPM metal QR code decoding method based on the difference in light source direction sensitivity according to claim 1, characterized in that, Photometric stereo processing is performed on the N light source images to obtain the depth difference and depth estimation signal-to-noise ratio, including: solving the surface normal vector of each pixel in the region based on the direction vector of each light source and the grayscale response of each pixel in the depth-dominant candidate code point region and its surrounding non-code point neighborhood in the N light source images; integrating the normal vector field formed by the surface normal vectors to obtain the relative depth distribution of the region relative to its surrounding non-code point neighborhood; taking the absolute value of the difference between the relative depth at the center of the region and the average relative depth of the surrounding non-code point neighborhood as the depth difference; and taking the reciprocal of the ratio of the fluctuation amplitude of the relative depth estimate in the region to the depth difference and a small constant as the depth estimation signal-to-noise ratio.

4. The DPM metal QR code decoding method based on the difference in light source direction sensitivity according to claim 1, characterized in that, The peripheral non-code point neighborhood is configured as a rectangular neighborhood surrounding the outer edge of the reflectivity-dominant candidate code point region. Its neighborhood width is selected according to a set ratio of the feature size of the candidate code point region and does not overlap with adjacent candidate code point regions.

5. The DPM metal QR code decoding method based on the difference in light source direction sensitivity according to claim 1, characterized in that, For depth-dominant candidate code point regions, generating code point confidence scores for those regions includes: inputting the depth difference and the depth estimation signal-to-noise ratio into a preset monotonically non-decreasing mapping function, and using the output of the monotonically non-decreasing mapping function as the code point confidence scores corresponding to the depth-dominant candidate code point regions.

6. The DPM metal QR code decoding method based on the difference in light source direction sensitivity according to claim 1, characterized in that, For reflectivity-dominant candidate code point regions, generating code point confidence scores for those regions includes: inputting the absolute value of the reflectivity contrast into a preset monotonically non-decreasing mapping function, and using the output of the monotonically non-decreasing mapping function as the code point confidence score corresponding to the reflectivity-dominant candidate code point region.

7. The DPM metal QR code decoding method based on the difference in light source direction sensitivity according to claim 1, characterized in that, The neural network decoding model includes a convolutional feature extraction module and a fully connected classification module cascaded in sequence. The convolutional feature extraction module includes at least one convolutional layer and a pooling layer that works in conjunction with it. The input of the convolutional feature extraction module is the code point confidence map. The fully connected classification module includes at least one fully connected layer. The output layer size of the fully connected classification module corresponds to the codeword dictionary size. The output of the fully connected classification module is the codeword recognition result.

8. The DPM metal QR code decoding method based on the difference in light source direction sensitivity according to claim 7, characterized in that, The neural network decoding model is obtained through offline training of training sample pairs with codeword annotations. The input of each training sample pair is the generated code point confidence map, and the corresponding annotation is the codeword.

9. The DPM metal QR code decoding method based on the difference in light source direction sensitivity according to claim 1, characterized in that, If the proportion of candidate code point regions with values ​​near the preset lower limit in the code point confidence map exceeds a preset ratio, the system switches to the next set of parameters in the preset light source combination parameter library, re-acquires N light source images, and again performs the following processes: dividing candidate code point regions, obtaining direction sensitivity indicators, obtaining shunting thresholds, generating code point confidence scores for depth-dominant and reflectivity-dominant candidate code point regions, constructing the code point confidence map, and feeding the reconstructed code point confidence map into the neural network decoding model to update the codeword recognition results.

10. A DPM metal QR code decoding system based on differences in light source direction sensitivity, characterized in that, include: The image acquisition and region segmentation module is used to sequentially activate N groups of light sources from different azimuths to acquire N light source images, and to segment candidate code point regions within their code areas, where N>1; The orientation sensitivity index calculation module is used to take the gray values ​​of the pixels in the candidate code point area in the order of the light source number to form a light source response sequence, and take the average of the ratio of the gray standard deviation to the gray mean in the area to obtain the orientation sensitivity index. The dominant type determination module is used to perform histogram statistics on the directional sensitivity index of all candidate code point regions in the current code area, and obtain the diversion threshold from the position where the inter-class variance is maximized. Those with a directional sensitivity index not less than the diversion threshold are judged as depth-dominant, and those with a non-directional sensitivity index less than the threshold are judged as reflectivity-dominant. The depth-dominant confidence generation module is used to perform photometric stereo processing on the N light source images to obtain the depth difference and depth estimation signal-to-noise ratio for the depth-dominant candidate code point region, and generate the region code point confidence accordingly. The reflectance-dominant confidence generation module is used to approximate the reflectance of a reflectance-dominant candidate code point region by using its grayscale mean value under the N light source images as the reflectance approximation, and to obtain the background reflectance approximation of the surrounding non-code point neighborhood in the same way. The ratio of the difference between the two and the background reflectance approximation is used as the reflectance contrast, and the absolute value of the contrast is used to generate the code point confidence of the region. The decoding module is used to arrange the code point confidence scores of all candidate code point regions according to their spatial positions to form a code point confidence map, which is then fed into the neural network decoding model to obtain the codeword recognition result.

Images