Head-mounted device-based bar code recognition method and device, and head-mounted device

Abstract

The application relates to the technical field of augmented reality wearable equipment, and discloses a bar code recognition method and device based on a head-mounted device and the head-mounted device. The method comprises the following steps: acquiring a target image through a camera of the head-mounted device; acquiring motion data of the head-mounted device, wherein the motion data is motion data within an exposure time of the camera; identifying a region of interest in the target image, wherein the region of interest is a region of interest corresponding to a target bar code to be identified; performing image enhancement on the region of interest based on the motion data; and performing bar code recognition on the region of interest after the image enhancement. The application can perform image enhancement on the region of interest by using the motion data of the head-mounted device, compensate for imaging quality, improve bar code recognition rate, solve the "smearing" problem caused by user head movement, realize high-performance bar code recognition without increasing hardware cost, and improve image processing efficiency.

Description

Technical Field

[0001] This application relates to the field of augmented reality wearable device molding technology, and more specifically, to a barcode recognition method, device, and head-mounted device based on a head-mounted device. Background Technology

[0002] In related technologies, when using head-mounted devices (such as AR glasses) for barcode recognition, user head movements can cause the head-mounted device to shake, resulting in inaccurate camera focus and affecting the accuracy of barcode recognition. For example, slight head tremors can cause noticeable pixelation in images captured by the camera at high resolution (such as 5MP).

[0003] There is currently no effective solution to the aforementioned technical problems. Summary of the Invention

[0004] The main objective of this application is to provide a barcode recognition method, device, and head-mounted device based on a head-mounted device, so as to solve the technical problem of low barcode recognition accuracy caused by user head movement when the head-mounted device performs barcode recognition in related technologies.

[0005] To address the aforementioned technical problems, a first aspect of this application provides a barcode recognition method based on a head-mounted device, comprising:

[0006] Acquire motion data of the head-mounted device, wherein the motion data is motion data within the exposure time of the camera;

[0007] Identify the region of interest in the target image, wherein the region of interest is the region of interest corresponding to the target barcode to be identified;

[0008] Image enhancement is performed on the region of interest based on the motion data;

[0009] Barcode recognition is performed on the region of interest after image enhancement.

[0010] Optionally, the image enhancement of the region of interest based on the motion data includes:

[0011] Based on the motion data, determine the pixel displacement vector caused by the head movement of the wearer of the head-mounted device;

[0012] Construct a point spread function based on the pixel displacement vector;

[0013] Wiener filtering is performed on the region of interest based on the point spread function to obtain the filtered region of interest.

[0014] Optionally, after enhancing the image of the region of interest based on the motion data, the method further includes enhancing the image edges of the region of interest, wherein enhancing the image edges of the region of interest includes:

[0015] Obtain the image percentage of the region of interest in the target image;

[0016] If the image proportion is less than a first preset image proportion threshold, the target image is resampled to obtain a resampled image;

[0017] The resampled image is dynamically binarized to increase the edge contrast of the region of interest in the resampled image;

[0018] The step of performing barcode recognition on the enhanced region of interest includes:

[0019] Barcode recognition is performed on the region of interest after edge contrast enhancement.

[0020] Optionally, after acquiring the target image through the camera of the head-mounted device, the method further includes:

[0021] The target image is downsampled to obtain a downsampled image;

[0022] The identification of the region of interest in the target image includes:

[0023] Identify the region of interest from the downsampled image;

[0024] The step of obtaining the image proportion of the region of interest in the target image includes:

[0025] Obtain the percentage of the region of interest in the downsampled image.

[0026] Optionally, the method further includes:

[0027] The blur type of the region of interest is determined based on the pixel displacement vector;

[0028] If the blur type of the region of interest is optical blur, then determine whether the blur degree of the region of interest exceeds a preset blur degree threshold;

[0029] If the value exceeds the limit, the step of performing barcode recognition on the enhanced region of interest includes:

[0030] An interface identifier and corresponding head-mounted device movement indication information are generated in the user interface of the head-mounted device to perform barcode recognition on the enhanced region of interest based on the interface identifier and the head-mounted device movement indication information. The interface identifier represents the target recognition window of the camera when performing the barcode recognition, and the head-mounted device movement indication information is used to instruct the wearer of the head-mounted device to move so that the region of interest is within the range of the interface identifier.

[0031] Optionally, determining the blur type of the region of interest based on the pixel displacement vector includes:

[0032] If the coordinate value of the pixel displacement vector is 0 or less than a first preset vector threshold, the blur type of the region of interest is determined to be optical blur.

[0033] If the coordinate value of the pixel displacement vector is greater than the second preset vector threshold, the blur type of the region of interest is determined to be motion blur.

[0034] Optionally, the step of generating an interface identifier and corresponding head-mounted device movement indication information on the user interface of the head-mounted device includes:

[0035] The imaging state of the region of interest is evaluated based on its visual features, and an evaluation result is obtained.

[0036] Based on the evaluation results, the marking information of the interface identifier and the movement indication information of the head-mounted device are generated, wherein the movement indication information of the head-mounted device is generated based on the relative positional relationship between the interface identifier and the region of interest.

[0037] Optionally, the evaluation of the imaging state of the region of interest based on its visual features to obtain an evaluation result includes:

[0038] If the edge of the region of interest spreads, and / or if the peak-to-peak contrast of the region of interest is lower than a preset contrast threshold, then it is determined that the region of interest is outside the target depth of focus range of the camera, and the distance between the region of interest and the camera is less than a preset first distance threshold, wherein the peak-to-peak contrast is the maximum grayscale difference between the black bars and white bars of the region of interest.

[0039] If the gradient energy of the region of interest is within a preset peak range, then the region of interest is determined to be within the target depth of field range of the camera.

[0040] If the region of interest has a smaller percentage of the target image than a second preset image percentage threshold, and / or if the region of interest has missing high-frequency image features, then the region of interest is determined to be outside the target depth of field of the camera, and the distance between the region of interest and the camera is greater than a preset second distance threshold.

[0041] A second aspect of this application provides a barcode recognition device based on a head-mounted device, comprising:

[0042] The first acquisition module is configured to acquire target images through the camera of a head-mounted device;

[0043] The second acquisition module is configured to acquire motion data of the head-mounted device, wherein the motion data is motion data within the exposure time of the camera;

[0044] The first recognition module is configured to recognize the region of interest in the target image, wherein the region of interest is the region of interest corresponding to the target barcode to be recognized;

[0045] The image enhancement module is configured to enhance the image of the region of interest based on the motion data;

[0046] The second recognition module is configured to perform barcode recognition on the region of interest after image enhancement.

[0047] A third aspect of this application provides a head-mounted device, including a camera, a motion data acquisition unit, a memory, and a processor. The camera is used to acquire a target image, the motion data acquisition unit is used to acquire motion data of the head-mounted device, the memory stores a computer program, and the processor implements the above-described method when executing the computer program in the memory.

[0048] The head-mounted device further includes a display unit, which is used to display interface identifiers and corresponding head-mounted device movement indication information. The processor implements the above method when executing the computer program in the memory.

[0049] By applying the technical solution of this application, a target image containing a barcode to be recognized is acquired using the camera of a head-mounted device; motion data of the AR glasses head-mounted device is acquired, wherein the motion data is motion data within the exposure time of the camera; a region of interest (ROI) is identified in the target image, wherein the ROI corresponds to the target barcode to be recognized; image enhancement is performed on the ROI based on the motion data; and barcode recognition is performed on the enhanced ROI. This method utilizes the motion data of the head-mounted device to enhance the image of the ROI, improving the image quality of the ROI and increasing the barcode recognition rate. It solves the "ghosting" problem caused by user head movement during barcode recognition by the head-mounted device, achieving high-performance barcode recognition without increasing hardware costs. Furthermore, in this embodiment, image enhancement is performed only on the ROI, without enhancing the entire target image, which improves image processing efficiency. Attached Figure Description

[0050] Figure 1 This is a flowchart of a barcode recognition method based on a head-mounted device according to an embodiment of this application;

[0051] Figure 2 This is a schematic diagram of the evaluation state of the region of interest and the corresponding AR guidance strategy in the barcode recognition method based on a head-mounted device provided in the embodiments of this application;

[0052] Figure 3 This is a schematic diagram showing the user interface of the barcode recognition method based on a head-mounted device provided in the embodiments of this application;

[0053] Figure 4 This is a schematic diagram of the structure of a barcode recognition device based on a head-mounted device according to an embodiment of this application;

[0054] Figure 5 This is a hardware structure block diagram of a head-mounted device used to implement the barcode recognition method based on a head-mounted device provided in the embodiments of this application. Detailed Implementation

[0055] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0056] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0057] According to an embodiment of this application, a method embodiment for barcode recognition based on a head-mounted device is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0058] Figure 1 This is a schematic flowchart of a barcode recognition method based on a head-mounted device provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0059] S101 acquires target images through the camera of a head-mounted device.

[0060] In this step, the camera on the head-mounted device captures real-time environmental frame images around the wearer (or user of the device). When the camera captures an image containing at least part of the barcode to be recognized, the target image is determined. That is, the target image is an image containing at least part of the barcode to be recognized. When the camera does not capture an image containing at least part of the barcode to be recognized, the wearer can move or adjust the camera's capture range until the target image is captured.

[0061] The head-mounted device can be AR glasses or an AR helmet, etc. The camera can be a fixed-focus camera (with a fixed focal length) or a non-fixed-focus camera (with a variable focal length that can autofocus). The target image can be a raw high-resolution image frame, with a pixel count of, for example, 5MP.

[0062] S102, acquire motion data of the head-mounted device, wherein the motion data is motion data within the exposure time of the camera.

[0063] Specifically, the motion data from the head-mounted device refers to the head movement data of the wearer. The head-mounted device is equipped with a motion data acquisition unit that can collect motion data from the device in real time.

[0064] The motion data acquisition unit can be an Inertial Measurement Unit (IMU), which includes a gyroscope and an accelerometer. The gyroscope measures the angular velocity of an object, and the accelerometer measures its acceleration. The IMU obtains the motion data of the head-mounted device by fusing the measurement data from the gyroscope and accelerometer. Alternatively, the motion data acquisition unit of the head-mounted device can be a speed sensor, magnetometer, etc. A magnetometer can measure the orientation and rotation of an object by detecting changes in magnetic field strength.

[0065] The motion data collected in this step is mainly the motion data within the camera's exposure time. That is, when the head-mounted device is working, it can simultaneously trigger the camera and the motion data acquisition unit. While acquiring the target image through the camera, the head motion data of the wearer of the head-mounted device is acquired through the inertial measurement unit set in the head-mounted device, resulting in time series data containing angular velocity and linear acceleration. The image frames of the target image are then time-aligned with the motion data corresponding to the timestamps to form a synchronized data pair, thereby using the motion data to process the target image.

[0066] In this embodiment, a motion data acquisition unit is used for high-frequency sampling, and the sampling frequency can be greater than 200Hz.

[0067] S103, Identify the region of interest in the target image, wherein the region of interest is the region of interest corresponding to the target barcode to be identified.

[0068] After acquiring the target image in step S101, the processor of the head-mounted device (e.g., a SoC chip) can analyze the target image, identify the region of interest (ROI), and thus identify the target barcode in the target image.

[0069] In this step, the processor can use a target detection algorithm based on morphology and edge features to identify the barcode to be identified, identify the bounding box of the barcode in the target image, and calculate the edge coordinates of the bounding box (e.g., the coordinates of the top left and bottom right pixels), thereby determining the range of pixel coordinates of the region of interest in the target image.

[0070] S104, perform image enhancement on the region of interest based on the motion data.

[0071] After identifying the region of interest (ROI) in the target image, the processor performs image enhancement on the ROI based on motion data from the camera's exposure time. This effectively removes motion blur from the ROI, restores clear details, and improves image quality (contrast and sharpness). Motion blur refers to image blurring caused by head movements (e.g., shaking) of the wearer. In this step, image enhancement is performed only on the ROI using motion data, rather than processing the entire target image. This improves processing efficiency and avoids the high power consumption of the head-mounted device caused by processing the entire image, thus enhancing its reliability. For example, when the head-mounted device is AR glasses, this ensures real-time image processing by the SoC chip at the glasses' distributed architecture and avoids overheating caused by processing the entire image, improving the performance of the AR glasses.

[0072] S105, perform barcode recognition on the region of interest after image enhancement.

[0073] Based on the motion data, the region of interest is enhanced to obtain a high-quality region of interest. The processor can then directly perform barcode recognition on the region of interest and accurately identify the barcode content.

[0074] The barcode recognition method based on a head-mounted device provided in this application embodiment acquires a target image containing a barcode to be recognized using the camera of the head-mounted device; acquires motion data of the AR glasses head-mounted device, wherein the motion data is motion data within the exposure time of the camera; identifies a region of interest (ROI) in the target image, wherein the ROI is the ROI corresponding to the target barcode to be recognized; performs image enhancement on the ROI based on the motion data; and performs barcode recognition on the enhanced ROI. This method utilizes the motion data of the head-mounted device to enhance the image of the ROI, improving the image quality of the ROI and increasing the barcode recognition rate. It solves the "ghosting" problem caused by the user's (head-mounted device wearer's) head movement during barcode recognition, achieving high-performance barcode recognition without increasing hardware costs. Furthermore, in this embodiment, image enhancement is performed only on the ROI, without enhancing the entire target image, which improves image processing efficiency.

[0075] In practical implementation, the camera for the head-mounted device can be determined according to actual needs. For example, to meet the lightweight design requirements of the head-mounted device while considering its size, power consumption, and structural stability, a fixed-focus camera module can be used. This effectively balances the high pixel requirements of the head-mounted device with its lightweight and low power consumption limitations, improving the device's performance and user experience. Conversely, to effectively increase the camera's recognition range, a non-fixed-focus camera can be used. This allows for automatic focusing and maintains a high recognition rate across the entire range, further improving barcode recognition accuracy. It achieves a high barcode recognition rate whether the target barcode is close or far away.

[0076] As an optional embodiment, step S104 can be implemented through the following steps: the image enhancement of the region of interest based on the motion data includes:

[0077] S1041, Based on the motion data, determine the pixel displacement vector caused by the head movement of the wearer of the head-mounted device;

[0078] S1042, Construct a point spread function based on the pixel displacement vector;

[0079] S1043, Wiener filtering is performed on the region of interest based on the point spread function to obtain the filtered region of interest.

[0080] In this step, the angular velocity acquired by the IMU can be integrated to obtain the pixel displacement vector v(x, y) caused by head movement. The magnitude of the pixel displacement vector represents the movement length (unit: pixels), and the direction represents the movement angle. Then, a point spread function (PSF) is constructed based on the pixel displacement vector v(x, y) to obtain the directional blur model corresponding to the region of interest. This directional blur model is used to analyze the degree of image blur in the region of interest. In an ideal imaging system, a point light source will be imaged as a sharp point on the image. However, in a real imaging system, due to various interference factors, the image of a point light source will spread into a blurred spot. The intensity distribution of this spot is the point spread function. The point spread function can comprehensively reflect the blur characteristics of the imaging system. Different blur causes will produce different point spread functions. For example, the PSF of blur caused by camera or wearer head movement is usually linear. In this embodiment, the point spread function for image blur caused by head movement can be obtained based on the pixel displacement vector v(x, y).

[0081] In motion blur, the displacement vector = motion length × motion direction. A linear PSF is a one-dimensional or two-dimensional kernel whose intensity is uniformly distributed along a straight line, representing the blur trajectory formed by a point light source moving at a constant speed along that direction during exposure. The construction of a linear PSF includes:

[0082] 1) Determine the displacement vector (L, θ);

[0083] Assume the starting point of the motion is the origin (0, 0) and the ending point is (L cosθ, L sinθ), where L is the total length of the motion (i.e. the magnitude of the displacement vector) and θ is the angle between the direction of motion and the x-axis.

[0084] 2) Draw a line segment of length L along this direction on the image plane, and uniformly assign the value 1 / L to the line segment, and assign the value 0 to the rest;

[0085] The linear PSF can be expressed as:

[0086] ;

[0087] 3) Discretize into convolution kernels for image restoration.

[0088] The linear PSF is discretized into a K×K matrix (K > L) using bilinear interpolation or nearest neighbor sampling for convolution operations. This K×K matrix serves as the convolution kernel, i.e., the blur kernel. Image blurring can typically be modeled as a convolution operation between the original sharp image and the point spread function. If the blurred image and the simulated blur kernel are known, the blur introduced by the convolution operation can be canceled out through deconvolution (transposed convolution), simulating the original sharp image.

[0089] A point spread function is constructed, and a Wiener filter is applied to the region of interest (ROI) in the original target image based on this constructed point spread function to obtain the filtered ROI. The ROI is then deconvolved using Wiener filtering, and by introducing a statistical optimality criterion, a balance is struck between deblurring and noise suppression to reconstruct or restore the ROI, thereby restoring a clear image of the ROI. The core idea of ​​Wiener filtering is to minimize the mean square error between the original image and the estimated image.

[0090] In this embodiment, through steps S1041 and S1042, motion data is converted into "image degradation parameters." These parameters are then used to process the region of interest (ROI) to obtain a degraded image. Image degradation parameters refer to quantifiable or modelable physical and mathematical factors that cause image quality degradation during image acquisition, transmission, or processing. In this embodiment, the image degradation parameters are primarily motion blur parameters, including blur direction (angle), blur scale (length), displacement parameters (x, y) in a uniform linear motion model, and shutter time T. Then, through step S1043, Wiener filtering is applied to the degraded image to restore a clear ROI image.

[0091] The pixel displacement vector v(x, y) is the core prior parameter for performing deconvolution reconstruction in the region of interest. In steps S1041 to S1043, by converting the non-visual motion information perceived by the motion data acquisition unit into pixel degradation paths within the region of interest space, a transition from 'blind deconvolution' to 'non-blind deconvolution' is achieved, thus enabling sharpening restoration of a specific target with extremely low computational overhead. Non-blind deconvolution refers to a process where the blur kernel is known; through non-blind deconvolution, the sharp image is recovered only from the blurred image, resulting in generally better image restoration and a more stable image processing method. In contrast, blind deconvolution, where the blur kernel is unknown, requires simultaneous estimation of both the sharp image and the blur kernel during image restoration, leading to computational complexity, non-unique solutions, and a tendency to converge to incorrect solutions. Therefore, in this embodiment, steps S1041 to S1043 achieve both sharp restoration of the region of interest and reduce computational overhead while improving computational efficiency.

[0092] As can be seen from the above, in this embodiment, given the known point spread function (PSF) that causes image blurring, non-blind deconvolution can be used to perform image processing more effectively, thereby quickly and efficiently removing blur from the image and restoring the clear details of the image.

[0093] In other words, steps S2041 to S2043 constrain the image degradation space from 'isotropic' to 'anisotropic' specific motion paths by utilizing the motion vectors calculated in real time by the motion data acquisition unit. This asymmetric linear PSF, constructed based on prior knowledge from the hardware sensor (motion data acquisition unit), avoids the blind search in the temporal and spatial domains of traditional Gaussian deblurring algorithms, significantly improving the accuracy of barcode feature recovery within the region of interest.

[0094] For example, based on motion data, it can be determined that "the glasses sway 5 pixels to the left". The processor calculates the pixel displacement vector based on the motion data and constructs a point spread function that can accurately reflect the blur characteristics based on the pixel displacement vector. Then, based on the point spread function, the region of interest is reconstructed by Wiener filtering to obtain the repaired region of interest. Since the glasses sway 5 pixels to the left, when reconstructing the region of interest, the content of the region of interest needs to be pulled to the right according to the trajectory of the glasses swaying 5 pixels to the left to correct it.

[0095] The process of determining "the glasses wobbled 5 pixels to the left" based on motion data is as follows: When the glasses "wobbled to the left", the gyroscope sensed the change in angular velocity around the vertical axis, and the accelerometer recorded the lateral acceleration. The motion data can be combined with the focal length and sensor pixel size to convert the motion data into pixel-level data. For example, if a single pixel corresponds to a 0.1° field of view, then 5 pixels ≈ 0.5° offset.

[0096] In practice, after constructing the point spread function, other filtering methods can be used to denoise and restore the region of interest, resulting in a clear image of the region of interest. In step S1043, the Wiener filtering process based on the point spread function on the region of interest is as follows:

[0097] 1) Frequency Domain Transformation (Fourier Transform)

[0098] Image convolution operations are very time-consuming in the spatial domain, but become simple multiplications in the frequency domain. Therefore, in this embodiment, the image g(x, y) of the region of interest is first transformed into G(u, v) using a Fast Fourier Transform (FFT). Simultaneously, the directional blur model (point spread function) built based on v(x, y) is transformed into its frequency domain representation H(u, v).

[0099] 2) Constructing a Wiener filter

[0100] The core of Wiener filtering lies in its balance between deblurring and noise suppression. If only a simple inverse filter is used, high-frequency noise in the image will be amplified infinitely (resulting in numerous snowflake-like pixels). Therefore, in this embodiment, a Wiener filter (Wiener filter reduction formula) is constructed for filtering. The Wiener filter reduction formula (frequency domain form) is:

[0101] ;

[0102] in, The spectrum of the restored image; It is the directional fuzzy transfer function, or degenerate function, generated by u(x, y). It is a line; It is the conjugate complex number of H; K is the signal-to-noise ratio coefficient (constant).

[0103] because To represent a line, therefore, It has attenuation characteristics only in the direction of displacement, which makes Wiener filtering calculations very focused.

[0104] When the camera of the head-mounted device is a fixed-focus camera, since the ambient light may be insufficient, a fixed or adaptively varying K value can be set to prevent artifacts from being generated when the target image is restored.

[0105] 3) Inverse spatial domain transformation

[0106] After processing the frequency domain data Then, the frequency domain data is transformed back to the spatial domain using the inverse fast Fourier transform (IFFT) to obtain the deblurred region of interest image. As can be seen from the above, in this embodiment, image enhancement is achieved by performing directional compensation on the region of interest through Wiener filtering based on a directional blur model. Compared with the traditional deblurring process based on a Gaussian blur model, which requires processing 360° of possibilities, it has advantages such as energy concentration, computing power saving, and barcode feature enhancement.

[0107] Specifically, when reconstructing the region of interest, the Wiener filter precisely "pulls back" the pixel along the opposite direction of the pixel displacement vector v(x, y), achieving energy concentration. After the Fast Fourier Transform, because the point spread function is linear, a large number of elements in the matrix of the directional blur model are zero or minimum values. Therefore, the Wiener filter can process only the effective spectrum within the region of interest, greatly saving computational power.

[0108] Since the barcode to be identified is usually composed of parallel black and white stripes, if the jitter direction is perpendicular to the black and white stripes, the edge sharpness of the stripes can be greatly restored by using the Wiener filter for directional restoration; if the jitter direction is parallel to the black and white stripes, it is determined that the stripes generated by motion blur have little impact on recognition.

[0109] It should be noted that in the frequency domain, motion blur creates periodic zero-value fringes. These zero-value fringes are perpendicular to the actual jitter / motion direction, which may affect the recognition of black and white stripes in the barcode. In this case, directional Wiener filtering (i.e., using a known blur direction and PSF point spread function) can be used to specifically recover the high-frequency information lost along the blur direction, thereby enhancing the sharpness of the stripe edges of the barcode. However, when the jitter direction is parallel to the black and white stripes, the zero-value fringes generated by motion blur are perpendicular to the black and white stripes, having less impact on the recognition of the stripes, and Wiener filtering is unnecessary for image reconstruction.

[0110] In this embodiment, the geometric relationships such as the dizziness direction being perpendicular to the black and white stripes and the dizziness direction being parallel to the black and white stripes can be automatically identified based on the pixel displacement vector v(x,y), thereby achieving barcode feature enhancement.

[0111] As an optional embodiment, the barcode recognition method based on AR glasses head-mounted devices can also be implemented through the following steps: after enhancing the image of the region of interest based on the motion data, the method further includes enhancing the image edges of the region of interest, wherein enhancing the image edges of the region of interest includes:

[0112] S201, Obtain the image proportion of the region of interest in the target image;

[0113] S202, if the image proportion is less than the first preset image proportion threshold, the target image is resampled to obtain a resampled image;

[0114] S203, perform dynamic binarization processing on the resampled image to increase the edge contrast of the region of interest in the resampled image.

[0115] Since Wiener filtering may introduce artifacts or residual noise during deblurring, this embodiment can further optimize the image quality of the region of interest (ROI) by performing image edge enhancement in steps S201 to S203. The edge contrast of the ROI refers to the contrast between the black and white boundaries of the barcode to be identified. The ROI's proportion in the target image can be either the pixel proportion or the coverage area proportion.

[0116] In practice, the effective pixel percentage of the region of interest in the target image can be calculated. When the percentage of the region of interest (the barcode to be identified) is too small, it is determined that the camera is far away from the barcode to be identified when capturing the image (far-distance image capture). At this time, the original target image captured in step S101 can be magnified by bicubic interpolation.

[0117] The effective pixel percentage of the region of interest (ROI) in the target image refers to the proportion of effective pixels of the ROI to the total pixels of the entire target image. Effective pixels of ROI = length × width of the ROI (in pixels), which is the total number of pixels actually involved in image processing calculations.

[0118] For example, in this embodiment, the barcode to be identified is usually rectangular. Therefore, it can be determined whether the short side of the region of interest is smaller than a preset pixel (e.g., 100 pixels). If so, the region of interest in the acquired target image is determined to be a "distant / small target".

[0119] At this point, a high-resolution slice of the corresponding region of interest is cropped from the original full-resolution target image (e.g., 5MP original pixels). Then, the original target image is resampled using bicubic interpolation, i.e., digital zoom is used to process the target image to improve the image quality of the region of interest. Digital zoom (also known as digital image zoom) is a technique that uses a processor to interpolate and amplify the image signal. It achieves a visual zoom-in by cropping the central area of ​​the image and filling in pixels, without changing the physical focal length of the lens.

[0120] Because the 5MP sensor used to acquire the original target image has more physical pixels, although the camera's optical focal length is fixed, weak edge gradients are preserved in the original pixels. By resampling using bicubic interpolation, these gradient information from the target image can be used to reconstruct sharper edges, making the barcode edges clearer. Bicubic interpolation considers a wider range of pixel information, enabling more accurate reconstruction of local gradients and curvature. Bicubic interpolation, also known as bicubic interpolation, is a method used to "interpolate" or increase the number / density of pixels in an image. Unlike simple nearest neighbor interpolation (which results in severe jagged edges) or bilinear interpolation (which blurs edges), bicubic interpolation considers the gray values ​​of 4×4 (16 in total) neighboring pixels around the sampling point (target point). During the interpolation process, a weighted average is calculated for the 16 pixels (4×4 neighborhood) around the target point, and a continuously differentiable cubic polynomial is used as the weight function to generate smooth image edges.

[0121] The specific steps of bicubic interpolation include: 1) Constructing the convolution kernel: Using a third-order polynomial (usually a bicubic convolution kernel) as the weight function, a convolution kernel is constructed. 2) Weighted calculation: In the x-axis direction, cubic spline interpolation is performed on the four rows of pixels to obtain four intermediate values; in the y-axis direction, cubic spline interpolation is performed again on these four intermediate values. 3) Edge smoothing: Not only the grayscale of the pixels is considered, but also the derivative (rate of change) of the grayscale change is considered to smooth the edges of the region of interest, so that the edges of the enlarged barcode can maintain a smooth transition and reduce the blurring caused by the "circle of confusion" of a fixed-focus lens.

[0122] After obtaining the resampled image through bicubic interpolation resampling, the magnified region of interest after resampling is dynamically binarized (dynamically adjusting the binarization effect of the image). The smooth edges generated by bicubic interpolation can provide a stable slope for the binarization algorithm, making the final generated black and white barcode image (Bitstream) extremely regular, strengthening the boundary of the region of interest, and effectively improving the decoding rate of the barcode scanning engine.

[0123] Image binarization can convert color or grayscale images into black and white binary images, thereby highlighting important features in the image. In this embodiment, binarization is mainly used to separate the region of interest from the image background.

[0124] Dynamic binarization is a binarization method that dynamically calculates thresholds based on the characteristics of local image regions. It is suitable for scenarios with uneven lighting and significant local contrast differences. Unlike global thresholding (global binarization), which uses a single threshold to distinguish the region of interest from the background of the entire image, dynamic binarization calculates a specific threshold for each pixel, thus preserving details more accurately and improving image segmentation precision. Local image regions with different brightness, contrast, and texture will use corresponding local binarization thresholds.

[0125] In this embodiment, Local Adaptive Thresholding (LAT) can be used to process soft edges generated by a fixed-focus lens, enhancing the contrast of the black and white boundaries of the barcode. The LLT algorithm no longer uses a global brightness threshold on the magnified high-resolution region of interest; instead, it calculates a dynamic "0 / 1" boundary based on the local contrast of the black and white bars of the barcode. This LLT algorithm no longer uses a uniform brightness value (e.g., 127) for segmentation across the entire image; instead, it calculates the "neighborhood mean" for each pixel within the region of interest.

[0126] In step S203, the specific process of processing soft edges using a local adaptive threshold is as follows:

[0127] 1) Local adaptive threshold processing

[0128] A) Sliding window

[0129] Define an n×n window, where the window size is typically related to the expected width of the finest stripe of the barcode to be identified. Move a fixed- or variable-sized window over the region of interest and analyze the local area within each window.

[0130] B) Local adaptive threshold calculation

[0131] The formula for calculating the local adaptive threshold is:

[0132] T(x, y) = mean(ROI local ) – C;

[0133] Where T(x, y) is the dynamic threshold of the pixel, C is a compensation constant (used to suppress background noise), and mean(ROI) local ) is the average value of the neighboring pixels of the pixel in the region of interest.

[0134] In this step, the threshold for each pixel is dynamically calculated based on the local features of its neighborhood (e.g., an n×n window). Specifically, the threshold for each pixel is dynamically calculated based on the statistical characteristics of the data within the window (e.g., mean, standard deviation, noise level), achieving pixel-by-pixel image processing. After enhancing the region of interest through local adaptive thresholding, even if part of the barcode is in shadow and another part is in strong light, the local thresholding ensures that each stripe in the barcode can be independently identified, eliminating interference from ambient light contrast.

[0135] 2) Suppress "soft edges" through gradient transformation

[0136] The "soft edges" produced by a fixed-focus camera appear as a gentle slope in a grayscale image. In this embodiment, dynamic binarization is used to artificially "sharpen" the boundaries by utilizing the gradient of this gentle slope:

[0137] A) Edge detection: Identify the point where the grayscale changes most drastically (i.e., the peak of the gradient).

[0138] B) Hard cropping: During the binarization process, pixels in the middle of the grayscale slope are brought closer to 0 (pure black) or 255 (pure white) based on their local mean.

[0139] C) Hard contrast enhancement: Compresses the edges of a Gaussian distribution into a step function, thereby artificially compensating for the loss of contrast caused by insufficient optical quality of fixed-focus lenses.

[0140] 3) Morphological Refinement: After the initial binarization, to further enhance the coherence of the black-and-white boundaries, this step performs lightweight morphological operations on the barcode features:

[0141] A) Dilation & Erosion

[0142] By using the closing operation (corrosion followed by expansion), the white noise points that may exist in the black stripes are filled (to resist interference).

[0143] By performing an opening operation, which involves first dilating and then eroding, scattered black isolated pixels in the background are eliminated.

[0144] B) Use of structural elements

[0145] Because the barcode to be identified has a "long and thin" geometric feature, rectangular structural elements can be used to enhance the continuity of the vertical stripes.

[0146] As shown above, when a user scans a barcode at close range using a fixed-focus lens, severe edge blurring occurs due to defocusing. In this case, steps S201 to S202 can be used to resample the original target image (digital zoom processing). Then, step S203 performs dynamic binarization on the resampled image to adjust the contrast of the region of interest (ROI) in the target image, enhancing the contrast of the black and white boundaries of the barcode, thereby improving the edge enhancement of the ROI and increasing the accuracy of subsequent barcode recognition. Close-range blurring refers to the phenomenon of severe edge blurring caused by defocusing when a user scans a barcode at close range.

[0147] Furthermore, in step S105, performing barcode recognition on the enhanced region of interest includes:

[0148] S1051, Barcode recognition is performed on the region of interest after edge contrast enhancement.

[0149] In this embodiment, after removing motion blur in the region of interest (ROI) by performing image enhancement based on the motion data in step S104 (first image enhancement), the ROI is further enhanced in steps S201 to S203 to increase the edge contrast (second image enhancement), highlighting the details and clarity of the ROI edges and optimizing the image quality. Then, in step S1051, barcode recognition is performed on the ROI after edge contrast enhancement to accurately identify the barcode content. In other words, this embodiment can effectively improve the image quality of the ROI through multiple image enhancements, thereby achieving accurate barcode recognition.

[0150] It should be noted that the above-mentioned steps for enhancing the image edges of the region of interest (steps S201 to S203) can be performed after step S104, which enhances the image of the region of interest based on the motion data. This achieves secondary image enhancement of the region of interest (first enhancing the image of the region of interest based on the motion data, and then refining and enhancing the edges of the region of interest), thereby improving the scanning clarity of the region of interest. In a specific implementation, after acquiring the target image through the camera of the head-mounted device in step S101 and identifying the region of interest in the target image in step S103, the image edges of the region of interest can be directly enhanced through steps S201 to S203 to improve the contrast of the black and white edges in the barcode to be recognized.

[0151] As an optional embodiment, after performing step S101, i.e., acquiring the target image through the camera of the head-mounted device, the method further includes:

[0152] S301, the target image is downsampled to obtain a downsampled image.

[0153] In step S103, identifying the region of interest in the target image includes:

[0154] Identify the region of interest from the downsampled image.

[0155] In step S201, obtaining the image proportion of the region of interest in the target image includes:

[0156] Obtain the percentage of the region of interest in the downsampled image.

[0157] As shown above, after acquiring the target image through the camera of the head-mounted device, the target image can be downsampled, for example, downsampled to a low resolution of 640×480, resulting in a low-resolution downsampled image. Then, the processor can quickly identify and locate the barcode to be recognized in the downsampled image, obtaining the region of interest (ROI) corresponding to the barcode. By downsampling the original target image to a low resolution (e.g., 640×480), the coordinates of the ROI (barcode to be recognized) within the entire downsampled image can be quickly located, achieving fast and lightweight ROI localization and improving image processing efficiency.

[0158] Since step S103 involves identifying the region of interest from the downsampled image and performing image processing, when performing edge enhancement on the region of interest through steps S201 to S203, the image proportion of the region of interest in the downsampled image is first obtained through step S201. For example, the effective pixel proportion of the region of interest in the 640×480 downsampled image is calculated, and then it is determined whether the effective pixel proportion of the region of interest is less than a preset pixel proportion threshold. If so, the processor no longer processes the 640×480 low-resolution downsampled image, but instead goes back to the original full-resolution target image (e.g., 5MP original pixels), resamples the original target image, performs dynamic binarization processing on the resampled image, and performs edge enhancement on the region of interest in the resampled image.

[0159] The downsampled image obtained through downsampling has certain limitations: if enlarged on a low-resolution 640×480 target image, due to information loss, the enlargement will only result in a "blurry mosaic". Therefore, in this embodiment, bicubic interpolation is used to resample the original target image. The 5MP sensor used to acquire the original target image has more physical pixels. Although the focal length of the camera optics is fixed, weak edge gradients are preserved in the original pixels. Through bicubic interpolation, these gradient information are used to reconstruct sharper edges, making the barcode edges clearer.

[0160] As an optional embodiment, the barcode recognition method based on a head-mounted device can also be implemented through the following steps: The method further includes:

[0161] S401, determine the blur type of the region of interest based on the pixel displacement vector;

[0162] S402, if the blur type of the region of interest is optical blur, determine whether the blur degree of the region of interest exceeds a preset blur degree threshold;

[0163] S403, if the value exceeds the limit, then in step S105, the step of performing barcode recognition on the enhanced region of interest includes:

[0164] S1052, an interface identifier and corresponding head-mounted device movement indication information are generated in the user interface of the head-mounted device to perform barcode recognition on the enhanced region of interest based on the interface identifier and the head-mounted device movement indication information. The interface identifier represents the target recognition window of the camera when performing the barcode recognition, and the head-mounted device movement indication information is used to instruct the wearer of the head-mounted device to move so that the region of interest is within the range of the interface identifier.

[0165] In this embodiment, the processor can evaluate the blur type of the region of interest (ROI) based on the pixel displacement vector. Then, if the blur type of the ROI is optical blur, the processor further evaluates the blur degree of the current ROI to determine whether it exceeds a preset blur degree threshold. Optical blur refers to the phenomenon of unclear imaging caused by the physical characteristics of the optical system (such as lenses, lens elements, etc.) or limitations of usage conditions. For example, when the lens is not in focus (the lens focus does not fall accurately on the subject), the object point forms a "circle of confusion" on the lens, causing image blur.

[0166] If the blurriness of the region of interest exceeds a preset blur threshold, it is determined that the blurriness exceeds the image processing correction limit, meaning that a clear image of the region of interest cannot be obtained through processing. In this case, the user interface (UI) of the head-mounted device can generate a virtual interface identifier (UI identifier) ​​for barcode recognition and corresponding head-mounted device movement instructions. This guides the user to move the head-mounted device back and forth until the region of interest enters the virtual interface identifier, thus determining that the barcode to be recognized is within the camera's "optimal recognition window." That is, the target recognition window is the "optimal recognition window" of the camera lens. The head-mounted device movement instructions (or head-mounted device movement guidance information) are instructions for the wearer to move the device. By moving according to these instructions, the user can bring the region of interest within the interface identifier range (within the target recognition window).

[0167] The interface identifier can be automatically generated when the blur type of the region of interest (ROI) is optical blur and the blur level of the ROI exceeds a preset blur level threshold. This indicates to the wearer of the head-mounted device whether the ROI (the barcode to be recognized) is within the camera's "optimal recognition window." If the ROI is not within the "optimal recognition window" (entirely or partially outside the interface identifier), the head-mounted device provides movement instructions to guide the wearer to move the ROI within the camera's "optimal recognition window." The camera then directly scans the ROI within the "optimal recognition window" to achieve high-definition image acquisition, avoiding image blur caused by optical blur. In this embodiment, the interactive guidance of the interface identifier and corresponding head-mounted device movement instructions forms a complete closed loop of hardware, software (image processing), and user interaction. This not only solves image blur caused by user head movement but also image blur caused by physical limitations of the hardware (optical system), improving barcode recognition accuracy. The interface identifier can be a box-shaped identifier, a color identifier, or a brightness identifier, etc., and the head-mounted device movement instructions can be directional arrows, color change indicators, brightness change indicators, and voice instructions, etc.

[0168] As an optional embodiment, step S401 can be achieved by the following steps: determining the blur type of the region of interest based on the pixel displacement vector, including:

[0169] S4011, if the coordinate value of the pixel displacement vector is 0 or less than the first preset vector threshold, determine that the blur type of the region of interest is optical blur;

[0170] S4012, if the coordinate value of the pixel displacement vector is greater than the second preset vector threshold, determine that the blur type of the region of interest is motion blur.

[0171] In this embodiment, the blur type of the region of interest can be quickly determined based on the magnitude of the pixel displacement vector v(x, y) coordinates. When the coordinates of v(x, y) are large and the blur is directional (along the vector direction), the blur type of the region of interest is determined to be motion blur, and the motion blur processing method described above can be used. However, when the coordinates of v(x, y) are small, such as close to 0, the image is still blurry, and the blur is "omnidirectional (isotropic)," the blur type of the region of interest is determined to be optical blur, that is, the blur is mainly caused by "optical depth of focus." Enhancing the image may not be able to identify the region of interest, and the camera needs to be adjusted.

[0172] Optical depth of focus is an important parameter in optical imaging systems. It describes the maximum range that the image plane (such as the plane where the camera sensor or microscope eyepiece is located) can move along the optical axis. Within this range, a clear image can still be maintained; but when it exceeds this range, a clear image cannot be formed.

[0173] As an optional embodiment, step S402 can be achieved through the following steps: determining whether the blur level of the region of interest exceeds a preset blur level threshold includes at least one of the following:

[0174] S4021, determine whether the blur level of the region of interest exceeds a preset blur level threshold based on the sharpness function of gradient energy;

[0175] S4022, Based on frequency domain high-frequency component analysis, determine whether the ambiguity of the region of interest exceeds a preset ambiguity threshold;

[0176] S4023, determine whether the blur level of the region of interest exceeds a preset blur level threshold based on the contrast distribution of the barcode.

[0177] In this embodiment, the blur level of the region of interest can be determined in a variety of ways to determine whether it exceeds a preset blur level threshold.

[0178] Step S4021 uses a gradient energy-based "sharpness function" to calculate the degree of change in pixel values ​​within the region of interest, thereby assessing the blur (or sharpness) of the region of interest.

[0179] Specifically, the region of interest is convolved using the Tenengrad gradient operator or the Laplacian operator.

[0180] For example, the Laplacian operator is a second-order differential operator used to measure the second derivative of the gray value of a point in an image with respect to the gray values ​​of its neighbors. It calculates the gray values ​​of pixels in the neighborhood through gray-level differences. The basic process includes: 1) comparing the gray value of the center pixel with the gray values ​​of other pixels around it; 2) if the gray value of the center pixel is higher, increasing the gray value of the center pixel; otherwise, decreasing the gray value of the center pixel, thus achieving image sharpening. The Laplacian operator is represented as:

[0181] .

[0182] The region of interest (ROI) is convolved using either the Tenengrad gradient operator or the Laplacian operator to obtain pixel gradients. Energy integration is then performed to calculate the sum of squares (variance) of gradients across all pixels within the ROI, yielding its energy value. If the energy value (variance) of the ROI is lower than a preset energy threshold, the ROI is considered blurred; if the variance is higher than the preset energy threshold, the ROI is considered unblurred. High energy values ​​indicate sharp edges, with extremely large gradients at the black-and-white boundary, placing it within the camera's "optimal recognition window." Low energy values ​​indicate smooth edges (long gray-scale transition bands), meaning severe optical blur.

[0183] Step S4022 involves high-frequency analysis in the frequency domain, specifically using Fourier transform to assess the blur level of the region of interest (ROI). Specifically, the spectral distribution can be calculated, where a sharp image contains a large number of high-frequency components (representing details and edges) in the frequency domain. The processor extracts the spectral energy distribution map of the ROI, calculates the ratio of high-frequency energy to low-frequency energy, and then assesses the blur level of the ROI based on this ratio. If the high-frequency energy decays rapidly, it indicates that the optical system cannot resolve the fine stripes of the barcode, and the ROI is determined to have a high degree of blur, falling outside the depth of focus.

[0184] In step S4023, based on the barcode's unique "Edge Contrast Profile," its geometric prior is used to more accurately assess the blur level of the region of interest. Specifically, a row of pixel grayscale curves perpendicular to the barcode stripes can be extracted through step response analysis. Then, the rise time (i.e., the number of pixels spanned from black (low brightness) to white (high brightness)) is calculated to determine if the number of pixels meets a preset condition. For example, if the number of pixels is ≤ 2 pixels, the region of interest is determined to be in excellent focus with low blur; if the number of pixels is > 5 pixels, the region of interest has soft edges and high blur.

[0185] Optionally, in step S4023, the maximum grayscale difference between the black and white bars in the region of interest can be calculated to determine the peak-to-peak contrast of the region of interest. Then, based on the peak-to-peak contrast, it can be determined whether the blurriness of the region of interest exceeds the preset compensation limit of image processing. For example, if the peak-to-peak contrast is less than 30%, it is determined that the blurriness of the region of interest exceeds the preset compensation limit, and the user needs to be guided to move using a head-mounted device so that the region of interest is in the "optimal recognition window" of the camera.

[0186] As an optional embodiment, step S1052, generating an interface identifier and corresponding head-mounted device movement indication information on the user interface of the head-mounted device, includes:

[0187] S501, the imaging state of the region of interest is evaluated based on the visual features of the region of interest to obtain an evaluation result;

[0188] S502, based on the evaluation results, generate the marking information of the interface identifier and the movement indication information of the head-mounted device, wherein the movement indication information of the head-mounted device is generated based on the relative positional relationship between the interface identifier and the region of interest.

[0189] In this embodiment, the imaging state of the region of interest (ROI) can be evaluated based on its visual characteristics. Based on the evaluation result, corresponding interface identifier marking information and head-mounted device movement indication information are generated on the user interface to achieve precise guidance. The interface identifier marking information includes at least one of shape markings, color markings, brightness markings, and flashing markings. The relative positional relationship between the interface identifier and the ROI includes direction and / or distance.

[0190] For example, when the region of interest is within the camera's "optimal recognition window," the interface marker can be changed to green to lock the region of interest within a green frame, thus enabling its recognition. Conversely, when the region of interest is not within the camera's "optimal recognition window," the interface marker can be changed to red to prompt the user to move the head-mounted device to bring the region of interest within the camera's "optimal recognition window." Based on the relative position of the interface marker and the region of interest, movement instructions for the head-mounted device are generated, prompting the user to move according to these instructions to bring the region of interest within the camera's "optimal recognition window."

[0191] As an optional embodiment, in step S501, evaluating the imaging state of the region of interest based on its visual features to obtain an evaluation result includes:

[0192] S5011, if the edge of the region of interest spreads, and / or if the peak-to-peak contrast of the region of interest is lower than a preset contrast threshold, then it is determined that the region of interest is outside the depth of focus of the camera, and the distance between the region of interest and the camera is less than a preset first distance threshold.

[0193] S5012, if the gradient energy of the region of interest is within a preset peak range, then the region of interest is determined to be within a preset depth of focus;

[0194] S5013, if the image proportion of the region of interest in the target image is less than a second preset image proportion threshold, and / or if the region of interest has missing high-frequency image features, then it is determined that the region of interest is outside the focal depth of the camera, and the distance between the region of interest and the camera is greater than a preset second distance threshold.

[0195] like Figure 2 As shown, the imaging state evaluation results of the region of interest (ROI) can include three types: near-focus, optimal depth of focus, and far-focus. Specifically, if the edges of the ROI are diffused, and / or if the peak-to-peak contrast of the ROI is extremely low, the imaging state of the ROI is determined to be near-focus, meaning the ROI is a close-range image and is outside the depth of focus; the head-mounted device needs to be moved to bring the ROI to the optimal depth of focus. If the gradient energy of the ROI is within a preset peak range, the imaging state of the ROI is determined to be at the optimal depth of focus. If the ROI's image proportion in the low-resolution target image is less than a third preset image proportion threshold, and / or if the ROI has missing high-frequency image features, the state of the ROI is determined to be far-focus, meaning the ROI is a distant-range image and is outside the depth of focus; the head-mounted device needs to be moved to bring the ROI to the optimal depth of focus.

[0196] As an optional embodiment, in step S502, generating an interface identifier and corresponding head-mounted device movement indication information on the user interface of the head-mounted device based on the evaluation result includes:

[0197] S5021, Determine the focal length offset of the region of interest based on the evaluation results;

[0198] S5022, Generate the head-mounted device movement indication information based on the focal length offset.

[0199] The evaluation result of the relative positional relationship between the interface identifier and the region of interest is mapped to a focal length offset. Then, the processor can generate corresponding head-mounted device movement indication information based on the focal length offset.

[0200] When the region of interest is assessed as near-focus, the UI (user interface) prompts "Please move it a little further away," or displays a backward arrow, prompting the user to move backward so that the region of interest is in the "optimal recognition window," for example, as... Figure 3 As shown in the figure, the upward and downward arrows represent backward movement; when the evaluation result of the region of interest is the optimal depth of focus, the region of interest is determined to be in the "optimal recognition window," which can be achieved as follows: Figure 3 As shown, the generated virtual interface identifier is locked in green, indicating to the user that the area of ​​interest is currently in the "optimal recognition window" and can be directly scanned for identification; when the evaluation result of the area of ​​interest is superfocal distance, the UI prompts "Please move closer to the target", or the UI displays a forward arrow, prompting the user to move backward so that the area of ​​interest is within the "optimal recognition window" (within the range of the interface identifier).

[0201] In summary, the barcode recognition method based on a head-mounted device provided in this application has the following beneficial effects:

[0202] (1) The motion data of the head-mounted device is used to enhance the image of the region of interest, and the fixed-focus lens is used to improve the image quality of the region of interest and improve the barcode recognition rate. This solves the problem of "momentation" caused by the fixed-focus shutter time under high resolution of the user's head motion image when the head-mounted device performs barcode recognition. High-performance barcode recognition is achieved without increasing hardware costs. Moreover, image enhancement is only required for the region of interest, which can improve image processing efficiency.

[0203] (2) After acquiring the target image through the camera, the target image is downsampled first, and then the region of interest in the downsampled image is enhanced. The region of interest can be quickly located using the low-resolution downsampled image, and the region of interest can be enhanced using motion data. This greatly balances the contradiction between the high pixel requirement and the low power consumption limitation of the head-mounted device, improves the efficiency and quality of image processing, and reduces power consumption.

[0204] (3) By transforming the physical limitations (out-of-focus areas) that software algorithms (image processing algorithms) cannot solve into UI guidance, a complete closed loop of hardware, software and user interaction is formed, which improves the barcode recognition accuracy and the interaction performance between the head-mounted device and the user.

[0205] Figure 4 This is a structural block diagram of the barcode recognition device based on a head-mounted device provided in the embodiments of this application, such as... Figure 4 As shown, based on the above-described barcode recognition method for head-mounted devices, this application embodiment also provides a barcode recognition device based on a head-mounted device, comprising:

[0206] The acquisition module 100 is configured to acquire a target image containing a barcode to be identified through the camera of the AR glasses head-mounted device;

[0207] The second acquisition module 200 is configured to acquire motion data of the head-mounted device, wherein the motion data is motion data within the exposure time of the camera;

[0208] The first recognition module 300 is configured to recognize the region of interest in the target image, wherein the region of interest is the region of interest corresponding to the target barcode to be recognized;

[0209] Image enhancement module 400 is configured to enhance the image of the region of interest based on the motion data;

[0210] The second recognition module 500 is configured to perform barcode recognition on the region of interest after image enhancement.

[0211] Optionally, the image enhancement module 400 may be further configured as follows:

[0212] Based on the motion data, determine the pixel displacement vector caused by the head movement of the wearer of the head-mounted device;

[0213] Construct a point spread function based on the pixel displacement vector;

[0214] Wiener filtering is performed on the region of interest based on the point spread function to obtain the filtered region of interest.

[0215] Optionally, the image enhancement module 400 is further configured to: after performing image enhancement on the region of interest based on the motion data, perform image edge enhancement on the region of interest, wherein the image edge enhancement on the region of interest includes:

[0216] Obtain the image percentage of the region of interest in the target image;

[0217] If the image proportion is less than a first preset image proportion threshold, the target image is resampled to obtain a resampled image;

[0218] The resampled image is subjected to dynamic binarization, and the edge contrast of the region of interest in the resampled image is increased.

[0219] The second identification module 500 is also configured as follows:

[0220] Barcode recognition is performed on the region of interest after edge contrast enhancement.

[0221] Optionally, the barcode recognition device based on the head-mounted device also includes a downsampling module, configured as follows:

[0222] After acquiring a target image through the camera of a head-mounted device, the target image is downsampled to obtain a downsampled image;

[0223] The first identification module 300 is further configured as follows:

[0224] Identify the region of interest from the downsampled image;

[0225] The image enhancement module 400 is also configured as follows:

[0226] Obtain the percentage of the region of interest in the downsampled image.

[0227] Optionally, the barcode recognition device based on the head-mounted device also includes an interaction module, configured as follows:

[0228] The blur type of the region of interest is determined based on the pixel displacement vector;

[0229] If the blur type of the region of interest is optical blur, then determine whether the blur degree of the region of interest exceeds a preset blur degree threshold;

[0230] If the range is exceeded, the second identification module 500 is further configured to: generate an interface identifier and corresponding head-mounted device movement indication information on the user interface of the head-mounted device, so as to perform barcode recognition on the image-enhanced region of interest based on the interface identifier and the head-mounted device movement indication information, wherein the interface identifier represents the target recognition window of the camera when performing the barcode recognition, and the head-mounted device movement indication information is used to instruct the wearer of the head-mounted device to move so that the region of interest is within the range of the interface identifier.

[0231] Optionally, the interaction module can also be configured as follows:

[0232] If the coordinate value of the pixel displacement vector is 0 or less than a first preset vector threshold, the blur type of the region of interest is determined to be optical blur.

[0233] If the coordinate value of the pixel displacement vector is greater than the second preset vector threshold, the blur type of the region of interest is determined to be motion blur.

[0234] Optionally, the interaction module can also be configured as follows:

[0235] The imaging state of the region of interest is evaluated based on its visual features, and an evaluation result is obtained.

[0236] Based on the evaluation results, the marking information of the interface identifier and the movement indication information of the head-mounted device are generated, wherein the movement indication information of the head-mounted device is generated based on the relative positional relationship between the interface identifier and the region of interest.

[0237] Optionally, the interaction module can also be configured as follows:

[0238] If the edge of the region of interest spreads, and / or if the peak-to-peak contrast of the region of interest is lower than a preset contrast threshold, then it is determined that the region of interest is outside the target depth of focus range of the camera, and the distance between the region of interest and the camera is less than a preset first distance threshold, wherein the peak-to-peak contrast is the maximum grayscale difference between the black bars and white bars of the region of interest.

[0239] If the gradient energy of the region of interest is within a preset peak range, then the region of interest is determined to be within the target depth of field range of the camera.

[0240] If the region of interest has a smaller percentage of the target image than a second preset image percentage threshold, and / or if the region of interest has missing high-frequency image features, then the region of interest is determined to be outside the target depth of field of the camera, and the distance between the region of interest and the camera is greater than a preset second distance threshold.

[0241] It should be noted that the first acquisition module 100, the second acquisition module 200, the first recognition module 300, the image enhancement module 300 and the second recognition module 400 mentioned above correspond to steps S101 to S105 in the embodiments. The multiple modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. Figure 5 A hardware structure block diagram of a head-mounted device according to an embodiment of this application is shown. Figure 5 As shown, an embodiment of this application can provide a head-mounted device, including a camera, a motion data acquisition unit, a memory, and a processor. The camera is used to acquire a target image, the motion data acquisition unit is used to acquire motion data of the head-mounted device, the memory stores a computer program, and the processor implements the above-described barcode recognition method when executing the computer program in the memory.

[0242] The head-mounted device also includes a display unit, which is used to display interface identifiers and corresponding head-mounted device movement indication information. When the processor executes the computer program on the memory, it implements the above-mentioned barcode recognition method (method steps executed by the interaction module) that interacts with the display unit.

[0243] like Figure 5As shown, a head-mounted device may include one or more processors (processors may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory for storing data, a camera, and a motion data acquisition unit. In addition, it may include: a display unit, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, or a power supply. Those skilled in the art will understand that... Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, a head-mounted device may also include... Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.

[0244] It should be noted that the aforementioned one or more processors or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits can be wholly or partially embodied in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuit can be a single, independent processing module, or wholly or partially integrated into any other element in the computer terminal 10. As involved in the embodiments of this application, the data processing circuit serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface). The memory can be used to store computer programs and modules, such as the program instructions / modules corresponding to the barcode recognition method and apparatus based on a head-mounted device in the embodiments of this application. The processor executes various functional applications and data processing by running the computer programs and modules stored in the memory, thereby realizing the aforementioned barcode recognition method based on a head-mounted device.

[0245] The memory may include high-speed random access memory (RAM), and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, which can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0246] The processor can invoke the information and computer program stored in the memory through the transmission device to execute the steps of the barcode recognition method based on the head-mounted device described above. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructing the hardware related to the terminal device through a program. The program can be stored in a non-volatile storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0247] Head-mounted devices can include AR glasses, AR helmets, or head-mounted shooting devices, etc.

[0248] Embodiments of this application also provide a non-volatile storage medium. Optionally, in this embodiment, the aforementioned non-volatile storage medium can be used to store the program code executed by the barcode recognition method based on a head-mounted device provided in the above embodiments.

[0249] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0250] Embodiments of this application also provide a computer program product, including a computer program. Optionally, in this embodiment, the computer program, when executed by a processor, can implement:

[0251] A target image is acquired using a camera on a head-mounted device; motion data of the head-mounted device is acquired, wherein the motion data is motion data within the exposure time of the camera; a region of interest (ROI) is identified in the target image, wherein the ROI corresponds to the target barcode to be identified; image enhancement is performed on the ROI based on the motion data; and barcode recognition is performed on the enhanced ROI.

[0252] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0253] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0254] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0255] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0256] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0257] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0258] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A barcode recognition method based on a head-mounted device, characterized in that, include: Acquire target images using the camera of a head-mounted device; Acquire motion data of the head-mounted device, wherein the motion data is motion data within the exposure time of the camera; Identify the region of interest in the target image, wherein the region of interest is the region of interest corresponding to the target barcode to be identified; Image enhancement is performed on the region of interest based on the motion data; Barcode recognition is performed on the region of interest after image enhancement.

2. The method according to claim 1, characterized in that, The image enhancement of the region of interest based on the motion data includes: Based on the motion data, determine the pixel displacement vector caused by the head movement of the wearer of the head-mounted device; Construct a point spread function based on the pixel displacement vector; Wiener filtering is performed on the region of interest based on the point spread function to obtain the filtered region of interest.

3. The method according to claim 1, characterized in that, After enhancing the image of the region of interest based on the motion data, the method further includes enhancing the image edges of the region of interest, wherein enhancing the image edges of the region of interest includes: Obtain the image percentage of the region of interest in the target image; If the image proportion is less than a first preset image proportion threshold, the target image is resampled to obtain a resampled image; The resampled image is dynamically binarized to increase the edge contrast of the region of interest in the resampled image; The step of performing barcode recognition on the enhanced region of interest includes: Barcode recognition is performed on the region of interest after edge contrast enhancement.

4. The method according to claim 3, characterized in that, After acquiring the target image through the camera of the head-mounted device, the method further includes: The target image is downsampled to obtain a downsampled image; The identification of the region of interest in the target image includes: Identify the region of interest from the downsampled image; The step of obtaining the image proportion of the region of interest in the target image includes: Obtain the percentage of the region of interest in the downsampled image.

5. The method according to claim 2, characterized in that, The method further includes: The blur type of the region of interest is determined based on the pixel displacement vector; If the blur type of the region of interest is optical blur, then determine whether the blur degree of the region of interest exceeds a preset blur degree threshold; If the value exceeds the limit, the step of performing barcode recognition on the enhanced region of interest includes: An interface identifier and corresponding head-mounted device movement indication information are generated in the user interface of the head-mounted device to perform barcode recognition on the enhanced region of interest based on the interface identifier and the head-mounted device movement indication information. The interface identifier represents the target recognition window of the camera when performing the barcode recognition, and the head-mounted device movement indication information is used to instruct the wearer of the head-mounted device to move so that the region of interest is within the range of the interface identifier.

6. The method according to claim 5, characterized in that, Determining the blur type of the region of interest based on the pixel displacement vector includes: If the coordinate value of the pixel displacement vector is 0 or less than a first preset vector threshold, the blur type of the region of interest is determined to be optical blur. If the coordinate value of the pixel displacement vector is greater than the second preset vector threshold, the blur type of the region of interest is determined to be motion blur.

7. The method according to claim 5, characterized in that, The process of generating an interface identifier and corresponding head-mounted device movement indication information on the user interface of the head-mounted device includes: The imaging state of the region of interest is evaluated based on its visual features, and an evaluation result is obtained. Based on the evaluation results, the marking information of the interface identifier and the movement indication information of the head-mounted device are generated, wherein the movement indication information of the head-mounted device is generated based on the relative positional relationship between the interface identifier and the region of interest.

8. The method according to claim 7, characterized in that, The evaluation of the imaging state of the region of interest based on its visual features yields an evaluation result, including: If the edge of the region of interest spreads, and / or if the peak-to-peak contrast of the region of interest is lower than a preset contrast threshold, then it is determined that the region of interest is outside the target depth of focus range of the camera, and the distance between the region of interest and the camera is less than a preset first distance threshold, wherein the peak-to-peak contrast is the maximum grayscale difference between the black bars and white bars of the region of interest. If the gradient energy of the region of interest is within a preset peak range, then the region of interest is determined to be within the target depth of field range of the camera. If the region of interest has a smaller percentage of the target image than a second preset image percentage threshold, and / or if the region of interest has missing high-frequency image features, then the region of interest is determined to be outside the target depth of field of the camera, and the distance between the region of interest and the camera is greater than a preset second distance threshold.

9. A barcode recognition device based on a head-mounted device, characterized in that, include: The first acquisition module is configured to acquire target images through the camera of a head-mounted device; The second acquisition module is configured to acquire motion data of the head-mounted device, wherein the motion data is motion data within the exposure time of the camera; The first recognition module is configured to recognize the region of interest in the target image, wherein the region of interest is the region of interest corresponding to the target barcode to be recognized; The image enhancement module is configured to enhance the image of the region of interest based on the motion data; The second recognition module is configured to perform barcode recognition on the region of interest after image enhancement.

10. A head-mounted device, characterized in that, The device includes a camera, a motion data acquisition unit, a memory, and a processor. The camera is used to acquire a target image, the motion data acquisition unit is used to acquire motion data of the head-mounted device, the memory stores a computer program, and the processor implements the method of any one of claims 1 to 4 when executing the computer program in the memory. The head-mounted device further includes a display unit for displaying interface identifiers and corresponding head-mounted device movement indication information, wherein the processor implements the method of any one of claims 5 to 8 when executing the computer program on the memory.

Images