Image recognition method based on hypertext markup language, electronic device, and program product

Abstract

The present disclosure provides a hyper text markup language-based image recognition method, an electronic device and a program product. The method of the present disclosure comprises: collecting a video stream in a hyper text markup language page running environment, and determining candidate video frames meeting image quality standards; registering and weighting fusion of a plurality of candidate video frames based on fusion weights to obtain a fusion image; selecting a corresponding image enhancement strategy for optimization processing based on the environmental scene features corresponding to the fusion image to obtain an enhanced image; and recognizing the enhanced image to obtain an image recognition result.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to image recognition methods, electronic devices, and program products based on Hypertext Markup Language. Background Technology

[0002] With the development of mobile internet technology, HyperText Markup Language 5 (HTML5) pages are being used more and more in business scenarios, and QR code interaction is becoming more and more frequent.

[0003] In existing technologies, mobile barcode recognition is mainly implemented based on native application software (APP), while recognition in the H5 page environment presents many technical challenges. Some solutions use traditional single-frame recognition algorithms, which have poor environmental adaptability. If the barcode is worn or the lighting is complex, it is difficult to recognize, and even if it is recognized, it requires high time and manpower costs. Although some solutions have taken optimization measures, due to the limited computing resources of H5 or the lack of interactive guidance, recognition failures or lags are prone to occur, resulting in low recognition efficiency. Therefore, there is a need for a solution that can improve the accuracy of barcode recognition and the user experience in the H5 environment. Summary of the Invention

[0004] This disclosure provides image recognition methods, electronic devices, and program products based on Hypertext Markup Language.

[0005] According to a first aspect of this disclosure, an image recognition method based on Hypertext Markup Language (HML) is provided. The method specifically includes: acquiring a video stream in a HML page runtime environment and determining candidate video frames that meet image quality standards; registering and weighting multiple candidate video frames based on fusion weights to obtain a fused image; selecting a corresponding image enhancement strategy based on the environmental scene features corresponding to the fused image for optimization processing to obtain an enhanced image; and recognizing the enhanced image to obtain an image recognition result.

[0006] According to at least one embodiment of this disclosure, the method of acquiring a video stream based on a Hypertext Markup Language (HTML) page runtime environment and determining candidate video frames that meet image quality standards includes: acquiring a video stream in an HTML page runtime environment to obtain at least one video frame; evaluating the quality of the current video frame using image quality standards to obtain an evaluation result; generating video acquisition guidance information when the evaluation result indicates that the quality of the current video frame does not meet the image quality standards; and determining candidate video frames when the evaluation result indicates that the quality of the current video frame meets the image quality standards.

[0007] According to at least one embodiment of this disclosure, the quality of the current video frame is evaluated using an image quality standard to obtain an evaluation result, including: calculating the sharpness score, contrast score, illumination score, and stability score of the current video frame; determining the comprehensive quality score of the current video frame based on the sharpness score, contrast score, illumination score, and stability score; and comparing the comprehensive quality score with a preset threshold in the image quality standard to obtain an evaluation result.

[0008] According to at least one embodiment of this disclosure, video stream acquisition based on a Hypertext Markup Language page runtime environment includes: real-time monitoring of the stability score of the current video frame; reducing the frame rate of the acquired video stream when the stability score is higher than a preset jitter threshold; and increasing the frame rate of the acquired video stream when the stability score is lower than the preset jitter threshold.

[0009] According to at least one embodiment of this disclosure, real-time monitoring of the stability score of the current video frame includes: extracting the coordinates of feature points between the current video frame and a reference video frame; calculating the displacement offset of the feature point coordinates; and determining the stability score based on the magnitude of the displacement offset, wherein the larger the displacement offset, the lower the stability score.

[0010] According to at least one embodiment of this disclosure, multiple candidate video frames are registered and weighted based on fusion weights to obtain a fused image, including: when the number of candidate video frames reaches a preset fusion threshold, multiple target video frames are selected from the candidate video frame buffer; feature points of the multiple target video frames are extracted, and inter-frame registration correction is performed on the multiple target video frames based on the feature points; fusion weights are dynamically calculated based on the quality scores corresponding to each target video frame; and pixel mixing is performed on the registered multiple target video frames based on the fusion weights to obtain a fused image.

[0011] According to at least one embodiment of this disclosure, based on the environmental scene features corresponding to the fused image, a corresponding image enhancement strategy is selected for optimization processing to obtain an enhanced image, including: performing environmental feature analysis on the fused image to obtain environmental analysis results; matching a target image enhancement strategy from a preset set of image enhancement strategies based on the environmental analysis results; and using the target image enhancement strategy to perform adaptive enhancement processing on the fused image to obtain an enhanced image.

[0012] According to at least one embodiment of this disclosure, environmental feature analysis is performed on the fused image to obtain environmental analysis results, including: calculating the average brightness of the fused image to determine the illumination level; detecting the noise density of the fused image to determine the noise level; measuring the gradient magnitude of the edges of the fused image to determine the blur level; statistically analyzing the black-and-white contrast intensity of the fused image to determine the contrast level; and using the illumination level, noise level, blur level, and contrast level as environmental analysis results.

[0013] According to at least one embodiment of this disclosure, a target image enhancement strategy is matched from a preset set of image enhancement strategies based on environmental analysis results; adaptive enhancement processing is performed on the fused image using the target image enhancement strategy, including: if the illumination level in the environmental analysis results is lower than a preset illumination threshold, the target image enhancement strategy is determined to be a weak light enhancement mode, and the brightness of the fused image is increased while noise is suppressed; if the illumination level is higher than the preset illumination threshold, the target image enhancement strategy is determined to be a strong light suppression mode, and the overly bright areas of the fused image are suppressed while details are restored; if the blur level in the environmental analysis results is higher than a preset blur threshold, the target image enhancement strategy is determined to be a deblurring mode, and the fused image is sharpened; if both the illumination level and the blur level are within a preset normal range, the target image enhancement strategy is determined to be a standard enhancement mode, and the contrast and tonal balance of the fused image are optimized.

[0014] According to at least one embodiment of this disclosure, the enhanced image is identified to obtain an image recognition result, including: starting a multi-threaded parallel processing architecture for web page operation and creating an idle computing thread; transmitting the data sequence of the enhanced image to the idle computing thread and calling a pre-loaded web page assembly language acceleration module to perform decoding operations to obtain preliminary recognition data; releasing the image cache occupied by the idle computing thread through the memory management system and returning the preliminary recognition data to the main thread as the image recognition result.

[0015] Generating video acquisition guidance information according to at least one embodiment of this disclosure includes: determining the priority of the guidance information based on the specific dimensions of the current video frame that do not meet the image quality standards; adjusting the display color of the scanning frame according to the priority and displaying the corresponding operation prompt text; wherein the display colors include red, yellow and green, corresponding to poor quality, average quality and excellent quality, respectively.

[0016] Before selecting multiple target video frames from the candidate video frame buffer according to at least one embodiment of this disclosure, the method further includes: determining the storage state of the candidate video frame buffer; when the candidate video frame buffer is full, comparing the overall quality score of the current video frame with the lowest quality score in the buffer; if the overall quality score of the current video frame is higher than the lowest quality score, then replacing the candidate video frame with the lowest quality score in the buffer.

[0017] According to at least one embodiment of this disclosure, the enhanced image is identified to obtain an image recognition result, which further includes: if the recognition fails or the image recognition result is invalid, the candidate video frame buffer is cleared; the fusion state is reset, and the step of acquiring the video stream in the hypertext markup language page runtime environment is returned.

[0018] According to a second aspect of this disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, such that the processor performs the method described in the first aspect of any embodiment of this disclosure.

[0019] According to a third aspect of this disclosure, a readable storage medium is provided, wherein executable instructions are stored therein, which, when executed by a processor, are used to implement the method described in the first aspect of any embodiment of this disclosure.

[0020] According to a fourth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect of any embodiment of this disclosure. Attached Figure Description

[0021] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0022] Figure 1 This is a flowchart illustrating the image recognition method based on Hypertext Markup Language provided in an embodiment of this disclosure.

[0023] Figure 2 This is a flowchart illustrating the candidate video frame determination method provided in an embodiment of the present disclosure.

[0024] Figure 3 This is a flowchart illustrating the evaluation result generation method provided in the embodiments of this disclosure.

[0025] Figure 4 This is a flowchart illustrating the video stream acquisition method provided in an embodiment of the present disclosure.

[0026] Figure 5 This is a flowchart illustrating the stability score calculation method provided in an embodiment of the present disclosure.

[0027] Figure 6 This is a schematic flowchart of the image fusion method provided in an embodiment of the present disclosure.

[0028] Figure 7 This is a schematic flowchart of an image enhancement method provided in an embodiment of the present disclosure.

[0029] Figure 8 This is a flowchart illustrating the environmental analysis method provided in an embodiment of the present disclosure.

[0030] Figure 9 This is a schematic flowchart of the enhancement processing method provided in an embodiment of the present disclosure.

[0031] Figure 10 This is a schematic flowchart of the image recognition method provided in an embodiment of the present disclosure.

[0032] Figure 11 This is a schematic diagram illustrating the effect of displaying guidance information as an example of this disclosure.

[0033] Figure 12 This is a schematic block diagram of an H5-based image recognition device according to one embodiment of the present disclosure.

[0034] Figure 13 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure. Detailed Implementation

[0035] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0036] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0037] Figure 1 This is a flowchart illustrating the image recognition method based on Hypertext Markup Language provided in an embodiment of this disclosure. Figure 1 The method shown includes steps S101 to S104. This method can be executed by electronic devices such as computers and mobile phones.

[0038] Specifically, Figure 1 The method shown includes step S101: acquiring video streams based on the Hypertext Markup Language page runtime environment and determining candidate video frames that meet image quality standards.

[0039] It should be noted that the Hypertext Markup Language (HTML) page runtime environment mentioned here refers to the sandbox environment in which web applications built based on the HTML5 standard run in mobile browsers. This is different from the operating system environment of native applications, and its computing resources and hardware access capabilities are limited by browser security policies. The video stream mentioned here refers to a sequence of images continuously captured through a camera interface. The image quality standard mentioned here refers to a quantitative evaluation system used to measure whether video frames are suitable for subsequent recognition and processing, which may include a comprehensive assessment of dimensions such as image sharpness, lighting, and stability. The candidate video frames mentioned here refer to video frames that have been screened from the video stream and meet the image quality standards.

[0040] In practical applications, the camera interface is invoked within the H5 page runtime environment to acquire continuous video streams. Since the H5 page runtime environment cannot directly call the Image Signal Processor (ISP) hardware for image optimization like native applications, and user-held devices are prone to shaking or operating under complex lighting conditions, the success rate of directly recognizing single-frame images is low. Therefore, the system performs real-time quality assessment on the current video frame in the acquired video stream. Based on preset image quality standards, various quality indicators for the current frame are calculated. If the quality indicators of the current frame meet the preset requirements, it is identified as a candidate video frame and stored in a buffer; if it does not meet the requirements, the frame is discarded or a guidance message is generated to prompt the user to adjust. Through this process, the system can filter out high-quality candidate video frames from the continuous video stream, ensuring the quality of the data source for subsequent processing and avoiding the waste of ineffective computing resources.

[0041] Step S102: Register and weighted fuse multiple candidate video frames based on fusion weights to obtain a fused image.

[0042] It should be noted that the fusion weight mentioned here refers to the numerical weight assigned based on the quality of candidate video frames, with higher-quality frames receiving greater weight. Registration refers to the process of aligning multiple candidate video frames to the same coordinate system through geometric transformation to eliminate inter-frame displacement. Weighted fusion refers to the mixing of pixel information from the registered multiple candidate video frames according to the fusion weight. The fused image refers to a single high-quality image generated after registration and weighted fusion, whose signal-to-noise ratio and sharpness are superior to the original single-frame image.

[0043] In practical applications, slight movement of the user's phone during shooting can cause image ghosting when directly fused. Therefore, the system first registers multiple candidate video frames, extracts feature points, and aligns each frame to the same reference position. Then, the system calculates the fusion weight based on the quality assessment results of each candidate video frame, assigning greater weight to higher-quality video frames. Based on the fusion weight, the system performs pixel-level weighted fusion on the registered multi-frame candidate video frames. Utilizing temporal information, the system cancels out random noise in a single frame and enhances the effective signal through multi-frame superposition, resulting in a fused image that is clearer and less noisy than any single frame. This further improves the quality and effect of single-frame imaging in the H5 environment.

[0044] Step S103: Based on the environmental scene features corresponding to the fused image, select the corresponding image enhancement strategy for optimization processing to obtain the enhanced image.

[0045] It should be noted that the environmental scene features mentioned here refer to parameters reflecting the environmental conditions in which the fused image was captured, such as illumination level, noise density, and blur level. The image enhancement strategy mentioned here refers to a combination of image processing algorithms pre-set for specific environmental scene features, used to improve image quality. The enhanced image mentioned here refers to an image optimized by the image enhancement strategy, making it more suitable for decoding and recognition.

[0046] In practical applications, after obtaining the fused image, the system further analyzes the environmental scene features corresponding to the fused image. The system calculates parameters such as the average brightness, noise density, and edge sharpness of the fused image to determine whether the current shooting environment is in low light, strong light, blurry, or normal conditions. Based on the analyzed environmental scene features, the system matches the most suitable target image enhancement strategy from a preset set of image enhancement strategies. For example, if the environmental scene features indicate low light, a strategy to increase brightness and suppress noise is selected; if it indicates overexposure, a strategy to suppress highlights and restore details is selected. The system uses the matched image enhancement strategy to adaptively optimize the fused image, ultimately obtaining the enhanced image. The specific strategy selection and implementation methods will be described in detail in subsequent embodiments and will not be repeated here. This ensures that the image processing algorithm can dynamically adapt to changing environmental conditions, rather than using fixed parameters, significantly improving image readability.

[0047] Step S104: Recognize the enhanced image to obtain the image recognition result.

[0048] It should be noted that the image recognition result mentioned here refers to the barcode information or data content decoded from the enhanced image. For ease of understanding, the following embodiments will use the recognition of barcodes (such as one-dimensional barcodes and two-dimensional barcodes) in images as an example for explanation.

[0049] In practical applications, the system decodes and recognizes the enhanced image after the aforementioned optimization steps. Because the enhanced image has undergone quality screening, multi-frame fusion noise reduction, and environmental adaptive enhancement, its barcode lines are clear and the contrast is suitable, greatly reducing the decoding difficulty. The system calls the decoding algorithm to process the enhanced image, extract the barcode information, and finally obtain the image recognition result. If the recognition is successful, the result is returned; if the recognition fails, the process can be reset and the data collected again.

[0050] Based on the publicly available solutions, multi-frame filtering and fusion within the H5 page runtime environment effectively compensates for the limitation of H5 environments in directly accessing underlying hardware optimization capabilities, significantly improving the quality of input images. Furthermore, by employing registration and weighted fusion based on fusion weights, the temporal information of the video stream is fully utilized, greatly improving tolerance to worn barcodes and noisy images compared to single-frame recognition schemes. An adaptive enhancement strategy based on environmental scene features addresses the pain point of low recognition rates in harsh environments such as low light and strong light, enhancing the solution's environmental adaptability. In barcode recognition applications within mobile H5 page scenarios, user experience is optimized while maintaining recognition accuracy.

[0051] In one or more embodiments of this disclosure, such as Figure 2 This is a flowchart illustrating the candidate video frame determination method provided in an embodiment of this disclosure. Figure 2 As shown, step S101, which involves acquiring a video stream in a Hypertext Markup Language (HTML) page runtime environment and determining candidate video frames that meet image quality standards, includes: Step S201: Acquiring a video stream in the HTML page runtime environment to obtain at least one video frame. Step S202: Evaluating the quality of the current video frame using image quality standards to obtain an evaluation result. Step S203: When the evaluation result indicates that the quality of the current video frame does not meet the image quality standards, generating video acquisition guidance information. Step S204: When the evaluation result indicates that the quality of the current video frame meets the image quality standards, determining candidate video frames.

[0052] It should be noted that the current video frame referred to here is a single frame of image data captured in real time from the video stream and awaiting processing. The image quality standard refers to a preset set of quality thresholds or rules used to determine the usability of a video frame, quantitatively measuring whether the video frame can be used for subsequent recognition. The evaluation result refers to the conclusion drawn after quality evaluation, indicating whether the image meets or does not meet the aforementioned image quality standard.

[0053] In practical applications, the system uses the media interface provided by the browser to call the camera hardware and establish a video stream acquisition channel within the H5 page runtime environment. The system continuously extracts image data from this video stream, acquiring at least one video frame for processing. Because the H5 page runtime environment differs from native applications, its video stream acquisition process is managed by browser security policies. Therefore, the system must ensure that it acquires a continuous sequence of video frames with user authorization to provide a data foundation for subsequent quality assessment.

[0054] Furthermore, the current video frame is selected from at least one acquired video frame and compared with a preset image quality standard. The image quality standard includes threshold values ​​for various indicators used to measure image usability. The system calculates various quality parameters of the current video frame using an algorithm and compares these parameters with the thresholds in the standard to obtain an evaluation result. The evaluation result is specifically expressed as a binary or graded determination of whether the quality of the current video frame meets the image quality standard. This provides a basis for subsequent branch processing, ensuring that only video frames that meet the basic quality requirements can proceed to the next stage.

[0055] When the evaluation result indicates that the quality of the current video frame does not meet the image quality standards, the system executes the step of generating video capture guidance information. At this point, the system determines that the current video frame is unusable, possibly due to improper user operation or poor environmental conditions. The system will generate corresponding video capture guidance information based on the specific reason for the non-compliance. For example, if the non-compliance is due to insufficient lighting, a message will be generated prompting the user to move to a brighter area; if the non-compliance is due to camera shake, a message will be generated prompting the user to maintain stability. This video capture guidance information will be displayed in real-time on the H5 page, helping users quickly adjust their shooting state and thereby increasing the probability of capturing qualified video frames subsequently.

[0056] When the evaluation result indicates that the quality of the current video frame meets the image quality standard, the step of determining the candidate video frames is executed. At this point, the system determines that the current video frame is of acceptable quality and suitable for subsequent image fusion and recognition. The system marks the current video frame as a candidate video frame and stores it in a specific buffer or queue, awaiting invocation in subsequent steps. Through this filtering mechanism, the system can effectively eliminate invalid frames that are blurry, overexposed, or severely jittery, ensuring that the data sources entering subsequent fusion steps are all high-quality frames.

[0057] Based on the publicly available solutions described above, by implementing real-time quality assessment within the H5 page runtime environment, quality control can be achieved at the data acquisition source, preventing invalid data from entering subsequent calculation stages and saving computing resources in the H5 environment. Furthermore, by generating video acquisition guidance information, a real-time interactive feedback mechanism between the user and the system is established, overcoming the limitation of the H5 environment's inability to directly control hardware focus or flash. Software guidance helps users proactively improve shooting conditions, significantly increasing the success rate of user operations.

[0058] In one or more embodiments of this disclosure, such as Figure 3 This is a flowchart illustrating the evaluation result generation method provided in an embodiment of this disclosure. Figure 3As shown, the current video frame is evaluated using image quality standards to obtain evaluation results, including: Step S301: Calculate the sharpness score, contrast score, illumination score, and stability score of the current video frame. Step S302: Determine the comprehensive quality score of the current video frame based on the sharpness score, contrast score, illumination score, and stability score. Step S303: Compare the comprehensive quality score with a preset threshold in the image quality standards to obtain the evaluation result.

[0059] It should be noted that the sharpness score mentioned here refers to a numerical value used to quantify the degree of blur in the current video frame image; a higher value indicates a sharper image. The contrast score refers to a numerical value used to quantify the degree of black-and-white contrast in the current video frame; a higher value indicates a more pronounced contrast. The lighting score refers to a numerical value used to quantify the uniformity and appropriateness of light in the current video frame; a higher value indicates better lighting conditions. The stability score refers to a numerical value used to quantify the stability of the barcode's geometric shape in the current video frame; a higher value indicates less barcode deformation or tilt. The overall quality score is a numerical value calculated based on the sharpness score, contrast score, lighting score, and stability score, used to comprehensively evaluate the overall quality of the current video frame.

[0060] In practical applications, the system runs in an H5 page environment and performs multi-dimensional analysis of the current video frame using image processing algorithms. For sharpness score, the system calculates it by detecting the sharpness of image edges or the energy of high-frequency components; blurry images receive a lower score. For contrast score, the system calculates it by statistically analyzing the dispersion of image pixel grayscale distribution or histogram span; clearly defined black and white barcodes receive a higher score. For illumination score, the system calculates it by measuring the overall brightness and local brightness differences of the image; excessively dark, overexposed, or unevenly lit images receive a lower score. For stability score, the system calculates it by analyzing the degree of geometric deformation in the barcode area or inter-frame displacement; severe jitter or tilting results in a lower score. These four scores reflect the usability of the current video frame from different dimensions, providing a data foundation for subsequent comprehensive evaluation.

[0061] After obtaining the scores from the four dimensions mentioned above, they are combined into a single comprehensive quality score using methods such as weighted calculation or logistic regression. For example, the system can assign a corresponding weight coefficient to each score, reflecting the degree of influence of that dimension on the final recognition success rate. The comprehensive quality score is obtained by multiplying each score by its weight coefficient and then summing the results. Transforming multi-dimensional quality indicators into a quantifiable comprehensive indicator simplifies the subsequent judgment logic and makes the quality assessment results more intuitive and consistent.

[0062] The system then reads a preset threshold from the image quality standard. This preset threshold is the minimum quality score determined based on a large amount of experimental data to guarantee a high recognition success rate. The system compares the calculated overall quality score with this preset threshold. If the overall quality score is greater than or equal to the preset threshold, the current video frame is determined to meet the image quality standard, and the evaluation result is "compliant"; if the overall quality score is less than the preset threshold, the current video frame is determined to not meet the image quality standard, and the evaluation result is "incompatible".

[0063] Based on the publicly available solutions described above, by establishing a four-dimensional quality assessment system encompassing sharpness, contrast, illumination, and stability, the limitations of single-indicator evaluations are avoided, allowing for a comprehensive and accurate reflection of the actual quality of the current video frame. Furthermore, by calculating a comprehensive quality score, complex multi-dimensional indicators are simplified into a unified quantitative standard, reducing the complexity of the judgment logic, improving processing efficiency, and better meeting the application limitations imposed by the limited performance conditions of the H5 page operating environment. The evaluation results are obtained by comparing with preset thresholds, achieving automation and standardization of quality judgment, ensuring that only high-quality video frames are identified as candidate video frames, thus guaranteeing the accuracy of subsequent image fusion and recognition from the source.

[0064] In one or more embodiments of this disclosure, such as Figure 4 This is a flowchart illustrating the video stream acquisition method provided in an embodiment of this disclosure. Figure 4 As shown, the video stream acquisition based on the Hypertext Markup Language (HTML) page runtime environment includes: Step S401: Real-time monitoring of the stability score of the current video frame. Step S402: When the stability score is higher than a preset jitter threshold, reduce the frame rate of the acquired video stream. Step S403: When the stability score is lower than the preset jitter threshold, increase the frame rate of the acquired video stream.

[0065] It should be noted that the stability score mentioned here refers to a numerical value used to quantify the stability of the barcode's geometric shape in the current video frame. A higher value indicates less barcode deformation or tilt, meaning a more stable phone grip. The preset jitter threshold is a pre-set critical value used for comparison with the stability score, distinguishing whether the current video frame is in a stable or jittery state. The frame rate refers to the number of video frames captured per unit of time; a higher frame rate indicates a higher capture frequency.

[0066] In practical applications, within the H5 page runtime environment, stability analysis is continuously performed on each current video frame as the video stream is acquired. By calculating the displacement offset of feature points between the current video frame and the reference frame, or analyzing the degree of geometric deformation in the barcode area, the stability score of the current video frame is calculated in real time. This monitoring process is dynamic and continuous, ensuring that the system can instantly perceive changes in the stability of the user's handheld device, providing real-time data support for subsequent adjustments to the acquisition strategy.

[0067] When the stability score exceeds a preset jitter threshold, the system reduces the frame rate of the captured video stream. A stability score above the threshold indicates that the current video frame is relatively stable, the user is holding the device steadily, and the captured image quality is high. In this situation, reducing the frame rate of the captured video stream serves two purposes: firstly, it reduces the generation of invalid or low-quality video frames, preventing the buffer from being filled with inferior frames; secondly, it reduces the computational load and memory consumption in the H5 page runtime environment, allowing resources to be used more for generating video capture guidance information to prompt the user to adjust their posture, thus implementing a resource optimization and interaction-first strategy.

[0068] When the stability score is lower than a preset jitter threshold, it indicates that the current video frame is jittery, the user's hand is unstable, and the captured image may be blurry or distorted. In this case, the system increases the frame rate of the acquired video stream, enabling the acquisition of more high-quality video frames per unit time. These high-quality frames can serve as rich data sources for subsequent multi-frame fusion, thereby further improving the signal-to-noise ratio and detail of the fused image, achieving a quality-first acquisition strategy.

[0069] Based on the publicly available solutions described above, by monitoring stability scores in real time, the system can perceive subtle changes in user operation states, achieving dynamic matching between the acquisition strategy and user behavior. Secondly, by adjusting the frame rate based on the comparison between the stability score and a preset jitter threshold, differentiated acquisition is achieved: increasing the frame rate in jittery conditions to pursue higher recognition quality, and decreasing the frame rate in stable conditions to save resources and avoid the accumulation of invalid data. This adaptive mechanism effectively balances the contradiction between image recognition quality and system resource consumption, improving the overall operating efficiency and user experience in the H5 page environment, making the video acquisition process more intelligent and efficient.

[0070] In one or more embodiments of this disclosure, such as Figure 5 This is a flowchart illustrating the stability score calculation method provided in an embodiment of this disclosure. Figure 5As shown, the real-time monitoring of the stability score of the current video frame includes: Step S501: Extracting the coordinates of feature points between the current video frame and the reference video frame. Step S502: Calculating the displacement offset of the feature point coordinates. Step S503: Determining the stability score based on the magnitude of the displacement offset, where a larger displacement offset results in a lower stability score.

[0071] It should be noted that the reference video frame mentioned here refers to the baseline video frame used for comparative analysis with the current video frame. This frame can be the previous frame or a pre-defined stable frame. The feature point coordinates mentioned here refer to the positional information of points with significant features in the image within the image coordinate system, such as corner points of a barcode, edge keypoints, or texture feature points. The displacement offset mentioned here refers to the distance or vector magnitude of the positional change of the same feature point between the reference video frame and the current video frame, used to quantify the amplitude of motion between images.

[0072] In practical applications, within the H5 page runtime environment, feature detection is performed on both the captured current video frame and the selected reference video frame. The system utilizes image processing algorithms to identify highly discriminative feature points in the image, such as the four corner points of a barcode or key points on the edge contour, and records the specific positions of these feature points in the image coordinate system, i.e., feature point coordinates. By extracting the corresponding feature point coordinates between two frames, the system establishes a benchmark for inter-frame comparison, providing data support for calculating motion amplitude.

[0073] Next, the system calculates the displacement offset of the feature point coordinates. It compares the feature point coordinates in the current video frame with the corresponding feature point coordinates in the reference video frame, calculating the positional difference between the two. This displacement offset can be obtained by calculating the Euclidean distance or the magnitude of the vector difference between the corresponding feature point coordinates. If there is no relative motion between the current video frame and the reference video frame, the displacement offset is close to zero; if the user's handheld device shakes or moves, the feature point position changes, and the displacement offset increases accordingly. By converting the visual changes between images into quantifiable numerical indicators, the system accurately reflects the degree of motion between frames.

[0074] Next, the stability score is determined based on the magnitude of the displacement offset, where a larger displacement offset results in a lower stability score. The mapping relationship between the displacement offset and the stability score can be established using an inverse proportional relationship or a piecewise function. When the calculated displacement offset is large, it indicates that the current video frame has undergone significant displacement relative to the reference video frame, indicating user instability, and the system assigns a lower stability score. Conversely, when the displacement offset is small, it indicates that the current video frame is relatively stable, and the system assigns a higher stability score. This method transforms the physical jitter caused by the user into a logical quality score, which can then be used to help users address the issue of low image quality caused by jitter through frame rate adjustment. In the technical implementation, this stability score is directly used for frame rate adjustment.

[0075] Based on the publicly available solutions described above, extracting feature point coordinates and calculating displacement offsets provides an objective and quantitative method for stability detection, avoiding the inaccuracies of subjective judgment. Furthermore, by establishing an inverse correlation between displacement offsets and stability scores, the stability score accurately reflects the stability of the user's device holding position; the larger the displacement, the lower the score. The adaptive frame rate adjustment scheme ensures that the video capture strategy can be dynamically optimized based on actual jitter.

[0076] In one or more embodiments of this disclosure, such as Figure 6 This is a schematic flowchart illustrating the image fusion method provided in an embodiment of this disclosure. Figure 6 As shown, multiple candidate video frames are registered and weighted based on fusion weights to obtain a fused image, including: Step S601: When the number of candidate video frames reaches a preset fusion threshold, multiple target video frames are selected from the candidate video frame buffer. Step S602: Feature points of the multiple target video frames are extracted, and inter-frame registration correction is performed on the multiple target video frames based on the feature points. Step S603: Fusion weights are dynamically calculated based on the quality scores corresponding to each target video frame. Step S604: Pixel blending is performed on the registered multiple target video frames based on the fusion weights to obtain a fused image.

[0077] It should be noted that the preset fusion threshold mentioned here refers to a pre-set critical value for the number of candidate video frames used to trigger multi-frame fusion processing. The candidate video frame buffer refers to a memory area in the H5 page runtime environment used to temporarily store candidate video frames that meet image quality standards. Inter-frame registration correction refers to the technical process of aligning multiple target video frames to the same coordinate system through geometric transformation. The quality score refers to a numerical value characterizing the quality of a video frame, which can be calculated based on dimensions such as sharpness, contrast, lighting, and stability. The fusion weight refers to a value allocated according to the quality score to control the contribution ratio of each frame in the fusion process. Pixel blending refers to the process of merging pixel values ​​at corresponding positions in multiple frames according to the fusion weight through mathematical operations. The fused image refers to a single high-quality image generated after registration and weighted fusion.

[0078] In practical applications, the amount of data in the candidate video frame buffer is monitored in real time within the H5 page runtime environment. As video acquisition progresses, candidate video frames that meet the image quality standards are continuously added to the buffer. When the number of candidate video frames in the buffer accumulates to a preset fusion threshold, it indicates that a sufficient number of high-quality frames have been collected for fusion processing. At this point, the system selects multiple target video frames from the buffer. The selection strategy can be to select the earliest added frame, the most recently added frame, or the frame with the highest quality score. This ensures that the fusion processing is neither ineffective due to too few frames nor affected by excessive waiting time, thus ensuring the real-time fusion effect is not compromised.

[0079] Further registration and correction are performed. Specifically, feature detection is conducted on each of the selected target video frames, extracting feature points from each frame. Since slight displacement of the device during shooting is inevitable, positional deviations occur between image content frames. The system calculates the inter-frame transformation matrix by matching corresponding feature points across multiple target video frames, and performs geometric correction on each frame based on this transformation matrix, aligning the barcode areas in all target video frames to the same reference position. This eliminates the ghosting risk caused by inter-frame displacement, laying the foundation for subsequent pixel-level precise fusion.

[0080] The system reads the quality score obtained from the previous quality assessment stage for each target video frame. A higher quality score indicates a clearer, more stable, and appropriately lit image. The system calculates fusion weights based on the quality scores of each frame; typically, frames with high quality scores are assigned larger fusion weights, while frames with low quality scores are assigned smaller weights. This dynamic calculation mechanism ensures that high-quality image information dominates the final result, while low-quality image information serves only as a supplement, thereby maximizing the fusion effect.

[0081] The registered target video frames are superimposed at the pixel level. For each pixel in the image, the system performs a weighted average or weighted summation operation based on the pixel value at that location in each frame and its fusion weight. Through this pixel mixing, random noise is canceled out during the multi-frame superposition process, while the effective image signal is enhanced. The final fused image has a higher signal-to-noise ratio and sharper clarity. Compared to the original single-frame image, its barcode edges are sharper and its details are richer, significantly reducing the difficulty of subsequent recognition steps.

[0082] Based on the publicly available solutions, by setting a preset fusion threshold to control the fusion timing, image processing quality and system response speed are balanced, avoiding delays caused by waiting for too many frames. Furthermore, through feature point-based inter-frame registration correction, the inter-frame misalignment problem caused by handheld shooting is effectively solved, preventing blurring or ghosting in the fused image. By dynamically calculating the fusion weight based on the quality score, intelligent information filtering is achieved, ensuring that high-quality image data maximizes its contribution to the final result and improving the reliability of the fused image. Generating the fused image through pixel blending utilizes algorithms to compensate for the limitation of hardware multi-frame noise reduction in the H5 page operating environment, significantly improving image quality and providing a foundation for high-precision recognition in complex environments.

[0083] In one or more embodiments of this disclosure, such as Figure 7 This is a schematic flowchart illustrating the image enhancement method provided in an embodiment of this disclosure. Figure 7 As shown, based on the environmental scene features corresponding to the fused image, a corresponding image enhancement strategy is selected for optimization processing to obtain an enhanced image, including: Step S701: Perform environmental feature analysis on the fused image to obtain environmental analysis results. Step S702: Match a target image enhancement strategy from a preset set of image enhancement strategies according to the environmental analysis results. Step S703: Perform adaptive enhancement processing on the fused image using the target image enhancement strategy to obtain an enhanced image.

[0084] It should be noted that the environmental scene features mentioned here refer to a set of parameters reflecting the environmental conditions in which the fused image was captured, including illumination level, noise density, blur level, and contrast statistics. The environmental analysis results mentioned here refer to the specific parameter values ​​or status indicators output after environmental feature analysis, used to characterize the environmental scene in which the current image exists. The preset image enhancement strategy set mentioned here refers to a database that pre-stores various combinations of image enhancement algorithms, each corresponding to a specific environmental scene. The target image enhancement strategy mentioned here refers to a specific combination of image enhancement algorithms matched from the preset image enhancement strategy set and suitable for the current fused image. The enhanced image mentioned here refers to an image optimized through adaptive enhancement processing, whose quality is more suitable for image recognition.

[0085] In practical applications, within the H5 page runtime environment, the system receives the fused image generated in the preceding steps and initiates an environmental feature analysis program. The system calculates various environmental indicators of the fused image using image processing algorithms. These include calculating the average pixel grayscale value to assess illumination levels, statistically analyzing high-frequency noise components to assess noise density, measuring edge gradients to assess blurriness, and calculating histogram distribution to assess contrast. The system then summarizes these calculated indicators to form the environmental analysis results. These results objectively quantify the shooting environment of the fused image, providing data for subsequent strategy selection and ensuring that the enhancement processing is targeted and effective.

[0086] Next, the system performs a step of matching a target image enhancement strategy from a pre-defined set of image enhancement strategies based on the environmental analysis results. The system accesses a pre-defined set of image enhancement strategies stored locally or in the cloud. This set contains various enhancement strategies preset for different environmental scenarios. The system compares the environmental analysis results obtained in the previous step with the applicable conditions of each strategy in the strategy set. For example, if the environmental analysis results indicate low illumination levels, a strategy suitable for low-light environments is matched; if they indicate high noise density, a strategy suitable for high-noise environments is matched. Through this matching mechanism, the system selects the target image enhancement strategy from the set that best matches the characteristics of the current environmental scene.

[0087] The system loads the selected target image enhancement strategy and applies it to the fused image. During processing, the system dynamically adjusts the processing parameters in the strategy based on the specific values ​​of the environmental analysis results, achieving adaptive enhancement. For example, if the environmental analysis results indicate extremely low lighting, the brightness enhancement is automatically increased; if the image is slightly blurry, the sharpening intensity is automatically adjusted. After this processing, the system outputs the enhanced image. The enhanced image outperforms the original fused image in terms of sharpness, contrast, and noise control, significantly improving image readability and creating favorable conditions for subsequent recognition steps.

[0088] Based on the publicly available solution, environmental analysis results are obtained by performing environmental feature analysis on the fused image, achieving accurate perception of the shooting environment. Furthermore, by matching a target image enhancement strategy from a pre-set set of image enhancement strategies according to the environmental analysis results, on-demand allocation of processing strategies is achieved, ensuring targeted optimization for each environmental scene. Adaptive enhancement processing using the target image enhancement strategy not only selects the correct strategy but also dynamically adjusts the processing intensity based on the severity of the environment, avoiding over-processing or under-processing. The enhanced image quality generated in this step is significantly improved, effectively overcoming the image quality bottleneck caused by hardware limitations in the H5 page operating environment, and greatly improving the success rate and accuracy of subsequent image recognition.

[0089] In one or more embodiments of this disclosure, such as Figure 8 This is a flowchart illustrating the environmental analysis method provided in an embodiment of this disclosure. Figure 8 As shown, environmental feature analysis is performed on the fused image to obtain environmental analysis results, including: Step S801: Calculate the average brightness of the fused image to determine the illumination level. Step S802: Detect the noise density of the fused image to determine the noise level. Step S803: Measure the gradient magnitude of the edges of the fused image to determine the blur level. Step S804: Statistically analyze the black-and-white contrast intensity of the fused image to determine the contrast level. Step S805: Use the illumination level, noise level, blur level, and contrast level as the environmental analysis results.

[0090] It should be noted that the average brightness mentioned here refers to the arithmetic mean of the brightness values ​​of all pixels in the fused image, used to reflect the overall brightness of the image. The illumination level mentioned here refers to the lighting condition level based on the average brightness, such as low light, normal light, or strong light. The noise density mentioned here refers to the number or proportion of noisy pixels per unit area in the fused image. The noise level mentioned here refers to the severity of noise quantized based on noise density. The blur level mentioned here refers to the image blur state quantized based on edge sharpness; the lower the sharpness, the higher the blur level. The black-and-white contrast intensity mentioned here refers to the statistical value of the brightness difference between black and white areas in the fused image, used to reflect the distinguishability of the barcode from the background.

[0091] In practical applications, the system runs within an H5 page environment, traversing all pixels of the merged image, acquiring the brightness value of each pixel, and calculating the arithmetic mean of all brightness values ​​to obtain the average brightness. Subsequently, the system compares the calculated average brightness with a preset brightness range to determine the illumination level. For example, if the average brightness is below a certain low threshold, the illumination level is determined to be weak; if it is above a certain high threshold, the illumination level is determined to be strong; if it falls between the two, the illumination level is determined to be normal. By quantifying the overall illumination of the image, a basis is provided for subsequent selection of brightness adjustment strategies.

[0092] The system analyzes the pixel distribution of the fused image using methods such as high-frequency filtering or statistical variance to identify noise pixels that deviate from the normal texture and calculates the number of noise pixels per unit area to obtain the noise density. The system determines the noise level based on the noise density; higher noise density indicates a higher noise level. This step assesses the image's purity, helping to determine whether and how much noise reduction is needed in subsequent processing.

[0093] Edge detection operators (such as the Sobel operator and the Laplacian operator) are used to calculate the gradient magnitude of edge regions in the fused image. A larger gradient magnitude indicates a sharper edge. The system uses the statistical results of the gradient magnitude as the edge sharpness and determines the blur level inversely based on the sharpness; that is, the lower the sharpness, the higher the determined blur level. This quantifies the image's clarity, providing a basis for determining whether subsequent sharpening processing is needed.

[0094] The grayscale histogram of the fused image is analyzed to calculate the grayscale difference between dark and bright areas, or the standard deviation of the image, to obtain the black-and-white contrast intensity. The system determines the contrast level based on the magnitude of the black-and-white contrast intensity; the lower the contrast intensity, the lower the determined contrast level. This step evaluates the distinguishability of the barcode from the background and is crucial for barcode recognition.

[0095] The illumination level, noise level, blur degree, and contrast level determined in the four steps above are summarized to form a complete environmental analysis result. This environmental analysis result comprehensively describes the environmental scene characteristics of the fused image and will serve as direct input data for matching target image enhancement strategies.

[0096] Based on the aforementioned publicly available solutions, a comprehensive quantitative evaluation of image environmental features is achieved through analysis across four dimensions: average brightness, noise density, edge sharpness, and black-and-white contrast intensity, avoiding the limitations of single-index evaluation. Furthermore, by converting specific physical quantities (such as brightness values ​​and gradient magnitudes) into state indicators like level or degree (such as illumination level and blur level), the logical complexity of subsequent strategy matching is simplified, improving processing efficiency. Moreover, the obtained environmental analysis results directly reflect the key environmental factors affecting barcode recognition, enabling the subsequently selected image enhancement strategies to accurately target specific defects in the current image (such as excessive darkness, excessive noise, blurriness, or low contrast), significantly improving the targeting and effectiveness of the enhancement process.

[0097] In one or more embodiments of this disclosure, such as Figure 9 This is a schematic flowchart illustrating the enhancement processing method provided in an embodiment of this disclosure. Figure 9As shown, based on the environmental analysis results, a target image enhancement strategy is matched from a preset set of image enhancement strategies. The target image enhancement strategy is then used to adaptively enhance the fused image, including: Step S901: If the illumination level in the environmental analysis results is lower than a preset illumination threshold, the target image enhancement strategy is determined to be a weak light enhancement mode, increasing the brightness of the fused image and suppressing noise. Step S902: If the illumination level is higher than a preset illumination threshold, the target image enhancement strategy is determined to be a strong light suppression mode, suppressing overly bright areas of the fused image and restoring details. Step S903: If the blur level in the environmental analysis results is higher than a preset blur threshold, the target image enhancement strategy is determined to be a deblurring mode, and the fused image is sharpened. Step S904: If both the illumination level and blur level are within a preset normal range, the target image enhancement strategy is determined to be a standard enhancement mode, optimizing the contrast and balancing the tone of the fused image.

[0098] It should be noted that the target image enhancement strategy mentioned here refers to a specific image enhancement mode matched from a set of preset image enhancement strategies and applicable to the current fused image. The preset illumination threshold refers to a critical brightness value used to determine whether the illumination level is too low or too high. The low-light enhancement mode refers to a preset image enhancement strategy for low-light environments. The strong light suppression mode refers to a preset image enhancement strategy for high-light environments. The preset blur threshold refers to a critical value used to determine whether the blur level is too high.

[0099] In practical applications, the system acquires the fused image generated in the preceding steps, along with the corresponding environmental analysis results, and reads a preset set of image enhancement strategies. The system then begins conditional judgments based on specific values ​​in the environmental analysis results to match the target image enhancement strategy. Specifically, it first checks the illumination level in the environmental analysis results and compares it with a preset illumination threshold. If the illumination level in the environmental analysis results is lower than the preset illumination threshold, the system determines the target image enhancement strategy to be a low-light enhancement mode. This indicates that the current fused image is in a low-light environment, and the system will perform operations to increase the brightness of the fused image while suppressing noise. The system improves the overall brightness of the image through a gain algorithm, while simultaneously using a noise reduction algorithm to reduce particle noise amplified by the increased brightness, thereby improving image visibility in low light.

[0100] If the illumination level exceeds a preset illumination threshold, the system determines the target image enhancement strategy to be strong light suppression mode. This indicates that the current fused image is in an environment with excessively strong or overexposed light, and the system immediately performs operations to suppress the overly bright areas of the fused image and restore details. Specifically, the system reduces the brightness of overly bright areas using a highlight suppression algorithm and recovers texture information lost due to overexposure using a detail restoration algorithm, preventing barcodes from becoming unreadable due to reflections or excessive brightness.

[0101] If the blur level in the environmental analysis results exceeds a preset blur threshold, the system determines the target image enhancement strategy to be deblurring mode. This indicates that the current fused image has significant motion blur or is out of focus, and the system immediately performs a sharpening operation on the fused image. Specifically, the system uses sharpening filters or deconvolution algorithms to enhance the gradient of image edges, making barcode lines clearer and reducing the impact of blur on decoding.

[0102] If the illumination level and blur level are both within a preset normal range, the system determines the target image enhancement strategy as the standard enhancement mode. At this point, it indicates that the environmental conditions of the current fused image are good, and extreme enhancement processing is unnecessary. The system then performs operations to optimize the contrast and balance the tones of the fused image. Specifically, the system uses histogram equalization or tone mapping algorithms to make the black-and-white contrast of the image more distinct and the color or grayscale distribution more uniform, further improving the compatibility of the recognition algorithm.

[0103] Finally, the system adaptively enhances the fused image using a determined target image enhancement strategy to obtain an enhanced image. The system calls the corresponding algorithm module based on the matched mode (weak light enhancement mode, strong light suppression mode, deblurring mode, or standard enhancement mode), and dynamically adjusts the algorithm parameters based on specific numerical values ​​from the environmental analysis results. After processing, the enhanced image is output. This enhanced image will serve as input data for subsequent recognition steps.

[0104] Based on the publicly available solutions described above, by matching the target image enhancement strategy from a pre-set set of image enhancement strategies according to the environmental analysis results, precise matching of processing strategies is achieved. This ensures that each environmental scenario receives the most suitable algorithm processing, avoiding the limitations of a single strategy. Furthermore, by setting pre-set illumination and blur thresholds, a quantitative judgment standard is established, making the strategy switching logic clear and highly automated. Specific enhancement operations are performed for different matching results (such as brightening and noise reduction in low light, suppression and restoration in strong light, sharpening in blurred conditions, and contrast optimization in normal conditions), achieving targeted repair of image defects and significantly improving image quality.

[0105] In one or more embodiments of this disclosure, such as Figure 10 This is a schematic flowchart illustrating the image recognition method provided in an embodiment of this disclosure. Figure 10As shown, step S104, which involves recognizing the enhanced image to obtain the image recognition result, includes: Step S1001: Starting a multi-threaded parallel processing architecture for web page workers (e.g., Web Workers) and creating an idle computing thread. Step S1002: Transmitting the data sequence of the enhanced image to the idle computing thread and calling a pre-loaded web page assembly language (e.g., WebAssembly) acceleration module to perform decoding operations to obtain preliminary recognition data. Step S1003: Releasing the image cache occupied by the idle computing thread through the intelligent memory management system and returning the preliminary recognition data to the main thread as the image recognition result.

[0106] It should be noted that the multi-threaded parallel processing architecture of web page workers (e.g., Web Workers; for ease of understanding, Web Workers will be used as an example in subsequent embodiments) refers to a technical architecture based on the HTML5 standard that allows scripts to run in background threads without interfering with the user interface. The idle computing thread mentioned here refers to a worker thread in this architecture that is not currently undertaking computing tasks and can be used to assign new tasks. The data sequence of the enhanced image mentioned here refers to the data stream that converts the enhanced image into a binary or array format suitable for transmission between threads.

[0107] The pre-loaded web page assembly language (e.g., WebAssembly; for ease of understanding, WebAssembly will be used as an example in subsequent embodiments) acceleration module mentioned here refers to a code module pre-compiled into binary format and loaded into the browser environment to perform calculations at near-native speed. The decoding operation mentioned here refers to the computational process of analyzing image data to extract barcode information. The preliminary recognition data refers to the raw recognition result data obtained through the decoding operation. The memory management system mentioned here refers to the mechanism used to actively manage memory allocation and release in the H5 page runtime environment.

[0108] In practical applications, because the H5 page runtime environment runs JavaScript code on a single thread by default, complex image recognition calculations can easily block the main thread, causing page lag. To solve this problem, the system initializes a Web Workers multi-threaded parallel processing architecture in the H5 page runtime environment and creates one or more background worker threads within this architecture. The system monitors the status of these threads and selects threads that are not currently undertaking computational tasks as idle computation threads. An independent execution environment is prepared for subsequent high-load computational tasks, ensuring that the computation process does not interfere with the responsiveness of the user interface.

[0109] Furthermore, the enhanced image generated in the previous steps is converted into an enhanced image data sequence, which is then transmitted to the created idle computing thread via an inter-thread communication mechanism. In the idle computing thread, the system invokes the pre-loaded WebAssembly acceleration module, which is pre-loaded into the browser environment. Since WebAssembly is a low-level binary instruction format, its execution efficiency is far higher than traditional JavaScript code. The system utilizes this module to perform high-density decoding operations on the received enhanced image data sequence. After the decoding operation is completed, the idle computing thread generates preliminary recognition data. By leveraging the computational performance advantages of WebAssembly, the decoding time is significantly shortened, while the idle computing thread avoids blocking the main thread.

[0110] After obtaining the initial recognition data, idle computing threads still retain temporary data such as enhanced image data sequences, occupying image cache. To prevent memory leaks common in H5 page runtime environments, the system proactively detects and releases the image cache occupied by idle computing threads through an intelligent memory management system, reclaiming memory resources. Subsequently, the system returns the processed initial recognition data to the main thread responsible for interface rendering via a communication mechanism. Upon receiving the initial recognition data, the main thread formats it and outputs it as the final image recognition result to the user or upstream business systems. This ensures the effective reclamation of memory resources.

[0111] Based on the publicly available solutions described above, by launching the WebWorkers multi-threaded parallel processing architecture and creating idle computing threads, the separation of computing tasks and UI rendering tasks is achieved. This solves the problem of page unresponsiveness or lag that easily occurs when H5 pages perform complex image recognition, significantly improving the user experience. Furthermore, by calling the pre-loaded WebAssembly acceleration module to perform decoding operations, the bottleneck of insufficient computing performance of traditional JavaScript in the H5 page runtime environment is overcome, greatly improving the processing speed and real-time performance of image recognition. The intelligent memory management system releases image cache, effectively avoiding memory accumulation and leaks caused by frequent image processing, ensuring system stability during long-term operation.

[0112] In one or more embodiments of this disclosure, generating video acquisition guidance information includes: determining the priority of the guidance information based on the specific dimensions of the current video frame that do not meet the image quality standards; adjusting the display color of the scanning frame according to the priority and displaying the corresponding operation prompt text; wherein the display colors include red, yellow and green, which correspond to poor quality, average quality and excellent quality, respectively.

[0113] like Figure 11This diagram illustrates the guidance information display effect as an example of this disclosure. In practical applications, when the system determines that the current video frame does not meet image quality standards, it further analyzes which specific dimension(s) caused the non-compliance. For example, is the current video frame's illumination score too low, or its stability score too low? The system evaluates the impact of different dimensions on the recognition success rate according to preset rules, thereby determining the priority of the guidance information. Typically, key dimensions affecting recognition success or failure (such as severe insufficient illumination or severe jitter) are given higher priority, while secondary dimensions (such as slight lack of contrast) are given lower priority. This step ensures that the system can identify the most pressing core problem and avoid displaying redundant or secondary prompts to the user.

[0114] Based on the priorities determined in the previous step, the system dynamically updates the display color of the scanning frame on the user interface, making it intuitively reflect the quality status of the current video frame. Simultaneously, the system displays corresponding operation prompts near the scanning frame or in designated areas of the page, providing solutions for the highest-priority specific issues. For example, if the highest priority is a lighting problem, it displays "Please move to a better-lit area"; if the highest priority is a stability problem, it displays "Please keep your phone stable." Through the combination of color changes and text prompts, the system provides multimodal feedback to the user, enabling them to quickly understand the current status and take the correct adjustments.

[0115] When the quality of the current video frame is determined to be poor, the system will adjust the display color of the scanning frame to red to warn the user that effective identification is not currently possible; when the quality is average, the display color will be adjusted to yellow to indicate that the user can try but the effect may not be good; when the quality is excellent, the display color will be adjusted to green to indicate that the current time is the best time to shoot.

[0116] Based on the publicly available solutions described above, by prioritizing guidance information according to the specific dimensions that do not meet image quality standards, precise problem localization is achieved, ensuring that users address the issues that have the greatest impact on recognition results first, thus improving adjustment efficiency. Furthermore, by adjusting the display color of the scanning frame according to priority and displaying corresponding operation prompts, an intuitive and information-rich interactive feedback mechanism is established, resolving the issue of blind user operation caused by the inability to directly control hardware in an H5 page runtime environment.

[0117] In one or more embodiments of this disclosure, before selecting multiple target video frames from the candidate video frame buffer, the method further includes: determining the storage state of the candidate video frame buffer; when the candidate video frame buffer is full, comparing the overall quality score of the current video frame with the lowest quality score in the buffer; if the overall quality score of the current video frame is higher than the lowest quality score, then replacing the candidate video frame with the lowest quality score in the buffer.

[0118] Before the system performs the operation of selecting multiple target video frames from the candidate video frame buffer, it first checks the current usage of the candidate video frame buffer. The system reads the number of candidate video frames currently stored in the buffer and compares it with the preset maximum storage capacity to determine the storage status of the candidate video frame buffer. This step is a prerequisite for subsequent differential processing logic, ensuring that the system can adopt appropriate management strategies based on the actual load of the buffer.

[0119] If the judgment result shows that the candidate video frame buffer is full, it means that there is no free space in the buffer to store a new candidate video frame. At this time, the system obtains the overall quality score of the currently processed video frame and iterates through all the candidate video frames stored in the candidate video frame buffer to find the lowest quality score. The system compares the overall quality score of the current video frame with the lowest quality score in the buffer to determine whether the quality of the new frame is better than the worst quality frame in the buffer.

[0120] If the comparison results show that the overall quality score of the current video frame is higher than the lowest quality score, it indicates that the quality of the newly acquired video frame is better than the worst existing frame in the buffer. The system then performs a replacement operation, removing the candidate video frame with the lowest quality score from the buffer, releasing the memory space it occupies, and storing the current video frame in the candidate video frame buffer. If the overall quality score of the current video frame is not higher than the lowest quality score, the current video frame is discarded, and the original contents of the buffer are retained. This ensures that the candidate video frame buffer always stores a batch of candidate video frames with the best quality at the current moment, providing a high-quality data source for subsequent selection of multiple target video frames.

[0121] Based on the publicly available solutions described above, real-time monitoring of memory resources is achieved by judging the storage status of the candidate video frame buffer, avoiding buffer overflows or memory waste caused by blind writing. Furthermore, by comparing the overall quality score with the lowest quality score when the candidate video frame buffer is full, a dynamic update strategy based on quality priority is implemented, ensuring that the buffer content always maintains a high standard and preventing low-quality frames from occupying memory resources for extended periods. By performing replacement operations, the buffer content is optimized, allowing subsequent multi-frame target video frame selection to be based on higher-quality candidate video frames, thereby indirectly improving the quality and recognition accuracy of the final fused image. In memory-constrained scenarios running in an H5 page environment, intelligent management of buffer content maximizes image recognition performance while ensuring processing performance.

[0122] In one or more embodiments of this disclosure, the process of recognizing the enhanced image to obtain an image recognition result further includes: if the recognition fails or the image recognition result is invalid, clearing the candidate video frame buffer; resetting the fusion state and returning to the step of acquiring the video stream in the H5 page-based runtime environment.

[0123] In practical applications, after the decoding operation is completed, the initial recognition data is validated to determine if there have been recognition failures or invalid image recognition results. The system checks whether the decoding algorithm returned an error code, or whether the recognized data content passed the checksum verification and format compliance checks. If the decoding algorithm fails to find any barcode structure, the recognition is deemed a failure; if decoding is successful but data verification fails, the image recognition result is deemed invalid. This ensures that only true and valid recognition results are output, preventing erroneous data from interfering with subsequent business processes.

[0124] Since a recognition failure may indicate that the overall quality of the candidate video frames stored in the current buffer is poor, or that a significant change in the shooting environment has rendered the old frame data inapplicable, the system can perform a clearing operation to remove all historical candidate video frames stored in the candidate video frame buffer. This releases occupied memory resources and prevents low-quality or outdated video frames from participating in the next fusion calculation, ensuring that the next recognition attempt is based on a completely new data source.

[0125] The internal state variables controlling the multi-frame fusion process are restored to their initial state, such as resetting the counter for the number of acquired candidate video frames to zero and resetting the fusion trigger flag. This ensures that subsequent processes can correctly re-accumulate candidate video frames and avoid fusion logic errors caused by residual states, such as preventing fusion processing from being triggered incorrectly when not enough frames have been collected.

[0126] The system will redirect the control flow back to the initial acquisition step and restart video stream acquisition in the H5 page environment. The system will continue to acquire new current video frames, re-evaluate, filter, fuse, and enhance the quality until a valid image recognition result is obtained. This forms a complete recognition retry closed loop, enabling the system to automatically recover and retry after a single recognition failure without requiring the user to manually refresh the page or restart the scan.

[0127] Based on the publicly available solutions described above, strict verification of the recognition results is achieved by judging recognition failure or invalid image recognition results, ensuring the accuracy and reliability of the output data. Furthermore, by clearing the candidate video frame buffer, interference from invalid or outdated data on subsequent recognition processes is avoided, while valuable memory resources in the H5 page runtime environment are released in a timely manner to prevent memory leaks. By resetting the fusion state, the logical correctness of the retry process is ensured, guaranteeing that the multi-frame fusion mechanism can operate normally from the beginning.

[0128] Based on any of the above embodiments, this disclosure also provides an H5-based image recognition device. Figure 12 This is a schematic block diagram of an H5-based image recognition device according to one embodiment of this disclosure. Figure 12 As shown, the image recognition device based on Hypertext Markup Language includes: a determination module 1201, used to acquire video streams in a Hypertext Markup Language page runtime environment and determine candidate video frames that meet image quality standards; a fusion module 1202, used to register and weightedly fuse multiple candidate video frames based on fusion weights to obtain a fused image; an enhancement module 1203, used to select a corresponding image enhancement strategy for optimization processing based on the environmental scene features corresponding to the fused image to obtain an enhanced image; and a recognition module 1204, used to recognize the enhanced image to obtain an image recognition result.

[0129] The determination module 1201 is used to acquire video streams in the Hypertext Markup Language page runtime environment and obtain at least one video frame; to evaluate the quality of the current video frame using image quality standards and obtain evaluation results; when the evaluation result is that the quality of the current video frame does not meet the image quality standards, to generate video acquisition guidance information; and when the evaluation result is that the quality of the current video frame meets the image quality standards, to determine candidate video frames.

[0130] The determination module 1201 is used to calculate the sharpness score, contrast score, illumination score, and stability score of the current video frame; based on the sharpness score, contrast score, illumination score, and stability score, the comprehensive quality score of the current video frame is determined; the comprehensive quality score is compared with a preset threshold in the image quality standard to obtain the evaluation result.

[0131] Optionally, it also includes a data acquisition module 1205, which is used to monitor the stability score of the current video frame in real time; when the stability score is higher than a preset jitter threshold, the frame rate of the acquired video stream is reduced; when the stability score is lower than the preset jitter threshold, the frame rate of the acquired video stream is increased.

[0132] The determination module 1201 is used to extract the coordinates of feature points between the current video frame and the reference video frame; calculate the displacement offset of the feature point coordinates; and determine the stability score based on the magnitude of the displacement offset, where the larger the displacement offset, the lower the stability score.

[0133] The fusion module 1202 is used to select multiple target video frames from the candidate video frame buffer when the number of candidate video frames reaches a preset fusion threshold; extract feature points of the multiple target video frames; perform inter-frame registration correction on the multiple target video frames based on the feature points; dynamically calculate the fusion weight according to the quality score corresponding to each target video frame; and perform pixel mixing on the registered multiple target video frames based on the fusion weight to obtain a fused image.

[0134] The enhancement module 1203 is used to perform environmental feature analysis on the fused image to obtain environmental analysis results; match the target image enhancement strategy from the preset image enhancement strategy set according to the environmental analysis results; and perform adaptive enhancement processing on the fused image using the target image enhancement strategy to obtain the enhanced image.

[0135] The enhancement module 1203 is used to calculate the average brightness of the fused image to determine the illumination level; detect the noise density of the fused image to determine the noise level; measure the gradient magnitude of the edges of the fused image to determine the blur level; statistically analyze the black-and-white contrast intensity of the fused image to determine the contrast level; and use the illumination level, noise level, blur level, and contrast level as environmental analysis results.

[0136] The enhancement module 1203 is used to determine the target image enhancement strategy as a weak light enhancement mode if the light level in the environmental analysis results is lower than a preset light threshold, and to increase the brightness of the fused image and suppress noise; if the light level is higher than the preset light threshold, it determines the target image enhancement strategy as a strong light suppression mode, and to suppress overly bright areas of the fused image and restore details; if the blur level in the environmental analysis results is higher than a preset blur threshold, it determines the target image enhancement strategy as a deblurring mode, and to sharpen the fused image; if both the light level and the blur level are within a preset normal range, it determines the target image enhancement strategy as a standard enhancement mode, and to optimize the contrast and balance tone of the fused image.

[0137] The recognition module 1204 is used to initiate a multi-threaded parallel processing architecture for web page workers (e.g., Web Workers), create an idle computing thread, transfer the data sequence of the enhanced image to the idle computing thread, and call the preloaded web page assembly language (e.g., WebAssembly) acceleration module to perform decoding operations to obtain preliminary recognition data; release the image cache occupied by the idle computing thread through the memory management system, and return the preliminary recognition data to the main thread as the image recognition result.

[0138] The determination module 1201 is used to determine the priority of the guidance information based on the specific dimensions of the current video frame that do not meet the image quality standards; adjust the display color of the scanning frame according to the priority, and display the corresponding operation prompt text; wherein, the display colors include red, yellow and green, which correspond to poor quality, average quality and excellent quality, respectively.

[0139] The fusion module 1202 is used to determine the storage status of the candidate video frame buffer; when the candidate video frame buffer is full, it compares the overall quality score of the current video frame with the lowest quality score in the buffer; if the overall quality score of the current video frame is higher than the lowest quality score, it replaces the candidate video frame with the lowest quality score in the buffer.

[0140] The recognition module 1204 is used to clear the candidate video frame buffer, reset the fusion state, and return to the step of acquiring the video stream in the hypertext markup language page runtime environment if the recognition fails or the image recognition result is invalid.

[0141] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0142] The execution subject of the image recognition method based on hypertext markup language in the specific embodiments of this disclosure can be an electronic device such as a server (including a local server or a cloud computing platform).

[0143] Therefore, based on any of the above embodiments, this disclosure also provides an electronic device that can execute the H5-based image recognition method of any of the embodiments described above.

[0144] Figure 13 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure.

[0145] The hardware architecture of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0146] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, this diagram uses only one connection line, but this does not imply that there is only one bus or one type of bus.

[0147] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0148] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0149] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0150] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] It is understood that before using the technical solutions disclosed in the embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained. Simultaneously, sensitive information should be de-identified (e.g., de-identification, anonymization, etc.). For example, in response to receiving a user's active request, a prompt message can be sent to the user to clearly inform them that the requested operation will require the acquisition and use of their personal information. This allows the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media that perform the operations of this disclosure's technical solutions, based on the prompt message. As an optional but non-limiting implementation, the method of sending a prompt message to the user in response to receiving a user's active request can be, for example, a pop-up window, where the prompt message can be presented in text form. Furthermore, the pop-up window can also include a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0152] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0153] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0154] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0155] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0156] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0157] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. An image recognition method based on Hypertext Markup Language, characterized in that, The method includes: Video streams are captured in a Hypertext Markup Language (HMR) page runtime environment to determine candidate video frames that meet image quality standards. Based on the fusion weights, multiple candidate video frames are registered and weighted and fused to obtain a fused image; Based on the environmental scene features corresponding to the fused image, a corresponding image enhancement strategy is selected for optimization processing to obtain an enhanced image; The enhanced image is then recognized to obtain the image recognition result.

2. The image recognition method based on Hypertext Markup Language according to claim 1, characterized in that, The process of acquiring video streams in a Hypertext Markup Language (HMR) page runtime environment and determining candidate video frames that meet image quality standards includes: Capture video streams in a Hypertext Markup Language (HTML) page runtime environment and obtain at least one video frame; The image quality standard is used to evaluate the quality of the current video frame, and the evaluation result is obtained. When the evaluation result indicates that the quality of the current video frame does not meet the image quality standard, video acquisition guidance information is generated. When the evaluation result indicates that the quality of the current video frame meets the image quality standard, the candidate video frame is determined.

3. The image recognition method based on Hypertext Markup Language according to claim 2, characterized in that, The process of evaluating the quality of the current video frame using the image quality standard to obtain the evaluation result includes: Calculate the sharpness score, contrast score, illumination score, and stability score of the current video frame; Based on the sharpness score, the contrast score, the illumination score, and the stability score, the overall quality score of the current video frame is determined; The overall quality score is compared with a preset threshold in the image quality standard to obtain the evaluation result.

4. The image recognition method based on Hypertext Markup Language according to claim 1, characterized in that, The process of registering and weighting multiple candidate video frames based on fusion weights to obtain a fused image includes: When the number of candidate video frames reaches a preset fusion threshold, multiple target video frames are selected from the candidate video frame buffer. Extract feature points from the multi-frame target video frame, and perform inter-frame registration correction on the multi-frame target video frame based on the feature points; The fusion weights are dynamically calculated based on the quality scores of each target video frame. The registered multi-frame target video is pixel-blended based on the fusion weights to obtain the fused image.

5. The image recognition method based on Hypertext Markup Language according to claim 1, characterized in that, The step of selecting a corresponding image enhancement strategy based on the environmental scene features corresponding to the fused image for optimization processing to obtain an enhanced image includes: Environmental feature analysis is performed on the fused image to obtain environmental analysis results; Based on the environmental analysis results, a target image enhancement strategy is matched from a preset set of image enhancement strategies; The enhanced image is obtained by adaptively enhancing the fused image using the target image enhancement strategy.

6. The image recognition method based on Hypertext Markup Language according to claim 5, characterized in that, The environmental feature analysis of the fused image to obtain the environmental analysis results includes: Calculate the average brightness of the fused image to determine the illumination level; The noise density of the fused image is detected to determine the noise level; Measure the gradient magnitude of the edges of the fused image to determine the degree of blurring; The contrast intensity of the fused image is statistically analyzed to determine the contrast level; The illumination level, noise level, blur level, and contrast level are used as the environmental analysis results.

7. The image recognition method based on Hypertext Markup Language according to claim 5, characterized in that, The target image enhancement strategy is matched from a preset set of image enhancement strategies based on the environmental analysis results. Adaptive enhancement processing of the fused image using the target image enhancement strategy includes: If the illumination level in the environmental analysis results is lower than the preset illumination threshold, the target image enhancement strategy is determined to be a low-light enhancement mode, and the brightness of the fused image is increased while noise is suppressed. If the illumination level is higher than the preset illumination threshold, the target image enhancement strategy is determined to be a strong light suppression mode, and the overly bright areas of the fused image are suppressed and details are restored. If the degree of blurring in the environmental analysis results is higher than the preset blurring threshold, the target image enhancement strategy is determined to be the deblurring mode, and the fused image is sharpened. If both the illumination level and the blur level are within a preset normal range, the target image enhancement strategy is determined to be the standard enhancement mode, and the contrast and tonal balance of the fused image are optimized.

8. The image recognition method based on Hypertext Markup Language according to claim 1, characterized in that, The step of recognizing the enhanced image to obtain the image recognition result includes: Start the multi-threaded parallel processing architecture for web page operations and create idle computing threads; The data sequence of the enhanced image is transmitted to the idle computing thread, and the preloaded web page assembly language acceleration module is called to perform decoding operations to obtain preliminary recognition data; The image cache occupied by the idle computing thread is released through the memory management system, and the preliminary recognition data is returned to the main thread as the image recognition result.

9. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes execution instructions stored in the memory, causing the processor to perform the method of any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images