A code scanning method, device, apparatus and computer readable storage medium

Abstract

The application provides a code scanning method, device, apparatus and computer readable storage medium. The code scanning method comprises the following steps: a camera scans a code in a high frame rate mode, acquires raw format image data and stores the raw format image data in a memory; after the raw format image data is subjected to frame rate reduction processing and image data format conversion processing, the raw format image data is delivered to a display module through a first path; the raw format image data retrieved from the memory is delivered to a decoding module through a second path, and code scanning information is acquired, so that more accurate and faster code scanning effect is achieved.

Description

Technical Field

[0001] This invention relates to the field of barcode scanning equipment technology, and in particular to a barcode scanning method, equipment, apparatus, and computer-readable storage medium. Background Technology

[0002] The use of point-of-sale (POS) systems in the retail industry is increasing. Currently, POS systems in retail stores are typically PC-based stationary POS machines, which generally use wired connections between the PC and peripherals such as barcode scanners and POS machines to complete the checkout process.

[0003] With the widespread adoption of mobile payments, the requirements for scanning speed and accuracy of smart POS machines or smart barcode scanners used in mobile financial payment terminals are becoming increasingly stringent. Improving the frame rate and image quality of the camera in the scanning process is crucial. Currently, most POS machines on the market, likely due to cost considerations, use low-to-mid-range ISP processors (Image Signal Processing modules), which typically only support a maximum camera frame rate of 30fps, limiting scanning speed. Secondly, mainstream POS machines on the market use an aiming light to align with the barcode, and the red light emitted can easily affect the quality of the acquired barcode or QR code image, resulting in lower decoding accuracy.

[0004] Currently, the main approach is to use high-end processor chips to improve the camera module's image data processing capabilities, thereby outputting high frame rate image data to increase scanning speed. However, this method is too costly and lacks competitiveness in the market. Secondly, the mainstream Android platform's scanning of camera data streams results in a decrease in the scanning module's performance in terms of scanning distance, code type recognition, and scanning speed. Using the camera's output raw image data directly in the scanning module's decoding library can achieve higher and faster scanning results. However, raw image data cannot be used for display, while YUV image data can. During the scanning process, the lack of visual feedback from the scanned object is a poor user experience, as errors go unnoticed, making the scanning process like the blind men and the elephant, significantly reducing efficiency.

[0005] Therefore, it is necessary to develop new intelligent barcode scanning terminal devices and scanning methods to solve the aforementioned problems existing in the current technology. Summary of the Invention

[0006] The purpose of this invention is to provide a barcode scanning method, device, apparatus, and computer-readable storage medium to reduce barcode scanning error rate and improve barcode scanning efficiency.

[0007] In a first aspect, embodiments of the present invention provide a scanning method comprising the following steps: a camera scans the code in a high frame rate mode to acquire raw format image data at 120fps and stores it in a memory; the raw format image data is processed by reducing the frame rate and converting the image data format, and then transmitted to a display module through a first path; the raw format image data retrieved from the memory is transmitted to a decoding module through a second path to obtain the scanning information.

[0008] In the optional scanning method, the camera scans the code in high frame rate mode to obtain raw format image data, which is then stored in a first memory and a second memory, respectively. The raw format image data in the first memory and the second memory are then used for subsequent operations through the first and second paths, respectively.

[0009] The beneficial effects of the barcode scanning device provided in this invention are as follows: This invention aims to improve efficiency by separating the data stream—which previously required video display followed by barcode information extraction—from two concurrently operating data streams. One data path automatically determines the range of high-quality multi-frame images supplied to the decoding module through the automatic setting of the 3A effect parameters of the display module. The other data path uses the decoding algorithm in the decoding module to analyze multiple images and fuse the results, resulting in higher confidence and accuracy. Regarding the data format for scanning / decoding, raw format image data is not used for display; it is generally used for displaying images requiring YUV format. Different algorithms have different format requirements for barcode scanning and decoding, but common algorithms generally perform worse on images processed by the ISP than on images without ISP processing. While YUV images processed by the ISP have better color and brightness and look better visually, the original raw image data is easier for decoding algorithms to recognize and analyze. Therefore, this invention uses two data streams: one YUV image data stream for previewing and one RAW image data stream for decoding.

[0010] A further feasible scanning method, based on the aforementioned process of reducing the frame rate of the raw format image data, further includes: a frame rate reduction module reducing the raw format image data at 120fps to raw format image data at 30fps, and then sending it to the ISP module.

[0011] A further feasible scanning method involves the ISP module converting raw format image data into YUV format image data.

[0012] A further feasible scanning method, in which the raw format image data is transmitted to the display module via the first path, further includes transmitting the yuv format image data first through the Android HAL layer to the application layer, and then sending it to the display module through the application layer.

[0013] In an optional scanning method, the raw format image data undergoes frame rate reduction processing, which further includes frame dropping or frame extraction processing.

[0014] In an optional scanning method, retrieving the raw format image data from the memory and transmitting it to the decoding module via a second path further includes: configuring a virtual channel (v_ch) in the kernel layer path to first transmit the raw format image data through the Android HAL layer to the application layer, and then sending it to the decoding module from the application layer.

[0015] In an optional scanning method, the raw format image data is transmitted to the display module via a first path to automatically set the display effect parameters.

[0016] Among the optional scanning methods, the automatic setting of display parameters includes automatic exposure, automatic focus, and automatic white balance.

[0017] In the optional scanning method, when the automatic settings of display effect parameters (auto exposure, auto focus, and auto white balance) meet the thresholds, the decoding module defines a range of multiple frames of image data from the raw format image data in the second path whose display effect parameters meet the thresholds. Each frame of image data within this range meets the requirements for high-quality image data, and these high-quality image data can be decoded by the decoding module. The decoding algorithm in the decoding module typically analyzes and merges multiple images, resulting in a higher confidence level for the analysis results.

[0018] In an optional scanning method, after the scanning application is started and before the camera performs scanning in high frame rate mode, the camera performs pre-capture in default frame rate mode, outputting raw format image data at 30fps, which is then converted into YUV format image data by the ISP module and sent to the display module.

[0019] In one possible implementation, the second aspect of the present invention provides a (smart) barcode scanning device, the barcode scanning device comprising at least: The camera is used to align with the code source and scan to obtain raw image data; The CSI module is used to convert the raw image data into raw format image data. The ISP module is used to convert the raw format image data into the yuv format image data; The display module is used to automatically set the display effect parameters based on the input YUV format image data; The decoding module is used to obtain scanning information based on the input raw format image data; The raw image data is transmitted to the display module via a first path; The original image data is synchronously transmitted to the decoding module through the second path to obtain the scanning information; When the automatic setting of the display effect parameters meets the threshold, the frame-finding module searches for and determines multiple frames of image data whose display effect parameters meet the threshold from the raw format image data in the second path, and provides them to the decoding module for decoding.

[0020] In an optional implementation of the barcode scanning device, the exemplary barcode scanning device further includes: A memory for storing the original raw format image data; The raw format image data retrieved from the memory is transmitted to the decoding module via the second path.

[0021] In an optional implementation of a barcode scanning device, the barcode scanning device further includes: The frame rate reduction module is used to process the raw format image data output by the CSI module at 120fps into raw format image data at 30fps, and then send it to the ISP module.

[0022] In an optional implementation of a barcode scanning device, the barcode scanning device further includes: The first path involves the camera acquiring raw image data, which then passes through the CSI module, the ISP module, the HAL layer of Android, and finally reaches the application layer, before being sent to the display module via the application layer.

[0023] In an optional implementation of a barcode scanning device, the barcode scanning device further includes: The second path involves the camera acquiring raw image data, which is then transmitted through the CSI module. In the kernel layer, a virtual channel is configured to first pass through the Android HAL layer to the application layer, and then the application layer sends it to the decoding module.

[0024] In one possible implementation, a third aspect of the present invention provides a barcode scanning device, comprising at least a processor and a memory, wherein the memory stores a computer program; the processor executes the computer program stored in the memory to cause the barcode scanning device to perform the barcode scanning method described in any one of the first aspects. A fourth aspect, embodiments of the present invention also provide a computer-readable storage medium storing a computer program such that when the computer program is executed by a processor, it implements the barcode scanning method described in any one of the first aspects. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the data flow structure of a scanned image provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating how a frame-finding instruction triggered by a frame-finding module during a barcode scanning process determines the range of image data, according to an embodiment of the present invention. Figure 3 A schematic diagram of a barcode scanning device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the execution steps of a QR code scanning application according to an embodiment of the present invention; Figure 5 This is a schematic diagram of a high frame rate raw data stream with slow motion enabled, provided in an embodiment of the present invention. Figure 6 A schematic diagram of an apparatus provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the scanning steps provided in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings. In the description of the embodiments of the present invention, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two). The term “and / or” is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0028] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0029] In embodiments of the present invention, the terms "exemplarily" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplarily" or "for example" in embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0030] The scanning method provided in the embodiments of the present invention, such as Figure 1 and Figure 7 As shown, the process includes the following steps: S300: The camera 100 scans the code in high frame rate mode, acquires raw format image data, and stores it in the memory 300; S500: After performing frame rate reduction and image data format conversion on the raw format image data, it is transmitted to the display module 700 through the first path; S700: The raw format image data retrieved from the memory 300 is transmitted to the decoding module 900 through the second path to obtain the scanning information.

[0031] In an optional scanning method implementation example, camera 100 scans the code in high frame rate mode, acquiring raw format image data, which is then stored in a first memory and a second memory respectively. The raw format image data in the first and second memories are then processed through the first and second paths respectively. Alternatively, the image data stream can be split into two paths and stored separately in two DDR memories via hardware, or it can be first stored in DDR and then copied via hardware or software for further processing.

[0032] In the optional scanning method implementation examples, this solution proposes a fast scanning system based on a smart POS machine. This system configures a preview data stream and a callback raw data stream. The preview data stream is processed by the ISP module to set the 3A effect parameters (AE auto exposure, AF auto focus, AWB auto white balance), and the YUV data is sent to the DPU (Data Processing Unit) for processing before being used for display. This replaces the aiming light solution by adding step S100 before step S300: after the scanning application is launched and before the camera performs scanning in high frame rate mode, the camera performs a pre-capture in default frame rate mode, outputting raw format image data at 30fps. This raw format image data is then converted to YUV format image data by the ISP module and sent to the display module. This eliminates light interference from the aiming light during the scanning process.

[0033] A further feasible scanning method, wherein the camera 100 performs scanning in a high frame rate mode, further includes: when the camera's high frame rate mode is enabled, the camera scans at a speed of 120fps to acquire the raw format image data.

[0034] A further implementable scanning method, according to the above-mentioned frame rate reduction processing of the raw format image data, further includes: the frame rate reduction module reduces the raw format image data at 120fps to the raw format image data at 30fps, and sends it to the ISP module 500.

[0035] A further feasible scanning method involves the ISP module 500 converting the raw format image data into YUV format image data. Since many low-to-mid-range ISP modules / chips lack the capability to process 120fps high frame rate raw format image data, in order to continue using low-to-mid-range ISP modules without increasing hardware costs, it is necessary to use 30fps low frame rate raw format image data.

[0036] A further feasible scanning method, in which the raw format image data is transmitted to the display module 700 via the first path, further includes transmitting the yuv format image data through the Android HAL layer 600 to the application layer, and then sending it to the display module 700 through the application layer.

[0037] In an optional scanning method, the raw format image data undergoes frame rate reduction processing, which further includes frame dropping or frame extraction. Specifically, an example is to reduce the frame rate of 120fps raw format image data to 30fps raw format image data to meet the requirements.

[0038] In an optional scanning method, retrieving the raw format image data from the memory 300 and transmitting it to the decoding module via the second path further includes: configuring a virtual channel (v_ch) in the kernel layer path to bypass the ISP module 500 (bypassing the ISP's Pipleline or passing through the ISP's Pipleline, but without any processing by the ISP), transmitting the high frame rate raw format image data through the Android HAL layer 600 to the application layer, and then sending it to the decoding module 900 from the application layer.

[0039] Among the optional methods for implementing QR code scanning, such as Figure 2 As shown, the raw format image data is transmitted to the display module 700 through the first path to automatically set display effect parameters. Automatic setting of display effect parameters includes automatic focus, automatic exposure, and automatic white balance. These parameters are determined based on information extracted from the image data. Simultaneously, when these parameters reach a threshold, the 3A algorithm (automatic focus, automatic exposure, and automatic white balance) in the HAL layer triggers the frame-finding module 750 to search for and determine the corresponding high frame rate image data (storage address) with time or sequence number parameters in the memory 300, and provides them (storage addresses) to the decoding module 900 for processing.

[0040] In the optional scanning method, when the automatic setting of the display effect parameters meets the threshold, the decoding module defines a range of multiple frames of image data from the raw format image data in the second path whose display effect parameters meet the threshold. Each frame of image data within this range meets the requirements for high-quality image data, and these high-quality image data can be decoded by the decoding module. The decoding algorithm in the decoding module typically parses and merges multiple images, resulting in a higher confidence level for the parsing results.

[0041] The barcode scanning device provided in the embodiments of the present invention, such as Figure 3 As shown, the barcode scanning device 1000 includes at least: a camera 100, a CSI module 200, a memory 300, an ISP module 500, a display module 700, and a decoding module 900; The image data passes sequentially through the camera 100, CSI module 200, memory 300, ISP module 500, HAL layer (Hardware Abstraction Layer) 600, and application layer via the first path, and is finally transmitted to the display module 700. The image data passes through the camera 100, CSI module 200, and memory 300 sequentially via the second path, then bypasses the ISP module (bypasses the ISP's Pipleline (or passes through the ISP's Pipleline, but the ISP does not perform any processing) via the virtual pipe (v_ch), reaches the application layer through Android's HAL layer 600, and is finally sent to the decoding module 900 to obtain the scanning information. The optional implementation scheme of the intelligent control process is as follows: when the display algorithm in the display module meets the threshold based on the automatic setting of the display effect parameters extracted from the image data, it notifies the frame finding module 750 to determine the multiple frames of image data whose display effect parameters meet the threshold from the raw format image data in the second path, so that the decoding module 900 can perform targeted decoding. Specifically, the frame finding module 750 is notified to find the corresponding data image range (multiple frames) from the memory based on the time information or sequence number information of the multiple frames of image data that meet the threshold, and to extract these frames for the decoding module 900 to decode.

[0042] In the optional implementation of barcode scanning device 1000, such as Figure 3 As shown, the scanning device also includes: The frame rate reduction module 750 is used to process the raw format image data output by the CSI module 200 at a speed of 120fps into raw format image data at a speed of 30fps, and send it to the ISP module 500.

[0043] In an optional implementation of a barcode scanning device, the barcode scanning device further includes: The first path involves the camera 100 acquiring raw image data, which then passes sequentially through the CSI module 200, the memory 300, the ISP module 500, and through the Android HAL layer to reach the application layer, before being sent to the display module 700 via the application layer.

[0044] In an optional implementation of a barcode scanning device, the barcode scanning device further includes: The second path involves the camera 100 acquiring raw image data, which is then transmitted through the CSI module via a configured virtual channel 800 in the kernel layer to the application layer via the Android HAL layer, and finally sent to the decoding module 900 through the application layer.

[0045] In one possible scenario, the image data stream scheme for the barcode scanning device is as follows: Figure 4 As shown, the scanning device uses two data streams internally to achieve preview display and scanning respectively. After the camera (sensor) outputs raw format image data stream, it passes through the mipi_csi interface (CSI (Camera Serial Interface) is a camera interface standard specified by the Camera Working Group under the MIPI (Mobile Industry Processor Interface) alliance. It mainly consists of application layer, protocol layer, and physical layer, supporting up to 4 channels of data transmission and a single-line transmission speed of up to 1Gb / s). The raw image data is then divided into two paths and output to the memory DDR. One path serves as the preview image data preview stream. After being processed by the ISP module 500 and converted to YUV format, it is reported to the Android HAL layer 600 and the application layer. The application layer then sends the data stream to the display module 700 for display in the scanning application. The other data stream serves as the callback raw data stream for decoding. It bypasses the ISP module 500's Pipleline (or goes through the ISP's Pipleline) by configuring a virtual channel (v_ch) in the kernel's underlying path. However, the ISP does not perform any processing and directly reports the data to the Android HAL layer 600 and the application layer. The HAL layer 600 and the application layer then send the data stream to the decoding module 900. The decoding library outputs the decoding result to obtain the barcode information. The callback data stream is to return the virtual address of one of the DDR memory blocks directly to the HAL / user layer without going through the ISP.

[0046] In some other embodiments, the method logic of the original single-path QR code scanning application is optimized, such as... Figure 4As shown, a dual-path system is formed. After the scanning application starts but before the scanning button is clicked to trigger the scanning action, the preview image data stream is opened. The camera sensor 100 outputs raw image data at 30fps in default frame rate mode. This data is converted to YUV format by the ISP module 500 and then sent to the display module for display, allowing users to intuitively see the camera's aiming position. When an external scanning action is triggered, the 30fps raw image data output by the camera sensor 100 is sent to the decoding module 900 for decoding, obtaining barcode, QR code, and color code information. The callback raw data stream is closed when the scan is successful or the scan is completed. The preview stream is closed when the scanning application is exited, and the camera sensor 100 is shut down.

[0047] In a further preferred embodiment, a high frame rate raw data stream solution is achieved by utilizing the slow motion recording function of the camera, such as... Figure 5 As shown, before the QR code scanning application starts and triggers the scanning action, the platform's default supported frame rate (e.g., 30fps) is used to preview the image data stream. After processing by the ISP, the stream is sent to the display module. When the scanning action is triggered, with the help of the Android platform's slow motion recording function, the camera sensor 100 outputs 120fps raw image format data in high frame rate mode. While preserving the preview stream to ensure normal display, considering the ISP's capabilities, the 120fps raw image format data must be downgraded to 30fps raw image format data before being sent to the ISP module 500. At this time, before entering the ISP, the preview stream can be sent to the ISP 500 for processing at the default frame rate (e.g., 30fps) through software means such as frame dropping or frame extraction to ensure the display effect. During the activation of the callback raw data stream, the scanner accesses the 120fps raw image format data already stored in memory. The callback raw data stream transmits the high frame rate (e.g., 120fps) raw format image data output by the camera sensor 100 to the decoding module 900 via a virtual channel, bypassing the ISP module 500, thus increasing the scanning speed. When the callback raw data stream is closed upon successful scanning or when the scanning process ends, the high frame rate mode of the camera sensor (which has slow-motion recording capabilities) is disabled, and image data continues to be output to the display module at the default frame rate (e.g., 30fps).

[0048] In other embodiments, the logic for optimizing the QR code scanning application, under the premise that low-end and mid-range ISP modules do not support high frame rate image data processing capabilities, utilizes the slow motion function of the camera 100 to achieve a high frame rate raw data stream solution, such as... Figure 4 As shown, before the scanning application starts and triggers the scanning action, the preview stream uses the platform's default supported frame rate (e.g., 30fps), which is processed by the ISP and sent to the display module. When the scanning action is triggered and the callback raw data stream is started, the Android platform's slow motion recording function is used to enable the high frame rate mode of the camera sensor (which has slow motion recording capabilities). The camera sensor starts outputting raw format image data at a high frame rate (e.g., 120fps). At this time, considering the ISP's processing capacity, the preview stream can reduce the frame rate before entering the ISP module 500 through software means such as dropping or skipping frames, so that it is still sent to the display module 700 at the default frame rate, such as 30fps, to ensure the display effect. The callback raw data stream from the memory 300 transmits the high frame rate, such as 120fps, raw format image data output by the camera to the decoding module 900, realizing high-speed scanning and decoding. When the scan is successful or the raw data stream is closed after the scan is completed, the slow-motion recording function of the camera 100, i.e., the high frame rate mode, is turned off, and the image data is continued to be output to the display module at the default frame rate (e.g., 30fps).

[0049] In some embodiments, Figure 6 A hardware configuration block diagram of the intelligent barcode scanning device 30 is shown. The intelligent barcode scanning device 30 includes at least one of the following: a tuner / demodulator 310, a mobile communication module 320, a wireless communication module 330, a data collector 340, an external device interface 350, a controller 360, a display 370, an audio output interface 380, a memory, a power supply, and a user interface.

[0050] In some other embodiments, the tuner 310 senses electromagnetic waves through an antenna, converts the sensed electromagnetic waves into electrical signals, and then processes and transforms them through circuitry to finally convert them into sound, such as receiving broadcast signals wirelessly and demodulating audio signals from the broadcast signals.

[0051] The mobile communication module 320 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use in the smart scanning device 30. The mobile communication module 320 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 320 can receive electromagnetic waves via an antenna, filter and amplify the received electromagnetic waves, and transmit them to the tuner / demodulator 310 for demodulation. The mobile communication module 320 can also amplify the signal modulated by the tuner / demodulator 310 and convert it into electromagnetic waves for radiation via the antenna. In some embodiments, at least some functional modules of the mobile communication module 320 may be housed in the controller 360. In some embodiments, at least some functional modules of the mobile communication module 320 and at least some modules of the controller 360 may be housed in the same device.

[0052] The wireless communication module 330 can provide solutions for wireless communication applications on the smart barcode scanning device 30, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 330 can be one or more devices integrating at least one communication processing module. The wireless communication module 330 receives electromagnetic waves via an antenna, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to the controller 360. The wireless communication module 330 can also receive signals to be transmitted from the controller 360, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via the antenna.

[0053] In other embodiments, the collector 340 is used to collect signals from the external environment or to interact with the outside world. For example, the collector 340 includes a light receiver, a sensor for collecting ambient light intensity; or, the collector 340 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, the collector 340 includes a sound collector, such as a microphone, for receiving external sounds.

[0054] In some other embodiments, the external device interface 350 may include, but is not limited to, one or more of the following interfaces: High Definition Multimedia Interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0055] In some other embodiments, the controller 360 and the tuner 310 may be located in different separate devices, that is, the tuner 310 may also be located in an external device of the main device where the controller 360 is located, such as an external set-top box.

[0056] In other embodiments, the controller 360 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 360 controls the overall operation of the smart barcode scanning device 30. For example, in response to receiving a user command to select a UI object to display on the display 370, the controller 360 can perform operations related to the object selected by the user command.

[0057] In some possible embodiments, the controller 360 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM, ROM, a first to an nth interface for input / output, a communication bus, etc.

[0058] A central processing unit (CPU) is used to execute operating system and application instructions stored in memory, as well as various interactive instructions received from external input, to execute various applications, data, and content in order to ultimately display and play various audio and video content. A CPU can include multiple processors, such as a main processor and one or more subprocessors.

[0059] In some embodiments, a graphics processor is used to generate at least one of various graphical objects, such as icons, operation menus, and user-input-based graphics. The graphics processor includes an arithmetic logic unit (ALU) that performs calculations based on various user-input interactive commands and displays various objects according to display attributes; it also includes a renderer that renders the various objects obtained from the ALU, the rendered objects being displayed on a monitor.

[0060] In some embodiments, the video processor is configured to receive external video signals and perform at least one of the following video processing operations according to the standard encoding and decoding protocol of the input signals: decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, etc., to obtain the signal displayed or played on the direct smart barcode scanning device 30.

[0061] In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image compositing module, a frame rate conversion module, and a display formatting module. The demultiplexing module demultiplexes the input audio and video data streams. The video decoding module processes the demultiplexed video signal, including decoding and scaling. The image compositing module, such as an image synthesizer, overlays and blends a graphical user interface signal generated by a graphics generator based on user input or its own generation with the scaled video image to generate a displayable image signal. The frame rate conversion module converts the input video frame rate. The display formatting module modifies the received frame rate-converted video output signal to conform to a display format, such as outputting RGB data signals.

[0062] In some embodiments, an audio processor is configured to receive external audio signals, perform decompression and decoding according to a standard codec protocol of the input signal, and at least one of the following processes: noise reduction, digital-to-analog conversion, and amplification, to obtain a sound signal that can be played in a speaker.

[0063] In some embodiments, the user can input user commands through a graphical user interface displayed on the display 370, and the user input interface receives the user input commands through the graphical user interface. Alternatively, the user can input user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0064] In some embodiments, a "user interface" is the medium through which an application or operating system interacts and exchanges information with a user, enabling the conversion between the internal form of information and a form acceptable to the user. A graphical user interface (GUI) refers to a user interface related to computer operation displayed graphically. It can be an interface element such as an icon, window, or control displayed on the screen of an electronic device, where controls can include at least one of the visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, and navigation bars.

[0065] In some embodiments, the display 370 includes a display screen component for presenting an image, a driving component for driving image display, a component for receiving image signals from the controller output, and a user control interface, etc.

[0066] In other embodiments, the display 370 may be at least one of a liquid crystal display, an organic light-emitting diode (OLED) display, and a projection display, and may also be a projection device and a projection screen.

[0067] In some other embodiments, the audio output interface 380 includes a speaker, external audio output electronics, etc.

[0068] In some embodiments, the user interface is an interface that can be used to receive control input (e.g., physical buttons on the display device body, or others).

[0069] In specific implementation, the aforementioned smart scanning device 30 can be a mobile phone, tablet computer, handheld computer, personal computer (PC), cellular phone, personal digital assistant (PDA), wearable device (such as smartwatch), smart home device (such as television), in-vehicle computer, game console, and augmented reality (AR) / virtual reality (VR) device or other electronic product containing a camera. This embodiment does not impose any special restrictions on the specific device form of the smart scanning device 30.

[0070] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0071] In the various embodiments of this invention, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0072] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0073] The above description is merely a specific implementation of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of the present invention should be covered within the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention should be determined by the protection scope of the claims.

Claims

1. A barcode scanning method, characterized in that, Includes the following steps: The camera scans the code in high frame rate mode, acquiring raw format image data at 120fps and storing it in the memory. The raw format image data is processed by reducing the frame rate and then sent to the ISP module. The ISP module converts the raw format image data after the frame rate reduction into YUV format image data and sends it to the display module through the first path to realize the automatic setting of display effect parameters. The automatic setting includes automatic exposure, automatic focus and automatic white balance. Raw format image data is retrieved from memory. By configuring a virtual channel in the kernel layer, the raw format image data is first sent to the application layer through the Android HAL layer, and then sent to the decoding module through the second path by the application layer. When the automatic setting of the display effect parameters meets the threshold, the frame finding module determines multiple frames of image data whose display effect parameters meet the threshold from the raw format image data in the second path, and provides them to the decoding module for decoding to obtain the scanning information.

2. The scanning method according to claim 1, characterized in that, The process of reducing the frame rate of the raw format image data includes: The frame rate reduction module reduces the 120fps raw format image data to 30fps raw format image data.

3. The scanning method according to claim 2, characterized in that, The process of sending the raw format image data to the display module via the first path further includes: first sending the yuv format image data through the Android HAL layer to the application layer, and then sending it to the display module through the application layer.

4. The scanning method according to claim 2, characterized in that, The frame rate reduction process for the raw format image data includes frame dropping or frame extraction.

5. The scanning method according to claim 1, characterized in that, After the scanning application is launched and before the camera performs scanning in high frame rate mode, the camera performs pre-capture in default frame rate mode, outputting raw format image data at 30fps. This raw format image data is then converted into YUV format image data by the ISP module and sent to the display module.

6. A barcode scanning device, characterized in that, The barcode scanning device includes at least: A camera is used to align with the code source and scan to acquire raw image data at 120fps. The CSI module is used to convert the raw image data into raw format image data. A memory for storing the raw format image data; The frame rate reduction module is used to reduce the frame rate of the raw format image data; The ISP module is used to convert the raw format image data after frame rate reduction processing into YUV format image data, and send it to the display module through the first path; The display module is used to automatically set display effect parameters based on the input YUV format image data; the automatic settings include automatic exposure, automatic focus, and automatic white balance. Raw format image data is retrieved from the memory and, by configuring a virtual channel in the kernel layer, is first sent to the application layer through the Android HAL layer, and then sent to the decoding module through the second path from the application layer. When the automatic setting of the display effect parameters meets the threshold, the frame finding module determines multiple frames of image data whose display effect parameters meet the threshold from the raw format image data in the second path, and provides them to the decoding module for decoding; The decoding module is used to obtain the scanning information based on the input multi-frame image data.

7. A barcode scanning device as described in claim 6, characterized in that: The frame rate reduction module is used to process the raw format image data output by the CSI module at 120fps into raw format image data at 30fps, and then send it to the ISP module.

8. A barcode scanning device as described in claim 6, characterized in that, The scanning device also includes: The first path involves the camera acquiring raw image data, which then passes through the CSI module, the ISP module, and the HAL layer of Android to reach the application layer, before being sent from the application layer to the display module.

9. A barcode scanning device, characterized in that, It includes at least: a processor and a memory, the memory being used to store a computer program; the processor being used to execute the computer program stored in the memory to cause the scanning device to perform the scanning method according to any one of claims 1 to 5.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the scanning method according to any one of claims 1 to 5.

Images