A processing method based on QCamera camera image data

Abstract

The application relates to the technical field of data processing, and provides a processing method based on QCamera camera image data, which comprises the following steps: a data capture thread acquires QCamera camera data packets and stores the QCamera camera data packets in a preset position; a data transcoding thread extracts the QCamera camera data packets from the preset position to perform a transcoding operation and obtains first image data packets; the first image data packets are transmitted to a data drawing thread to be drawn to obtain second image data packets, so as to obtain data processed by the QCamera camera image data. The processing method based on the QCamera camera image data integrates a data capture thread, a data transcoding thread and a data drawing thread in a Qt multimedia framework, the data capture thread is transmitted to the data transcoding thread to perform transcoding processing, finally, the data drawing thread is used for drawing, the memory burden of an MCU of the Qt multimedia framework is greatly shared, and the performance of secondary processing of original camera data under the Qt multimedia framework is obviously improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a processing method based on QCamera camera image data. BACKGROUND

[0002] In the Qt multimedia framework, camera data is generally collected by using a QCameraImageCapture object to collect data. This method has performance problems in the secondary processing of video streams at a high frame rate, which affects the interactive experience of the software UI layer. At the same time, the existing data is processed by the Qt multimedia framework, which causes CPU memory to be blocked and affects image acquisition and data processing. SUMMARY

[0003] To solve the above problems in the prior art, the present application provides a processing method based on QCamera camera image data, comprising:

[0004] S100, a data capture thread acquires a QCamera camera data packet and stores it in a preset position, and sends a transcoding operation signal to a data transcoding thread;

[0005] S200, after receiving the transcoding operation signal, the data transcoding thread determines whether transcoding operation is needed; if needed, the data transcoding thread extracts the QCamera camera data packet from the preset position for transcoding operation to obtain a first image data packet, and sends a data processing signal to a data rendering thread;

[0006] S300, after receiving the data processing signal, the data rendering thread determines whether data processing operation is needed; if needed, the first image data packet is transmitted to the data rendering thread for rendering to obtain a second image data packet, so as to obtain the processed data of the QCamera camera image data.

[0007] Further, the data capture thread uses a registration callback mechanism to acquire the QCamera camera data packet.

[0008] Further, the data capture thread stores the acquired QCamera camera data packet in a program running memory area, and the data transcoding thread extracts the QCamera camera data packet from the program running memory area for transcoding operation.

[0009] Further, the latest QCamera camera data packet acquired in the data capture thread is cached at the front end of a shared queue, and when the QCamera camera data packet in the shared queue reaches a preset cache data amount, the old QCamera camera data packet is discarded from the end of the shared queue.

[0010] Further, the data transcoding thread converts the QCamera camera data packet in yuyv format into a first image data packet in RGB format.

[0011] Further, the data transcoding thread copies at least one QCamera camera data packet from the end of the shared queue, the shared queue deletes the copied QCamera camera data packet, and the data transcoding thread performs transcoding processing on the copied QCamera camera data packet.

[0012] Further, the data rendering thread adopts a UI double-buffering mechanism to render the first image data packet to obtain a second image data packet.

[0013] The application further provides a processing device based on QCamera camera image data, comprising a data capture thread module configured to store QCamera camera data packets in a preset position and send a transcoding operation signal to a data transcoding thread;

[0014] The data transcoding thread module is configured to determine whether transcoding operation is needed after receiving the transcoding operation signal; if needed, the data transcoding thread extracts the QCamera camera data packet from the preset position to perform transcoding operation to obtain a first image data packet, and sends a data processing signal to a data rendering thread;

[0015] The data rendering thread module is configured to determine whether data processing operation is needed after receiving the data processing signal; if needed, the first image data packet is transmitted to the data rendering thread to be rendered to obtain a second image data packet, so as to obtain data processed by the QCamera camera image data.

[0016] The application further provides a computer readable storage medium, which stores computer instructions, and the computer is executed by a processor to realize the processing method based on the QCamera camera image data according to any one of the above embodiments.

[0017] The application further provides an electronic device, which comprises at least one processor and a memory in communication connection with the processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to make the processor execute the processing method based on the QCamera camera image data according to any one of the above embodiments.

[0018] Based on the above, compared with the prior art, the processing method based on QCamera camera image data provided by the application integrates data capture threads, data transcoding threads and data rendering threads in the Qt multimedia framework, transmits to the data transcoding threads for transcoding processing through the data capture threads, and finally renders through the data rendering threads, greatly sharing the memory burden of the Qt multimedia framework MCU, and significantly improving the performance of secondary processing of camera raw data based on the Qt multimedia framework.

[0019] Other features and advantages of the present application will be set forth in the description that follows, and in part will be apparent from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings described in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings. In the following description, the positional relationship described in the drawings is the direction of the components drawn in the drawings as the reference, unless otherwise specified.

[0021] Figure 1 The flow chart of the processing method based on QCamera camera image data provided by the first embodiment of the present application;

[0022] Figure 2 The structural block diagram of the processing device based on QCamera camera image data provided by the second embodiment of the present application;

[0023] Figure 3 The structural block diagram of the electronic device provided by the fourth embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. The technical features designed in different embodiments of the present application can be combined with each other as long as they do not conflict with each other. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0025] In the description of the present application, it should be noted that all the terms (including technical terms and scientific terms) used in the present application have the same meaning as that generally understood by the ordinary skilled person in the field to which the present application belongs, and should not be understood as a limitation on the present application; it should be further understood that the terms used in the present application should be understood as having the same meaning as the terms in the context of the present application and the related field, and should not be understood in an idealized or overly formal sense, unless otherwise defined in the present application.

[0026] Embodiment one

[0027] The present application provides a processing method based on QCamera camera image data, as shown in Figure 1 The present application provides a processing method based on QCamera camera image data, as shown in

[0028] S100, the data capture thread stores the QCamera camera data packet to the preset position and sends a transcoding operation signal to the data transcoding thread.

[0029] Specifically, the data capture thread uses a registration callback mechanism to obtain the QCamera camera data packet, and stores the obtained QCamera camera data packet to the program running memory area. The data capture thread and the data transcoding thread communicate through the transcoding operation signal, and the data capture thread sends the transcoding operation signal to the data transcoding thread every time it obtains the QCamera camera data packet (without synchronous blocking waiting); when there is new data update, the data capture thread will trigger an asynchronous signal to inform the transcoding thread to process.

[0030] In specific implementation, when CPU resources are insufficient or software is in the background, the shared queue buffer mechanism is used to automatically discard data packets with a longer time distance to reduce delay time and memory occupation.

[0031] The latest QCamera camera data packet obtained in the data capture thread will be cached to the front end of the shared queue, and when the QCamera camera data packet in the shared queue reaches the preset cache data amount, the old QCamera camera data packet will be discarded from the end of the shared queue.

[0032] Preferably, the QCamera camera data packet is a frame-by-frame camera image collected by the camera.

[0033] S200, after receiving the transcoding operation signal, the data transcoding thread judges whether transcoding operation is needed; if needed, the data transcoding thread extracts the QCamera camera data packet from the preset position to perform transcoding operation to obtain a first image data packet, and sends a data processing signal to the data drawing thread.

[0034] Specifically, the data transcoding thread converts the QCamera camera data packet in yuyv format into a first image data packet in RGB format. After receiving the transcoding operation signal, the data transcoding thread first determines whether transcoding operation is needed, and if so, extracts the QCamera camera data packet from the program running memory area for transcoding operation to obtain the first image data packet. Preferably, the sending of the transcoding operation signal and the extraction of the QCamera camera data packet by the data transcoding thread do not need to be synchronized, and will not cause blocking and waiting conditions. After the first image data packet is generated, the data transcoding thread sends a data processing signal to the data rendering thread.

[0035] Preferably, if the environmental resources (cpu and memory) are sufficient, the number of queues in the cache will be maintained at 1 data packet; if the system resources are insufficient, the data packets in the shared queue will be adjusted between 1 and a preset cache data amount, thereby automatically implementing a mechanism for efficiently and low-delay processing of data.

[0036] Preferably, the data transcoding thread copies at least one QCamera camera data packet from the end of the shared queue, the shared queue deletes the copied QCamera camera data packet, and the data transcoding thread performs transcoding processing on the copied QCamera camera data packet, which ensures that the data memory is not wasted and improves the efficiency of system operation.

[0037] S300, after receiving the data processing signal, the data rendering thread determines whether data processing operation is needed; if so, the first image data packet is transmitted to the data rendering thread for rendering to obtain a second image data packet, so as to obtain the data processed by the QCamera camera image data.

[0038] Preferably, the data processing signal is a signal generated by the data rendering thread by sending a refresh interface signal or directly calling an update UI interface processing function. The data transcoding thread triggers the data processing signal through the assembled Q Image object. Q Image is a memory object of Qt multimed ia about image data, and conversion into Q Image facilitates transmission and processing on the Qt multimed ia platform. When the data rendering thread receives the data processing signal and needs to perform data processing, the data rendering thread renders the first image data packet to obtain the second image data packet.

[0039] The result of the secondary processing of the camera data is updated on the UI interface. If the background image is updated first and then the objects drawn thereon are updated in a unit time, the time difference of the update will cause a visual flickering phenomenon, and therefore all the objects processed twice are first processed on the memory image and then updated once in a unit time on the interface.

[0040] In a specific implementation, the data drawing thread draws the first image data packet to obtain a second image data packet by using a UI double buffering mechanism. The principle of the double buffering drawing mechanism is to draw various objects in a virtual screen in memory and then copy them to an actual display screen to eliminate flicker, that is, the core drawing work is completed outside the screen, and when the screen is displayed, only a simple memory copy is performed. Alternatively, the memory of the second image data packet can be processed; or a mask layer can be overlaid on the image of the second image data packet for image processing.

[0041] If real-time labeling of camera data is required, the scene requires low latency and high smoothness. If all operations are concentrated in the data drawing thread, that is, the UI thread, it will cause serious lag and high latency. In the present application, the data capture thread is responsible for receiving data, the data transcoding thread is responsible for transcoding and assembling and processing image objects, and the data drawing thread is responsible for receiving user operations, such as drawing images by using a mouse, and simultaneously performing shared queue cache processing according to environmental resources, so as to achieve the goal of high efficiency and low latency.

[0042] Preferably, the present embodiment integrates the data capture thread, the data transcoding thread, and the data drawing thread in the Qt multimedia framework. When labeling objects on a data stream with a pixel of 1920*1080 30 frames / s, the latency of the present application is less than 300 ms, the smoothness is high, and the occupied memory is less than 120 MB.

[0043] Embodiment two

[0044] The present application also provides a processing device based on QCamera camera image data, as shown in Figure 2 The device includes a data capture thread module configured to acquire a QCamera camera data packet and store the QCamera camera data packet in a preset position, and send a transcoding operation signal to a data transcoding thread.

[0045] The data transcoding thread module is configured to determine whether transcoding operation is required after receiving the transcoding operation signal; if so, the data transcoding thread extracts the QCamera camera data packet from the preset position to perform transcoding operation and obtain a first image data packet, and sends a data processing signal to a data drawing thread.

[0046] The data drawing thread module is configured to determine whether data processing operation is required after receiving the data processing signal; if so, the first image data packet is transmitted to the data drawing thread to draw the first image data packet and obtain a second image data packet, so as to obtain data processed by the QCamera camera image data.

[0047] The device uses a Qt multimedia framework to integrate a data capture thread module, a data transcoding thread module and a data drawing thread module, and completes secondary processing of camera data. The data capture thread module is responsible for receiving data, the data transcoding thread module is responsible for transcoding and assembling and processing of image objects, and the data drawing thread module is responsible for receiving user operations, such as drawing an image through a mouse, and simultaneously performs shared queue cache processing according to environmental resources, so that the goal of high efficiency and low delay is achieved.

[0048] Embodiment three

[0049] The application further provides a computer readable storage medium, which stores computer instructions, and a computer is executed by a processor to realize the processing method based on QCamera camera image data according to any one of the above embodiments.

[0050] In specific implementation, the computer readable storage medium is a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; and the computer readable storage medium can further include a combination of the above types of memories.

[0051] Embodiment four

[0052] The application further provides an electronic device, such as Figure 3 As shown, the electronic device includes at least one processor and a memory connected to the processor in communication, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the processor to execute the processing method based on QCamera camera image data according to any one of the above embodiments.

[0053] In practice, the number of processors can be one or more, and the processor can be a central processing unit (CPU). The processor can also be other general purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof. The general purpose processor can be a microprocessor or the processor can be any conventional processor.

[0054] The memory and the processor can be connected by a bus or other means, and the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the processor to perform the processing method based on the QCamera camera image data as in the above embodiments.

[0055] In addition, those skilled in the art should understand that although there are many problems in the prior art, each embodiment or technical solution of the present application can only improve in one or several aspects, and it is not necessary to solve all the technical problems listed in the prior art or background art at the same time. Those skilled in the art should understand that what is not mentioned in a claim should not be regarded as a limitation on the claim.

[0056] Although terms such as data capture thread, QCamera camera data packet, data transcoding thread, first image data packet, data rendering thread, and second image data packet are used more frequently herein, the possibility of using other terms is not excluded. The use of these terms is only to facilitate the description and explanation of the essence of the present application; any additional limitation by interpreting them is contrary to the spirit of the present application; the terms "first", "second", etc. (if any) in the specification and claims of the embodiments of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0057] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A processing method based on QCamera camera image data, characterized in that, The application relates to a method for processing QCamera camera image data, and a device thereof. S100, a data capture thread acquires QCamera camera data packets and stores the QCamera camera data packets to a preset position, and sends a transcoding operation signal to a data transcoding thread; S200, after receiving the transcoding operation signal, the data transcoding thread judges whether transcoding operation is needed; if needed, the data transcoding thread extracts the QCamera camera data packets from the preset position to perform transcoding operation and obtains first image data packets, and sends a data processing signal to a data drawing thread; S300, after receiving the data processing signal, the data drawing thread judges whether data processing operation is needed; if needed, the first image data packets are transmitted to the data drawing thread to draw second image data packets, so as to obtain processed data of the QCamera camera image data.

2. The method of claim 1, wherein, The data capture thread acquires the QCamera camera data packets by using a registration callback mechanism.

3. The method of claim 1, wherein, The data capture thread stores the acquired QCamera camera data packets to a program running memory area, and the data transcoding thread extracts the QCamera camera data packets from the program running memory area to perform transcoding operation.

4. The method of claim 3, wherein, The latest QCamera camera data packets acquired in the data capture thread are cached to the front end of a shared queue, and when the QCamera camera data packets in the shared queue reach a preset cache data volume, the old QCamera camera data packets are discarded from the tail end of the shared queue.

5. The method of claim 1, wherein, The data transcoding thread converts the QCamera camera data packets in yuyv format into the first image data packets in RGB format.

6. The method of claim 4, wherein, The data transcoding thread copies at least one QCamera camera data packet from the tail end of the shared queue, the shared queue deletes the copied QCamera camera data packet, and the data transcoding thread performs transcoding processing on the copied QCamera camera data packet.

7. The method of claim 1, wherein, The data drawing thread adopts a UI double buffering mechanism to draw the first image data packets to obtain the second image data packets.

8. A processing device based on QCamera camera image data, characterized by, The application relates to a method for processing QCamera camera image data, and a device thereof. The data capture thread acquires QCamera camera data packets and stores the QCamera camera data packets to a preset position, and sends a transcoding operation signal to a data transcoding thread; S200, after receiving the transcoding operation signal, the data transcoding thread judges whether transcoding operation is needed; if needed, the data transcoding thread extracts the QCamera camera data packets from the preset position to perform transcoding operation and obtains first image data packets, and sends a data processing signal to a data drawing thread; S300, after receiving the data processing signal, the data drawing thread judges whether data processing operation is needed; if needed, the first image data packets are transmitted to the data drawing thread to draw second image data packets, so as to obtain processed data of the QCamera camera image data.

9. A computer-readable storage medium, characterized in that: The computer readable storage medium stores computer instructions, and the computer is configured to implement the processing method based on the QCamera camera image data according to any one of claims 1-7 when the computer instructions are executed by a processor.

10. An electronic device, comprising: A device comprising at least one processor and a memory connected to the processor in communication, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the processor to perform the processing method based on the QCamera camera image data according to any one of claims 1-7.

Images