Image rendering method, device, equipment and medium in live camera

Abstract

The present application is applicable to the field of image processing technology, and in particular to an image rendering method, device, equipment and medium in a live camera. The method sends the pixel format data of the current frame image to the model detection thread, and sends the copied corresponding texture format data to the rendering thread. The model detection thread processes the current frame image, and sends the pixel format data corresponding to the next frame image to the model detection thread. After waiting for the model detection thread to output the face detection result of the current frame image, the pixel format data of the next frame image is detected and processed, and the face detection result is sent to the rendering thread. The rendering thread renders the texture format data and the face detection result. The current frame image and the next frame image enter the model detection thread in turn without waiting for the rendering thread to complete the processing, thereby shortening the execution time, and the rendering thread waits for the model detection thread to output the face detection result to ensure the synchronization of face detection and rendered image frames.

Description

Technical Field

[0001] The present application is applicable to the field of image processing technology, and in particular, relates to an image rendering method, device, equipment and medium in a live camera. Background Art

[0002] At present, the beauty functions mainly include skin beautification (whitening, tanning, skin smoothing, etc.), makeup (lipstick, blush, eye shadow, etc.), shape beautification (big eyes, thin face, thin nose, etc.), body beautification, and style makeup (a combination of multiple beauty makeup), etc. The realization of these functions is mainly based on the input YUV image to perform model detection, spit out the detection results, and then render according to the spit out results and the texture of the same frame passed in from the upper layer.

[0003] In the prior art, to ensure that the YUV image input to the model detection and the texture for subsequent rendering are the same frame, the model detection and beauty rendering are performed serially. However, when multiple beauty functions are enabled, different models are required for detection and then rendering. At this time, the original serial design scheme will add the time consumption of the corresponding models and rendering of multiple functions, resulting in a significant performance degradation. The more faces there are, the longer the model detection takes, and the more severe the performance degradation.

[0004] Therefore, how to avoid processing blockage caused by serialization while ensuring the synchronization of rendering frames when superimposing multiple functional renderings becomes an urgent problem to be solved. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a method, apparatus, device and medium for image rendering in a live camera to solve the problem of how to avoid processing blockage caused by serialization while ensuring the synchronization of rendering frames when superimposing multiple function renderings.

[0006] In a first aspect, an embodiment of the present application provides a method for rendering an image in a live camera, wherein the method is applied to an image rendering system consisting of at least a model detection thread and a rendering thread, and includes:

[0007] When the current frame image is acquired at the current moment, the pixel format data of the current frame image is sent to the model detection thread, and the texture format data corresponding to the copied current frame image is sent to the rendering thread, the model detection thread is used to detect and process the input pixel format data and output the face detection result when the model detection thread is available, and the model detection thread is unavailable before the face detection result is output;

[0008] When a next frame of image is acquired at the next moment, the pixel format data of the next frame of image is sent to the model detection thread. If the model detection thread is unavailable, the pixel format data of the next frame of image is detected and processed when the model detection thread is available.

[0009] If the face detection result output by the model detection thread is obtained after waiting, the model detection thread is determined to be available, and the face detection result is sent to the rendering thread, and the rendering thread is used to render the texture format data and the face detection result;

[0010] After the rendering process is completed in the rendering thread, the next frame image is used as the current frame image, and the process of sending the copied texture format data corresponding to the current frame image to the rendering thread is returned.

[0011] In a second aspect, an embodiment of the present application provides an image rendering device in a live camera, wherein the image rendering device in a live camera is applied to an image rendering system consisting of at least a model detection thread and a rendering thread, including:

[0012] The current frame image processing module is used to send the pixel format data of the current frame image to the model detection thread when the current frame image is acquired at the current moment, and send the texture format data corresponding to the copied current frame image to the rendering thread. The model detection thread is used to detect and process the input pixel format data and output the face detection result when the model detection thread is available. The model detection thread is unavailable before the face detection result is output;

[0013] A next frame image processing module is used to send the pixel format data of the next frame image to the model detection thread when the next frame image is acquired at the next moment, and if the model detection thread is unavailable, wait for the model detection thread to be available to perform detection processing on the pixel format data of the next frame image;

[0014] A rendering module, configured to determine that the model detection thread is available if a face detection result output by the model detection thread is obtained after waiting, and send the face detection result to the rendering thread, wherein the rendering thread is configured to render the texture format data and the face detection result;

[0015] The loop execution module is used to, after the rendering process in the rendering thread is completed, use the next frame image as the current frame image and return to execute the sending of the texture format data corresponding to the copied current frame image to the rendering thread.

[0016] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image rendering method in a live camera as described in the first aspect when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image rendering method in the live camera as described in the first aspect is implemented.

[0018] Compared with the prior art, the embodiments of the present application have the following beneficial effects: when the current frame image is acquired at the current moment, the present application sends the corresponding pixel format data to the model detection thread, and sends the copied corresponding texture format data to the rendering thread. The model detection thread is used to detect and process the input pixel format data when available and output the face detection result. The model detection thread is unavailable before the face detection result is output. When the next frame image is acquired at the next moment, the corresponding pixel format data is sent to the model detection thread. If the model detection thread is unavailable, the pixel format data of the next frame image is detected and processed when the model detection thread is available. If the face detection result output by the model detection thread is acquired after waiting, the model detection thread is determined to be available, and the face detection result is sent to the rendering thread. The rendering thread is used to render the texture format data and the face detection result. After the rendering process is completed in the rendering thread, the next frame image is used as the current frame image, and the execution is returned to send the copied corresponding texture format data to the rendering thread. Among them, the current frame image and the next frame image can enter the model detection thread in turn without waiting for the rendering thread to complete processing, that is, the next frame image does not need to wait for the current frame image to be rendered before executing detection, which can achieve parallelism to a certain extent, thereby shortening the execution time, and the rendering thread waits for the model detection thread to output the face detection result, ensuring the synchronization of face detection and rendered image frames. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 This is a schematic diagram of an application environment of an image rendering method in a live camera provided in Embodiment 1 of the present application;

[0021] Figure 2 It is a flowchart of an image rendering method in a live camera provided in Embodiment 2 of the present application;

[0022] Figure 3 This is a timing diagram of an image rendering method in a live camera provided in Embodiment 2 of the present application;

[0023] Figure 4 It is a structural schematic diagram of an image rendering device in a live camera provided in Embodiment 3 of the present application;

[0024] Figure 5 It is a structural diagram of a computer device provided in Example 4 of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0026] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0029] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0031] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0032] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0033] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0034] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0035] The image rendering method in a live camera provided in the first embodiment of the present application can be applied to Figure 1In the application environment, the corresponding application is started in the mobile terminal, and the application can call the camera, and the camera provides an image, which is input into the image rendering system in the application. The image rendering system includes detection and rendering. For example, for beauty, detection is to detect the face, and rendering is to whiten and smooth the face. Detection uses a model detection thread, and rendering uses a rendering thread. Both threads are independent threads. After rendering, the output is displayed on the display of the mobile terminal. Among them, the mobile terminal includes but is not limited to handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPC), netbooks, cloud computer devices, personal digital assistants (PDA) and other computer devices.

[0036] See also Figure 2 , is a flow chart of an image rendering method in a live camera provided in Embodiment 2 of the present application, wherein the image rendering method in the live camera is applied to Figure 1 Mobile terminals in. Figure 2 As shown, the image rendering method in the live camera may include the following steps:

[0037] Step S201, when a current frame image is acquired at the current moment, the pixel format data of the current frame image is sent to the model detection thread, and the texture format data corresponding to the copied current frame image is sent to the rendering thread.

[0038] The model detection thread is used to detect and process the input pixel format data and output the face detection result when the model detection thread is available, and the model detection thread is unavailable before the face detection result is output.

[0039] The model detection thread runs the face detection model, so that it can perform face detection on the image and obtain the face detection result, which can be used for subsequent beautification, etc.

[0040] The model detection thread is available when it is not processing any data. If the model detection thread is performing detection processing and has not yet output the face detection result, the model detection thread is unavailable.

[0041] The live camera can be understood as an application running on a mobile terminal such as a mobile phone. After the application is started, the user sets the desired rendering effect according to the needs, the image rendering system is started, and then the corresponding algorithm can be provided in the rendering process to achieve it. When the camera is turned on, an image enters the image rendering system, and the pixel format data corresponding to the image is sent to the model detection thread, and the texture format data corresponding to the image is sent to the rendering thread.

[0042] At this time, for the first frame image, the model detection thread and the rendering thread are both available, and the two data can be sent to the corresponding threads for processing at the same time. For the second frame image and subsequent images, the model detection thread and the rendering thread may be occupied, and it may not be possible to send the two data to the corresponding threads for processing at the same time.

[0043] Optionally, the pixel format data is YUV format data;

[0044] The image rendering method in the live camera also includes:

[0045] After acquiring the current frame image, duplicating the current frame image to obtain two current frame images;

[0046] Performing YUV format conversion on a current frame image to obtain YUV format data corresponding to the current frame image;

[0047] The texture format of another current frame image is converted to obtain texture format data corresponding to the current frame image.

[0048] Among them, YUV is a picture pixel arrangement format, and the texture format is not used for the image format of rendering. After the image is acquired, the image is directly copied to form two images, one is converted into YUV format data, and the other is converted into texture format data, which can help improve the processing efficiency of the thread.

[0049] Optionally, after the model detection thread is used to detect and process the input pixel format data and output the face detection result when the model detection thread is available, the method further includes:

[0050] Generate a detection completion notification;

[0051] When the model detection thread outputs the face detection result, the detection completion notification is output, where the detection completion notification is used to indicate that the model detection thread is available.

[0052] The state of the model detection thread is informed by generating a notification, and the detection completion notification can indicate that the model detection thread is available, and subsequent images can be sent to the model detection thread for detection processing.

[0053] Step S202, when the next frame image is obtained at the next moment, the pixel format data of the next frame image is sent to the model detection thread. If the model detection thread is unavailable, wait until the model detection thread is available to detect and process the pixel format data of the next frame image.

[0054] Among them, the next moment is the next sampling time point of the above current moment, and the interval between the two moments is related to the sampling frequency of the camera. When the time comes to the next moment, the next frame image is processed as the current frame, and the image of the previous frame has been distributed to the corresponding thread. The next frame image can get the corresponding pixel format data and send it to the model detection thread. Although it is sent to the model detection thread, if the model detection thread has not completed the detection task of the previous frame image, the model detection thread is unavailable and needs to wait for the model detection thread to be available before detection processing.

[0055] Step S203: If the face detection result output by the model detection thread is obtained after waiting, it is determined that the model detection thread is available, and the face detection result is sent to the rendering thread.

[0056] Among them, if the model detection thread outputs a face detection result for the current frame image, it indicates that the model detection thread is available, and the above-mentioned next frame image can be processed by the model detection thread without waiting for the rendering of the current frame image.

[0057] The face detection result is sent to the rendering thread, and the rendering thread is used to render the texture format data and the face detection result to achieve the rendering of the current frame image. Of course, after the rendering is completed, it can be output to the mobile terminal display or the network adapter of the mobile terminal, wherein the display is used to display the rendered image on the mobile terminal, and the network adapter is used to send the rendered image to the corresponding network node through the network so that other users can watch it.

[0058] Step S204, after the rendering process is completed in the rendering thread, the next frame image is used as the current frame image, and the process of sending the copied texture format data corresponding to the current frame image to the rendering thread is returned to be executed.

[0059] After the rendering process of the rendering thread is completed, the texture format data of the next frame image can be received, and the model detection thread can be waited for to output the detection result of the next frame image, so as to render the next frame image. Specifically, after the rendering process is completed, the next moment is taken as the current moment, that is, the next frame image is taken as the current frame image, and the step of copying the texture format data and sending it to the rendering thread is executed, so as to realize the beauty processing of the live video stream through a loop.

[0060] For example, Figure 3 As shown, it is a timing diagram of an image rendering method in a live camera provided in Embodiment 2 of the present application, and the specific timing process is as follows:

[0061] 1. After the current frame image enters, the YUV format data is copied to the task, and the texture format data is copied to the rendering thread. The task transfers the YUV format data to the model detection thread for detection, and the model detection thread performs detection processing until it spits out the detection results and notifications;

[0062] 2. The task returns, the next frame of image enters, and the YUV format data of the next frame of image is copied to the task, and the YUV format data of the next frame of image is not sent to the model detection thread temporarily;

[0063] 3. The task obtains the detection results and notifications spit out by the model detection thread, returns the detection results to the main line, and sends the YUV format data of the next frame of image to the model detection thread. The model detection thread performs detection processing until it spits out the detection results and notifications;

[0064] 4. The main line sends the detection result to the rendering thread. The rendering thread renders the detection result and the texture format data of the current frame image obtained above, obtains and outputs the rendering result. At this time, the texture format data of the next frame image can be sent to the rendering thread to realize the loop.

[0065] Optionally, the image rendering method in the live camera further includes:

[0066] Check if the image input is empty;

[0067] If it is detected that the image input is not empty, data is copied from the image input to obtain the current frame image.

[0068] Optionally, before copying data from the image input to obtain the current frame image, the method further includes:

[0069] Detecting whether the model detection thread is available;

[0070] The step of sending the pixel format data of the current frame image to the model detection thread includes:

[0071] If it is detected that the model detection thread is available, the pixel format data of the current frame image is sent to the model detection thread.

[0072] Here, the image input needs to be detected to determine whether there is an image therein, and whether the thread is available needs to be detected.

[0073] Optionally, the rendering thread is deployed in a GPU.

[0074] Optionally, the model detection thread is deployed in the DSP, the CPU or the GPU.

[0075] Among them, the above deployment can greatly improve the performance of the software without changing the current chip computing power, and can better explore the computing power boundaries of the current chip and make full use of the current chip computing power to meet product requirements.

[0076] In an embodiment of the present application, when the current frame image is acquired at the current moment, the corresponding pixel format data is sent to the model detection thread, and the copied corresponding texture format data is sent to the rendering thread. The model detection thread is used to detect and process the input pixel format data when available and output the face detection result. The model detection thread is unavailable before outputting the face detection result. When the next frame image is acquired at the next moment, the corresponding pixel format data is sent to the model detection thread. If the model detection thread is unavailable, the pixel format data of the next frame image is detected and processed when the model detection thread is available. If the face detection result output by the model detection thread is acquired after waiting, the model detection thread is determined to be available, and the face detection result is sent to the rendering thread. The rendering thread is used to render the texture format data and the face detection result. After the rendering process is completed in the rendering thread, the next frame image is used as the current frame image, and the execution is returned to send the copied corresponding texture format data to the rendering thread. Among them, the current frame image and the next frame image can enter the model detection thread in turn without waiting for the rendering thread to complete processing, that is, the next frame image does not need to wait for the current frame image to be rendered before executing detection, which can achieve parallelism to a certain extent, thereby shortening the execution time, and the rendering thread waits for the model detection thread to output the face detection result, ensuring the synchronization of face detection and rendered image frames.

[0077] Corresponding to the image rendering method in the live camera of the above embodiment, Figure 4 The structure block diagram of the image rendering device in the live camera provided by the third embodiment of the present application is shown. The image rendering device in the live camera is applied to Figure 1 For ease of description, only the parts related to the embodiment of the present application are shown.

[0078] See also Figure 4 , the image rendering device in the live camera includes:

[0079] The current frame image processing module 41 is used to send the pixel format data of the current frame image to the model detection thread when the current frame image is acquired at the current moment, and send the texture format data corresponding to the copied current frame image to the rendering thread. The model detection thread is used to detect and process the input pixel format data and output the face detection result when the model detection thread is available. The model detection thread is unavailable before the face detection result is output;

[0080] The next frame image processing module 42 is used to send the pixel format data of the next frame image to the model detection thread when the next frame image is acquired at the next moment, and if the model detection thread is unavailable, wait for the model detection thread to be available to perform detection processing on the pixel format data of the next frame image;

[0081] A rendering module 43 is used for determining that the model detection thread is available if a face detection result output by the model detection thread is obtained after waiting, and sending the face detection result to the rendering thread, wherein the rendering thread is used for rendering the texture format data and the face detection result;

[0082] The loop execution module 44 is used to, after the rendering process in the rendering thread is completed, use the next frame image as the current frame image and return to execute the sending of the texture format data corresponding to the copied current frame image to the rendering thread.

[0083] Optionally, the pixel format data is YUV format data;

[0084] The image rendering device in the live camera also includes:

[0085] An image duplication module is used to, after acquiring the current frame image, perform duplication processing on the current frame image to obtain two current frame images;

[0086] A YUV conversion module is used to perform YUV format conversion on a current frame image to obtain YUV format data corresponding to the current frame image;

[0087] The texture conversion module is used to perform texture format conversion on another current frame image to obtain texture format data corresponding to the current frame image.

[0088] Optionally, the image rendering device in the live camera further includes:

[0089] A notification generation module, configured to generate a detection completion notification after the model detection thread is used to perform detection processing on the input pixel format data and output the face detection result when the model detection thread is available;

[0090] A notification output module is used to output the detection completion notification when the model detection thread outputs the face detection result, and the detection completion notification is used to indicate that the model detection thread is available.

[0091] Optionally, the image rendering device in the live camera further includes:

[0092] Input detection module, used to detect whether the image input is empty;

[0093] The data copy module is used to copy data from the image input to obtain the current frame image if it is detected that the image input is not empty.

[0094] Optionally, the image rendering device in the live camera further includes:

[0095] A thread status detection module, used for detecting whether the model detection thread is available before copying data from the image input to obtain the current frame image;

[0096] The current frame image processing module 41 includes:

[0097] The current frame data sending unit is used to send the pixel format data of the current frame image to the model detection thread if it is detected that the model detection thread is available.

[0098] Optionally, the rendering thread is deployed in a GPU.

[0099] Optionally, the model detection thread is deployed in the DSP, the CPU or the GPU.

[0100] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0101] Figure 5 This is a schematic diagram of the structure of a computer device provided in Example 4 of the present application. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor, wherein when the processor executes the computer program, the steps in any of the above-mentioned embodiments of the method for rendering an image in a live camera are implemented.

[0102] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Figure 5 This is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0103] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0104] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments, it may also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory may also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0105] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0106] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing the computer program product.

[0107] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0108] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0109] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for rendering an image in a live camera, characterized in that: The method for rendering an image in a live camera is applied to an image rendering system consisting of at least a model detection thread and a rendering thread, and includes: When the current frame image is acquired at the current moment, the pixel format data of the current frame image is sent to the model detection thread, and the texture format data corresponding to the copied current frame image is sent to the rendering thread, the model detection thread is used to detect and process the input pixel format data and output the face detection result when the model detection thread is available, and the model detection thread is unavailable before the face detection result is output; When a next frame of image is acquired at the next moment, the pixel format data of the next frame of image is sent to the model detection thread. If the model detection thread is unavailable, the pixel format data of the next frame of image is detected and processed when the model detection thread is available. If the face detection result output by the model detection thread is obtained after waiting, the model detection thread is determined to be available, and the face detection result is sent to the rendering thread, and the rendering thread is used to render the texture format data and the face detection result; After the rendering process is completed in the rendering thread, the next frame image is used as the current frame image, and the process of sending the texture format data corresponding to the copied current frame image to the rendering thread is returned; The pixel format data is YUV format data; The image rendering method in the live camera also includes: After acquiring the current frame image, duplicating the current frame image to obtain two current frame images; Performing YUV format conversion on a current frame image to obtain YUV format data corresponding to the current frame image; Performing texture format conversion on another current frame image to obtain texture format data corresponding to the current frame image; After the model detection thread is used to detect and process the input pixel format data and output the face detection result when the model detection thread is available, it also includes: Generate a detection completion notification; When the model detection thread outputs the face detection result, the detection completion notification is output, where the detection completion notification is used to indicate that the model detection thread is available.

2. The method for rendering an image in a live camera according to claim 1, characterized in that: The image rendering method in the live camera also includes: Check if the image input is empty; If it is detected that the image input is not empty, data is copied from the image input to obtain the current frame image.

3. The image rendering method in a live camera according to claim 2, characterized in that: Before copying data from the image input to obtain the current frame image, the process further includes: Detecting whether the model detection thread is available; The step of sending the pixel format data of the current frame image to the model detection thread includes: If it is detected that the model detection thread is available, the pixel format data of the current frame image is sent to the model detection thread.

4. The method for rendering an image in a live camera according to any one of claims 1 to 3, characterized in that: The rendering thread is deployed in the GPU.

5. The method for rendering images in a live camera according to claim 4, characterized in that: The model detection thread is deployed in the DSP, the CPU or the GPU.

6. An image rendering device in a live camera, characterized in that: The image rendering device in the live camera is applied to an image rendering system consisting of at least a model detection thread and a rendering thread, and includes: The current frame image processing module is used to send the pixel format data of the current frame image to the model detection thread when the current frame image is acquired at the current moment, and send the texture format data corresponding to the copied current frame image to the rendering thread. The model detection thread is used to detect and process the input pixel format data and output the face detection result when the model detection thread is available. The model detection thread is unavailable before the face detection result is output; A next frame image processing module is used to send the pixel format data of the next frame image to the model detection thread when the next frame image is acquired at the next moment, and if the model detection thread is unavailable, wait for the model detection thread to be available to perform detection processing on the pixel format data of the next frame image; A rendering module, configured to determine that the model detection thread is available if a face detection result output by the model detection thread is obtained after waiting, and send the face detection result to the rendering thread, wherein the rendering thread is configured to render the texture format data and the face detection result; A loop execution module, configured to, after the rendering process in the rendering thread is completed, use the next frame image as the current frame image, and return to execute the process of sending the texture format data corresponding to the copied current frame image to the rendering thread; The pixel format data is YUV format data; The image rendering device in the live camera also includes: An image duplication module is used to, after acquiring the current frame image, perform duplication processing on the current frame image to obtain two current frame images; A YUV conversion module is used to perform YUV format conversion on a current frame image to obtain YUV format data corresponding to the current frame image; A texture conversion module, used for performing texture format conversion on another current frame image to obtain texture format data corresponding to the current frame image; The image rendering device in the live camera also includes: A notification generation module, configured to generate a detection completion notification after performing detection processing on the input pixel format data and outputting the face detection result when the model detection thread is available; A notification output module is used to output the detection completion notification when the model detection thread outputs the face detection result, and the detection completion notification is used to indicate that the model detection thread is available.

7. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the image rendering method in a live camera as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image rendering method in a live camera according to any one of claims 1 to 5 is implemented.

Images