Image processing method and device, electronic equipment and readable storage medium

Abstract

The invention discloses an image processing method and device, electronic equipment and a readable storage medium, and belongs to the technical field of image processing. In the process of processing the image stream, first main body position information of a first image in the image stream is transmitted through a first channel, a second image corresponding to the first image and a first processing parameter are transmitted through a second channel, and the first image is any image in the image stream; wherein the second image is a result image obtained by processing the first image according to the first processing parameter; the first subject position information, the second image and the first processing parameter are all associated with the image identifier of the first image. And the electronic equipment determines the position information of the main body in the second image based on the first main body position information and the first processing parameter.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method, device, electronic device and readable storage medium. Background Art

[0002] With the continuous development of terminal technology, users have higher and higher requirements on the performance of electronic devices. For example, users need electronic devices to accurately capture the position of the subject in the image.

[0003] In the related art, in the process of processing an image stream, the electronic device can detect the subject position information of the subject in the image through the subject detection module, and transmit the detected subject position information through the image processing path. Specifically, the subject position information and the global processing parameters corresponding to the image can be packaged in a metadata package for transmission, and the global processing parameters corresponding to the image are the global processing parameters used by each processing module in the image processing path to process the image. Then, the electronic device can capture the position of the subject in the image (hereinafter referred to as the target image) processed by the image processing path in the application layer according to the subject position information and the global processing parameters in the metadata package corresponding to the image, for example, first calibrate the subject position information through the global processing parameters in the metadata package, and then determine the position of the subject in the target image through the calibrated position information.

[0004] However, since the subject detection module and the image processing pathway work in parallel, and it takes a certain amount of time for the subject detection module to detect a subject's position information, it may result in the subject position information carried in the metadata package corresponding to each image being the subject position information corresponding to at least one image located before the image, resulting in the inability to accurately capture the position of the subject in the image. Summary of the invention

[0005] The purpose of the embodiment of the present application is to provide an image processing method that can accurately capture the position of a subject in an image.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising: in the process of processing an image stream, transmitting first subject position information of a first image in the image stream through a first path, and transmitting a second image and a first processing parameter corresponding to the first image through a second path, the first image being any image in the image stream; wherein the second image is a result image obtained by processing the first image according to the first processing parameter; the first subject position information, the second image and the first processing parameter are all associated with an image identifier of the first image. The electronic device determines the position information of the subject in the second image based on the first subject position information and the first processing parameter.

[0007] In a second aspect, an embodiment of the present application provides an image processing device, which includes: a transmission module, which is used to transmit the first subject position information of the first image in the image stream through a first path during processing of the image stream, and transmit the second image and the first processing parameter corresponding to the first image through a second path, and the first image is any image in the image stream. The second image is the result image obtained by processing the first image according to the first processing parameter; the first subject position information, the second image and the first processing parameter are all associated with the image identifier of the first image. A determination module is used to determine the position information of the subject in the second image transmitted by the transmission module based on the first subject position information and the first processing parameter transmitted by the transmission module.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the above-mentioned processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, in the process of processing an image stream, since the subject position information of the subject in each image can be transmitted through a first path, and the processing parameters corresponding to each image and the processed result image, such as the second image, can be transmitted through a second path; that is, the subject position information of the subject in the image does not need to be transmitted along with the processing parameters corresponding to the image, and the subject position information, result image and processing parameters of each image are all associated with the image identifier of the image, it can be ensured that the electronic device uses the correct subject position information to determine the position of the subject in the corresponding result image, thereby improving the accuracy of the electronic device in determining the position of the subject in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of a system framework for applying the image processing method provided in an embodiment of the present application;

[0014] Figure 2It is a flowchart of an image processing method provided in an embodiment of the present application;

[0015] Figure 3 is a structural schematic diagram of an image signal processing path applied by an image processing method provided in an embodiment of the present application;

[0016] Figure 4 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0017] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0018] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0020] The following first explains the nouns or terms in the embodiments of the present application.

[0021] System on Chip (SoC) is the overall control and processing module in electronic devices.

[0022] Image Signal Processor (ISP) is used to process the image signal output by the image signal sensor. ISP plays a core role in the camera system and is an important device in the camera. ISP completes the effect processing of digital images through a series of digital image processing algorithms.

[0023] Metadata is a concept in Android that is used to convey and describe additional information about a specific resource or component.

[0024] Application Program Interface (API), an application program interface is a collection of definitions, programs and protocols that enable mutual communication between computer software through the API interface.

[0025] Application framework layer (FrameWork): The application framework layer includes a series of APIs and tools provided to developers to implement and call system-level functions and features; the application framework layer is composed of various service and management modules.

[0026] The application layer (Application, APP) is composed of various applications installed on electronic devices, which are directly facing users and meet their different needs.

[0027] Image Front End (IFE), IFE is a hardware processing unit in ISP, responsible for receiving raw data from the image sensor and performing preliminary processing.

[0028] Image Process Engine (IPE), IPE is a hardware processing unit in ISP, which mainly performs secondary processing on the image processed by IFE, such as denoising, sharpening, cropping, color adjustment, etc. Among them, ISP can include multiple image processing engines, for example, it can include parallel IPE0 and IPE19.

[0029] For example, a face detection algorithm may be set in IPE19 to detect subject position information of a subject in an image through IPE19; for example, IPE19 may be used to perform processing such as cropping, rotating, scaling (interpolation), etc. on an image processed by IFE.

[0030] Enhanced Video Analytics Engine (EVA): EVA is a processing module in ISP. EVA is mainly aimed at computer vision applications. It can provide real-time image processing decision-making capabilities with stronger performance and lower latency, thereby saving bandwidth and reducing power consumption for digital signal processors (DSP), graphics processors (GPU), and central processing units (CPU) used in other artificial intelligence (AI) applications.

[0031] Image sensor (Sensor), Sensor is one of the core components of the camera, responsible for converting the light entering through the lens into electrical signals to form a digital image.

[0032] The display panel (Panel) is used to display the image captured by the camera or the real-time view or preview image of the camera.

[0033] Android Interface Definition Language (Aidl), a language mechanism for implementing inter-process communication.

[0034] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0035] The electronic device in the embodiment of the present invention may be an electronic device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present invention.

[0036] The following takes the Android operating system as an example to introduce the software environment used by the image processing method provided in the embodiment of the present invention.

[0037] like Figure 1 FIG. 1 is a schematic diagram of a possible Android operating system architecture provided by an embodiment of the present invention. Figure 1 The architecture of the Android operating system includes four layers: application layer, application framework layer, system runtime layer and kernel layer (specifically, the Linux kernel layer).

[0038] Among them, the application layer includes various applications in the Android operating system (including system applications and third-party applications).

[0039] The application framework layer is the framework of the application. Developers can develop some applications based on the application framework layer while complying with the development principles of the application framework.

[0040] The system runtime layer includes libraries (also called system libraries) and the Android operating system runtime environment. The libraries mainly provide the Android operating system with various resources it needs. The Android operating system runtime environment is used to provide a software environment for the Android operating system.

[0041] The kernel layer is the operating system layer of the Android operating system and belongs to the bottom layer of the Android operating system software hierarchy. The kernel layer provides core system services and hardware-related drivers for the Android operating system based on the Linux kernel.

[0042] Taking the Android operating system as an example, in the embodiment of the present application, developers can Figure 1The system architecture of the Android operating system shown in FIG. 1 is used to develop a software program for implementing the image processing method provided by an embodiment of the present invention, so that the image processing method can be based on Figure 1 The Android operating system shown is running. That is, the processor or electronic device can implement the image processing method provided by the embodiment of the present invention by running the software program in the Android operating system.

[0043] The image processing method, device, electronic device and readable storage medium provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0044] An image processing method, device, electronic device and readable storage medium provided in the embodiments of the present application can be applied to electronic devices that need to mark the subject in the image (such as marking the subject in the monitoring screen) or optimize it, or need to process the background in the image, or in any scenario such as autofocus and automatic exposure that needs to identify and capture the position of the subject in the image, and the present application does not limit it.

[0045] In a possible application scenario, taking the automatic focus of an electronic device as an example, after the image sensor outputs a preview image, the electronic device can process the image in a parallel manner, that is, obtain the subject position information of the subject in the preview image and process the preview image, such as cropping or scaling, etc. Then the electronic device can transmit the subject position information to the application layer through one path, and transmit the processed preview image and processing parameters to the application layer through another path, and each subject position information, each processing parameter and each result preview image are associated with the image identifier of the original preview image. In this way, in the process of processing the preview image stream, since the subject position information of the subject in each preview image can be transmitted through the first path, and the processing parameters corresponding to each preview image and the preview image obtained after processing, such as the second preview image, can be transmitted through the second path, that is, the subject position information of the preview image does not need to be transmitted along with the processing parameters corresponding to the preview image, it is possible to avoid carrying erroneous subject position information in the processing parameters corresponding to the preview image, thereby avoiding the electronic device using erroneous subject position information to identify the position information of the subject in the processed preview image, thereby improving the accuracy of subject position recognition.

[0046] In some embodiments of the present application, after determining the processed preview image, that is, the position information of the subject in the second image, the electronic device can display the second image on a display screen and display a focus mark on the second image based on the position information of the subject in the second image.

[0047] In a possible application scenario, taking the case where a user needs to use an electronic device to complete face recognition verification during exercise as an example, when the image sensor outputs a face image, the electronic device can process the image in a parallel manner, that is, obtain the subject position information of the subject in the face image and process the face image, such as cropping or scaling, etc., and then the electronic device can transmit the subject position information to the application layer through one path, and transmit the processed face image and processing parameters to the application layer through another path, and each subject position information, each processing parameter and each result face image are associated with the image identifier of the original face image. In this way, in the process of processing the face image stream, since the subject position information of the subject in each face image can be transmitted through the first path, and the processing parameters corresponding to each face image and the processed face image such as the second face image can be transmitted through the second path, that is, the subject position information of the face image does not need to be transmitted along with the processing parameters corresponding to the face image, the face position information can be accurately matched with the face image, so that the face display effect is better and the face recognition verification is faster and more accurate.

[0048] In a possible application scenario, taking the case where a user needs to use an electronic device to shoot a video of an animal running, when the image sensor outputs an animal image, the electronic device can process the image in a parallel manner, that is, obtain the subject position information of the subject in the animal image and process the animal image, such as cropping or scaling, etc., and then the electronic device can transmit the subject position information to the application layer through one path, and transmit the processed animal image and processing parameters to the application layer through another path, and each subject position information, each processing parameter and each result animal image are associated with the image identifier of the original animal image. In this way, in the process of processing the animal image stream, since the subject position information of the subject in each animal image can be transmitted through the first path, and the processing parameters corresponding to each animal image and the processed animal image such as the second animal image can be transmitted through the second path, that is, the subject position information of the animal image does not need to be transmitted along with the processing parameters corresponding to the animal image, the animal position information can be accurately matched with the animal image, so that the display effect of the animal running is better, that is, the running action of the animal in each frame of the picture can be clearly displayed.

[0049] In a possible application scenario, taking the example that a user needs an electronic device to beautify a photographed subject, when an image sensor outputs a person image, the electronic device can process the image in a parallel manner, that is, obtain the subject position information of the subject in the person image and process the person image, such as cropping or scaling, etc., and then the electronic device can transmit the subject position information to the application layer through one path, and transmit the processed person image and processing parameters to the application layer through another path, and each subject position information, each processing parameter and each result person image are associated with the image identifier of the original person image. In this way, in the process of processing the person image stream, since the subject position information of the subject in each person image can be transmitted through the first path, and the processing parameters corresponding to each person image and the person image obtained after processing, such as the second person image, can be transmitted through the second path, that is, the subject position information of the person image does not need to be transmitted along with the processing parameters corresponding to the person image, so the person position information can be accurately matched with the person image, so that the display effect of the person is better and the beauty preview effect of the person photo is better.

[0050] The image processing method provided in the embodiment of the present application can be executed by an electronic device or at least one of a functional module and an entity module in the electronic device that can implement the image processing method. The specific execution can be determined according to actual use requirements and is not limited in the embodiment of the present application. In the embodiment of the present application, an electronic device executing the image processing method is taken as an example to illustrate the image processing method provided in the embodiment of the present application.

[0051] The present application embodiment provides an image processing method, Figure 2 A schematic diagram of the process of the image processing method provided in the embodiment of the present application is shown as follows: Figure 2 As shown, the image processing method provided in the embodiment of the present application may include step 101 and step 102.

[0052] Step 101: When processing an image stream, the electronic device transmits first subject position information of a first image in the image stream to an application layer through a first path, and transmits a second image and a first processing parameter corresponding to the first image to the application layer through a second path.

[0053] The first image is any image in the above-mentioned image stream, the second image is a result image obtained by processing the first image according to the first processing parameter; the first subject position information, the second image and the first processing parameter are all associated with the image identifier of the first image.

[0054] In some embodiments of the present application, the above-mentioned image stream may include any of the following: an image stream collected in real time by a camera of the electronic device, and an image stream in a video file saved in the electronic device.

[0055] In some embodiments of the present application, the first subject position information of the first image may include subject position information of at least one subject in the first image.

[0056] For example, when the first image includes multiple subjects, the electronic device can obtain subject position information of each of the multiple subjects to obtain first subject position information, or the electronic device can identify subject position information of a subject among the multiple subjects that meets the first condition.

[0057] The first condition may include at least one of the following: subject size, subject type.

[0058] The subject type may be any of the following: animals, people, plants, etc.

[0059] In some embodiments of the present application, the first subject position information may include at least one of the following: position information of the subject outline in the first image, and position information of key points of the subject in the first image.

[0060] The position information of the subject contour may be: the position information of the circumscribed regular figure of the subject, such as the position information of a rectangular frame that can frame the subject; or the position information of all positions constituting the subject contour, that is, the position information of the contour line of the subject contour.

[0061] In some embodiments of the present application, subject key points may be used to indicate subject type, posture, shape, or expression, etc.

[0062] For example, assuming that the subject in the image is a human face, the key points of the subject may include but are not limited to at least one of the following: eyes, nose, mouth, eyebrows, ears, etc.

[0063] In some embodiments of the present application, the first processing parameter may include a global processing parameter for performing global processing on the first image, such as a cropping parameter for cropping the first image or a filter parameter for adding a filter to the first image.

[0064] In some embodiments of the present application, the first path is different from the second path.

[0065] Specifically, Figure 3 As shown, the first path 31 can be a transmission path between the first processing engine 304 in the ISP path 305 and the application framework layer 306. The first processing engine 304 can be used to detect the subject position information of the subject in the image. The electronic device can first detect the subject position information of the subject in the image through the first processing engine 304, and then transmit the subject position information to the application framework layer 306 through the first path 31. For example, the first processing engine can be IPE19 in the ISP.

[0066] The second path 32 may be a path between the second processing engine 303 and the application framework layer 306 in the ISP path. The second processing engine 303 may be used to process an image. The electronic device may first process the image through the second processing engine 303 to obtain a result image and processing parameters corresponding to the image. Then, the processing parameters used to process the image and the processed result image are transmitted to the application framework layer 306 through the second path 32.

[0067] Among them, Figure 3 As shown, the second processing engine may include IPE0 and EVA connected in ISP path 305.

[0068] It should be noted that the first processing engine and the second processing engine are performed synchronously without interfering with each other.

[0069] It should be noted that since the speed at which the first processing engine detects the subject's position information is greater than the speed at which the second processing engine processes the image, and the data size of the subject position information is smaller than the data size of the processing result image and the data size of the processing parameters, for each image in the image stream, the subject position information of an image arrives at the Framework layer earlier than the time when the result image and processing parameters of the image arrive at the Framework layer.

[0070] In some embodiments of the present application, a subject detection algorithm may be set in the first processing engine, so that the electronic device can detect subject position information of the subject in the image based on the subject detection algorithm.

[0071] In some embodiments of the present application, the second image is a result image obtained by processing the first image according to the first processing parameters. For example, the second image may be an image obtained by processing the first image by the second processing engine using the first processing parameters.

[0072] In some embodiments of the present application, Figure 3 As shown, the ISP path also includes an image front-end module 302 connected to the image sensor 301, wherein the output end of the image front-end module 302 is respectively connected to the input end of the first processing engine 304 and the input end of the second processing engine 303. Before the electronic device processes the first image through the second processing engine and obtains the first subject position information of the subject in the first image through the first processing engine, it can pre-process the first image through the image front-end module in the ISP path.

[0073] In some embodiments of the present application, the first processing parameters may include at least part of the processing parameters used or generated by the second processing engine to process the first image. For example, the first processing parameters may include the processing parameters used by each processing module in the second processing engine, such as EVA and IPE0, to process the first image.

[0074] In some embodiments of the present application, the first processing parameters may also include parameters used by the above-mentioned front-end module to pre-process the first image.

[0075] In some embodiments of the present application, the first processing parameter may be at least one of the following: cropping information of the first image, such as cropping coordinates for cropping the first image, a zoom factor for the first image, and a rotation angle for the first image.

[0076] The rotation angle may be a two-dimensional rotation or a three-dimensional rotation of the first image.

[0077] In some embodiments of the present application, each image in the image stream has or corresponds to an image identifier, and the image identifier is used to indicate identity information of each image in the image stream. For example, the image identifier of the first image may indicate the first image.

[0078] In some embodiments of the present application, the first subject position information, the second image, and the first processing parameter are all associated with the image identifier of the first image, which can be understood as: the electronic device binds the first subject position information, the second image, and the first processing parameter to the image identifier of the first image for transmission. Among them, the image identifier of the image may include: the frame number of the image, and the timestamp when the image sensor captures the image. For the description of the association between the second image, the first processing parameter, and the image identifier of the first image, refer to the relevant description of the association between the first subject position information and the image identifier of the first image, which will not be repeated here.

[0079] For example, the first subject position information and the image identifier of the first image are carried in the same data packet for transmission, that is, the image identifier of the first image is carried in the data packet transmitting the first subject position information.

[0080] For example, the image identifier of the 5th frame image output by the image sensor is the number 5. Then, after the subject position detection of the subject of the 5th frame image, the subject position information of the subject of the 5th frame image can be associated with the number 5, and the number 5 is transmitted to the Framework layer together with the subject position information of the subject of the 5th frame image.

[0081] For example, the electronic device may package the first processing parameter and the image identifier of the first image into a data packet, and then transmit the data packet through the second path.

[0082] For example, the first processing parameter of the first image and the image identifier of the first image may be packaged in a data packet, and then the data packet is transmitted through the second path.

[0083] For example, the second image may be associated with the image identifier of the first image, and then the second image and the image identifier of the first image are transmitted through the second path.

[0084] In some embodiments of the present application, the electronic device may transmit the second image and the first processing parameter respectively through the second path.

[0085] For example, the electronic device may carry the image of the second image and the first processing parameter in two data packets, respectively, wherein the data packet carrying the first processing parameter may be a metadata packet.

[0086] Step 102: The electronic device determines position information of a subject in a second image based on the first subject position information and a first processing parameter.

[0087] In some embodiments of the present application, the electronic device determines the location information of the subject in the second image for at least one of the following purposes:

[0088] 1) The auxiliary camera focuses on the subject corresponding to the main subject;

[0089] 2) Assist the camera to perform automatic exposure;

[0090] 3) performing identity recognition on the subject in the second image;

[0091] 4) Optimizing the subject in the second image;

[0092] 5) Processing the background in the second image, such as blurring;

[0093] 6) Rapidly identify the type of the subject in the second image to achieve rapid image classification.

[0094] In some embodiments of the present application, optimization processing of the subject in the second image may include but is not limited to at least one of the following: beautification processing of the subject, beauty processing of the subject, anti-shake processing of the subject, and focusing processing of the subject.

[0095] In some embodiments of the present application, the electronic device determines the position information of the subject in the second image based on the first subject position information and the first processing parameter, which may include:

[0096] The electronic device can correct the first subject position information according to the first processing parameter to obtain the second subject position information so that the second subject position information matches the second image. Then, the electronic device can determine the position information of the subject in the second image according to the second subject position information.

[0097] Exemplarily, assuming that the first processing parameter includes a zoom factor for the first image, the electronic device may zoom the first subject position information by the same factor according to the zoom factor to obtain the second subject position information.

[0098] For example, assuming that the reduction factor in the first processing parameter is 2 times, and the first subject position information is the coordinate information of the subject outline in the first image, then: the electronic device can reduce each coordinate information in the coordinate information of the subject outline by two times.

[0099] It should be noted that the purpose of the electronic device correcting the first subject position information based on the first processing parameter is to align the coordinate system of the corrected subject position information with the coordinate system of the second image, so as to ensure that the image area where the subject is located in the second image can be accurately determined through the subject position information.

[0100] In some embodiments of the present application, the first path is a path between the first processing engine and the application framework layer, and the first processing engine can be used to detect the subject position information of the subject in the image. The "transmitting the first subject position information of the first image in the image stream through the first path" in the above step 101 can be implemented by the following steps A and B.

[0101] Step A: The electronic device transmits first subject position information to the application framework layer through a first path.

[0102] Step B: The electronic device transmits the first subject location information to the application layer through the third path.

[0103] The third path is a path between the application framework layer and the application layer. For example, the third path can be Figure 3 Passage 33 is shown.

[0104] In some embodiments of the present application, after detecting the first subject position information, the electronic device may transmit the detected first subject position information to the Framework layer through the first path.

[0105] In some embodiments of the present application, the first path can be implemented by using an Aidl interface between the first processing engine and the Framework layer, and this interface can be used for transmitting the first subject location information. Alternatively, the first subject location information can be carried by the Aidl interface.

[0106] In some embodiments of the present application, the electronic device may store or cache each subject location information in the form of a subject location information queue in the application framework layer.

[0107] In this way, since the subject position information of the subject in the image can be sent directly to the application framework layer through the first path, compared with the transmission method in the related art in which the subject position information of the subject is carried in the second path and continuously read and written, the image processing method provided in the embodiment of the present application can, on the one hand, reduce the amount of calculation in the second path; on the other hand, it can avoid carrying subject position information that does not match the image and processing parameters in the second path, thereby avoiding the electronic device from using erroneous subject position information to determine the subject position of the subject in the corresponding result image, and further enabling the electronic device to more accurately determine the position of the subject in the image.

[0108] In some embodiments of the present application, the above step B can be implemented by the following step B1.

[0109] Step B1: When the fourth image transmitted through the second path reaches the application framework layer, the electronic device transmits the first N subject position information in the subject position information queue to the application layer through the third path.

[0110] Among them, the above-mentioned first N subject position information may include the first subject position information, the fourth image is the result image corresponding to the fifth image, the fifth image is an image located before the first image in the image stream or is the first image, and the subject position information in the subject position information queue is arranged in the order of the time when the subject position information arrives at the application framework layer, and N can be a positive integer.

[0111] In some embodiments of the present application, the fourth image is an image obtained by processing the fifth image according to the fifth processing parameters. For example, the fourth image may be an image obtained by processing the fifth image by the second processing engine using the fifth processing parameters.

[0112] In some embodiments of the present application, the number of images between the fifth image and the first image may be related to the processing time of the second processing engine for processing a single image and the value of N.

[0113] The shorter the processing time of the second processing engine is, the fewer the number of images is; and the smaller the value of N is, the fewer the number of images is.

[0114] For example, assuming that the processing time of the second processing engine for processing one image is 0.1 ms, and the subject detection time of the subject in the first path is 0.05 ms, then:

[0115] One possible transmission method is that if N=2, after the result image of the first frame reaches the Framework layer, the Framework layer may output to the application layer the subject position information of the first and second frames, and when the fifth frame reaches the Framework layer, the Framework layer may output to the application layer the subject position information of the ninth and tenth frames;

[0116] Another possible transmission method is that if N=3, after the first frame image reaches the Framework layer, the Framework layer may output to the application layer the subject position information of the first to third frames of image; when the fifth frame image reaches the Framework layer, the Framework layer may output to the application layer the subject position information of the 13th to 15th frames of image.

[0117] It should be noted that, in the above embodiment, the subject position information is stored in the form of a subject position information queue in the Framework layer, and each result image reaching the Framework layer is used as an example to illustrate the timing of sending part of the subject position information in the subject position information queue. In actual implementation, the electronic device can also immediately send the first subject position information to the application layer through the third path after each subject position information is transmitted to the Framework layer, that is, the Framework layer sends the subject position information to the application layer every time it receives a subject position information.

[0118] It should be noted that after the electronic device transmits the first N subject location information in the subject location information queue to the application layer, the electronic device may delete the first N subject location information in the subject location information queue to update the subject location information queue.

[0119] It can be understood that in actual implementation, the electronic device may also periodically transmit the first N subject position information in the above subject position information queue.

[0120] In this way, since the electronic device uses each result image that reaches the application framework layer as the timing for sending the first N subject position information in the subject position information queue, the amount of subject position information in the application layer can be controlled, thereby shortening the time for determining the subject position information with the same identifier associated with each result image from all the subject position information in the application layer, thereby enabling the electronic device to quickly and accurately determine the position of the subject in each result image.

[0121] In some embodiments of the present application, the second path may be a path between the second processing engine and the application framework layer, and the second processing engine may be used to perform image processing on the image. The "transmitting the second image and the first processing parameter corresponding to the first image through the second path" in the above step 101 may be implemented through the following steps C, D, and E.

[0122] Step C: The electronic device transmits the second image and the first processing parameter to the application framework layer through the second path.

[0123] Step D: When the fourth image reaches the application framework layer, the electronic device transmits the first N processing parameters in the processing parameter queue to the application layer through the third path.

[0124] Step E: When the second image reaches the application framework layer, the electronic device transmits the second image to the application layer through the third path.

[0125] Among them, the third path is the path between the application framework layer and the application layer.

[0126] Among them, the fourth image can be the result image corresponding to the fifth image, the fifth image is an image located before the first image in the above-mentioned image stream or the first image, and the processing parameters in the above-mentioned processing parameter queue can be arranged in the order of the time when each processing parameter arrives at the application framework layer. The first N processing parameters can include the first processing parameter, and N can be a positive integer.

[0127] In some embodiments of the present application, the fourth image is a result image obtained by processing the fifth image according to the fifth processing parameters. For example, the fourth image may be an image obtained by processing the fifth image by the second processing engine using the fifth processing parameters.

[0128] It should be noted that after the fourth image reaches the application framework layer, the electronic device may also transmit the fourth image to the application layer through the third path.

[0129] In some embodiments of the present application, the electronic device may transmit the fourth image and the first N processing parameters respectively through the third path. Since the image size of the fourth image is larger than the data size of the first N processing parameters, the time when the first N processing parameters arrive at the application framework layer is earlier than the time when the fourth image arrives at the application framework layer.

[0130] It should be noted that after the electronic device transmits the first N processing parameters in the processing parameter queue to the application layer, the electronic device may delete the first N processing parameters in the processing parameter queue to update the processing parameter queue.

[0131] For other descriptions of step D, refer to the relevant descriptions of step B and step B1 in the above embodiments.

[0132] In this way, since the electronic device uses each result image that reaches the application framework layer as the sending opportunity of the first N processing parameters in the processing parameter queue, the number of processing parameters in the application layer can be controlled, thereby shortening the time for determining the processing parameters with the same image identifier associated with each result image from all the processing parameters in the application layer, thereby enabling the electronic device to quickly and accurately determine the processing parameters in each result image.

[0133] In the image processing method provided in the embodiment of the present application, in the process of processing the image stream, since the subject position information of the subject in each image can be transmitted through the first path, and the processing parameters corresponding to each image and the processed result image, such as the second image, can be transmitted through the second path; that is, the subject position information of the subject in the image does not need to be transmitted along with the processing parameters corresponding to the image, and the subject position information, result image and processing parameters of each image are all associated with the image identifier of the image, it can ensure that the electronic device uses the correct subject position information to determine the position of the subject in the corresponding result image, thereby improving the accuracy of the electronic device in determining the position of the subject in the image.

[0134] In some embodiments of the present application, before the above step 102, the image processing method provided by the embodiment of the present application may further include the following step 103.

[0135] Step 103: The electronic device selects first subject position information from the subject position information in the application layer based on the image identifier associated with the second image, and selects a first processing parameter from the processing parameters in the application layer.

[0136] In some embodiments of the present application, the electronic device selects first subject position information from the subject position information in the application layer based on the image identifier associated with the second image, and selects first processing parameters from the processing parameters in the application layer, which may include: the electronic device compares the image identifier associated with the second image with the image identifier associated with each subject position information in the application layer until the subject position information with the same image identifier associated with the second image is found, that is, the first subject position information; and the electronic device may compare the image identifier associated with the second image with the image identifier associated with each processing parameter in the application layer until the processing parameter with the same image identifier associated with the second image is selected, that is, the first processing parameter.

[0137] For example, assuming that after the second image reaches the application layer, the application layer includes 15 subject position information and 15 processing parameters, and the image identifier associated with the second image is frame number 3, then: the electronic device can compare frame number 3 with the frame number associated with the first subject position information among the 15 subject position information; if the frame number associated with the first subject position information is 3, the electronic device uses the first subject position information as the first subject position information; otherwise, the electronic device can continue to compare frame number 3 with the frame number associated with the second subject position information among the 15 subject position information until the subject position information associated with frame number 3 is compared.

[0138] It should be noted that the electronic device deletes the subject position information and processing parameters that have been used from the application layer to further narrow the selection range of the subject position information and processing parameters.

[0139] In this way, since the first subject position information, the second image and the first processing parameters are all associated with the image identifier of the first image, when the electronic device needs to determine the position information of the subject in the second image, the electronic device can quickly and accurately select the first processing parameters and the first subject position information corresponding to the second image from the application layer according to the image identifier associated with the second image, so that the subject position information of the subject in the image can be accurately matched with the image, thereby achieving a better display effect of the subject.

[0140] The complete process of the image processing method provided in the embodiment of the present application is exemplarily described below in conjunction with the accompanying drawings.

[0141] Exemplarily, after the image sensor in the electronic device outputs the first image, the subject position information of the image subject, the image processed by each processing module, and the processing parameter information of the image processed by each processing module can be divided into two paths for transmission in the image signal processing path, and finally given to the display panel. Among them, the subject position information of the image subject is transmitted in the first path, and the image processed by each processing module and the processing parameter information of the image processed by each processing module are transmitted in the second path. The subject position information of the image subject and the processing parameter information of the image processed by each processing module will be stored in the application framework layer to form a subject position information queue and a processing parameter information queue during the transmission of their respective paths, and transmitted to the application layer together with the processed image. The application layer selects the subject position information and processing parameter information corresponding to the processed image in the subject position information queue and the processing parameter information queue for the processed image, and corrects the subject position information, and finally draws the image and the position of the subject in the image on the display panel. In this way, the subject position information of the subject in the image can be accurately matched with the image, so that the display effect of the subject is better.

[0142] It should be noted that the position information of the above-mentioned subject can be any information that can express the position of the subject, such as the coordinate data of the rectangular frame that frames the subject, the coordinate data of the key points of the subject, the outline coordinate data of the subject, etc., and the embodiment of the present application is not limited to this.

[0143] In some embodiments of the present application, the image drawn to the display panel and the position of the subject in the image may or may not display the position information of the subject in the display panel. The specific selection can be made based on the actual usage scenario of the user, and the present application does not limit this.

[0144] Combine the following Figure 3 Assuming that the above image stream includes 30 frames of images, and the image identifier is the image frame number, then any frame of the 30 frames of images can be the first image, that is, each frame of the 30 frames of images needs to execute the following steps 201 to 208.

[0145] Step 201: An image sensor captures a first image.

[0146] In some embodiments of the present application, after the image sensor acquires the first image, the first image can be transmitted to the image front-end module.

[0147] Step 202: The image front-end module performs preprocessing on the first image and outputs a first processing result.

[0148] The first processing result may include two processing results, namely a second processing result and a third processing result. The second processing result is output to the first processing engine, and the third processing result is output to the second processing engine.

[0149] Among them, the second processing result may include the preprocessed first image, and the second processing result includes the frame number of the first image; the third processing result includes the preprocessed first image, the processing parameters for preprocessing the first image and the frame number of the first image.

[0150] For example, if the preprocessing performed by the image front-end module on the first image is image compression processing, the second processing result is different from the third processing result, and the second processing result may be the compressed image and the frame number of the first image; the third processing result may be the compressed image, the compression multiple of the first image and the frame number of the first image.

[0151] Step 203: The electronic device performs subject position recognition on the image in the second processing result through the first processing engine to obtain first subject position information, and sends the first subject position information to the application framework layer through the first path.

[0152] The first subject position information is associated with the frame number of the first image.

[0153] In some embodiments of the present application, after the electronic device sends the first subject position information and the frame number of the first image to the application framework layer through the first path, the application framework layer can receive the first subject position information and the frame number of the first image.

[0154] Step 204: The electronic device processes the image in the third processing result through the second processing engine to obtain a second image; and sends the second image and the first processing parameter to the application framework layer through the second path.

[0155] The second image and the first processing parameter are respectively associated with the frame number of the first image.

[0156] In some embodiments of the present application, after the electronic device sends the second image and the first processing parameter to the application framework layer through the second path, the application framework layer can receive the second image and the first processing parameter.

[0157] In some embodiments of the present application, the electronic device may store subject location information arriving at the application framework layer in the form of a subject location information queue in the application framework layer, and store processing parameters arriving at the application framework layer in the form of a processing parameter queue.

[0158] Step 205: When the fourth image reaches the application framework layer, the electronic device may send the first N processing parameters in the processing parameter queue, the first N subject position information in the subject position information queue, and the fourth image to the application layer through the third path.

[0159] It can be understood that if the second image arrives at the application framework layer later than the first processing parameter, the sending time of the electronic device transmitting the first processing parameter and the second image through the third path may be different.

[0160] In some embodiments of the present application, the electronic device may send the second image to the application layer through a third path after the second image reaches the application framework layer, as well as the first N information in the above two queues when the second image reaches the application framework layer.

[0161] Step 206: The electronic device may select subject position information associated with the frame number in the subject position information in the application layer based on the frame number associated with the second image in the application layer, and select processing parameters associated with the frame number in the processing parameters in the application layer.

[0162] Step 207: The electronic device determines third subject position information of the subject in the second image based on the selected subject position information and the processing parameters.

[0163] Step 208: The electronic device displays the second image and the first logo on the display screen.

[0164] The first identifier is determined based on the third subject position information, and is used to indicate the position of the subject in the second image. For example, the first identifier may be a rectangular frame that selects the subject in the second image on the display screen.

[0165] In this way, by redesigning the transmission path of the subject detection results, avoiding the limitations caused by the current ISP path design and the Android system framework, it is possible to use the subject position information corresponding to each original image to determine the position information of the subject in the result image corresponding to each original image, so that the subject detection coordinates can be accurately drawn in the result image of the corresponding frame number for display.

[0166] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.

[0167] like Figure 4 As shown, the embodiment of the present application also provides an image processing device 400, which may include: a transmission module 401, which may be used to transmit the first subject position information of the first image in the image stream through a first path during processing of the image stream, and transmit the second image and the first processing parameter corresponding to the first image through a second path, wherein the first image is any image in the image stream; wherein the second image is the result image obtained by processing the first image according to the first processing parameter; the first subject position information, the second image and the first processing parameter are all associated with the image identifier of the first image; and a determination module 402, which may be used to determine the position information of the subject in the second image transmitted by the transmission module 401 based on the first subject position information and the first processing parameter transmitted by the transmission module 401.

[0168] In some embodiments of the present application, the first path is a path between the first processing engine and the application framework layer, and the first processing engine is used to detect the subject position information of the subject in the image; the above-mentioned transmission module can be specifically used to: transmit the first subject position information to the application framework layer through the first path; transmit the first subject position information to the application layer through the third path, and the third path is a path between the application framework layer and the application layer.

[0169] In some embodiments of the present application, the above-mentioned transmission module can be specifically used to: when the fourth image transmitted through the second path reaches the application framework layer, transmit the first N subject position information in the subject position information queue to the application layer through the third path, the subject position information in the subject position information queue is arranged in the time sequence of the subject position information arriving at the application framework layer, the first N subject position information include the first subject position information, N is a positive integer; wherein the fourth image is the result image corresponding to the fifth image, and the fifth image is an image located before the first image in the image stream or is the first image.

[0170] In some embodiments of the present application, the above-mentioned second path is a path between the second processing engine and the application framework layer, and the second processing engine is used to perform image processing on the image; the above-mentioned transmission module can be specifically used to: transmit the second image and the first processing parameter to the application framework layer through the second path; when the fourth image reaches the application framework layer, transmit the first N processing parameters in the processing parameter queue to the application layer through the third path, and the first N processing parameters include the first processing parameter; when the second image reaches the application framework layer, transmit the second image to the application layer through the third path; wherein the processing parameters in the processing parameter queue are arranged in the time order when the processing parameters arrive at the application framework layer, and N is a positive integer.

[0171] In some embodiments of the present application, the above-mentioned image processing device may further include: a selection module, which can be used to select the first subject position information from the subject position information in the application layer based on the image identifier associated with the second image, and select the first processing parameter from the processing parameters in the application layer before the above-mentioned determination module determines the position information of the subject in the second image based on the first subject position information and the first processing parameter.

[0172] In the image processing device provided in the embodiment of the present application, in the process of processing the image stream, the subject position information of the subject in each image can be transmitted through the first path, and the processing parameters corresponding to each image and the processed result image, such as the second image, can be transmitted through the second path; that is, the subject position information of the subject in the image does not need to be transmitted along with the processing parameters corresponding to the image, and the subject position information, result image and processing parameters of each image are all associated with the image identifier of the image, so it can be ensured that the image processing device uses the correct subject position information to determine the position of the subject in the corresponding result image, thereby improving the accuracy of the image processing device in determining the position of the subject in the image.

[0173] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0174] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0175] The image processing device provided in the embodiment of the present application can achieve Figures 1 to 3The various processes implemented in the method embodiment achieve the same technical effect and will not be described again here to avoid repetition.

[0176] Alternatively, if Figure 5 As shown, an embodiment of the present application also provides an electronic device 500, including a processor 501 and a memory 502, wherein the memory 502 stores programs or instructions that can be executed on the processor 501, and when the program or instructions are executed by the processor 501, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0177] It should be noted that the electronic devices in the embodiments of the present application include mobile electronic devices and non-mobile electronic devices.

[0178] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0179] The electronic device 1500 includes but is not limited to: a radio frequency unit 1501, a network module 1502, an audio output unit 1503, an input unit 1504, a sensor 1505, a display unit 1506, a user input unit 1507, an interface unit 1508, a memory 1509, and a processor 1510 and other components.

[0180] Those skilled in the art will appreciate that the electronic device 1500 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1510 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0181] Among them, the processor 1510 can be used to transmit the first subject position information of the first image in the image stream through the first path during the process of processing the image stream, and transmit the second image and the first processing parameter corresponding to the first image through the second path, the first image is any image in the image stream; wherein the second image is the result image obtained by processing the first image according to the first processing parameter; the first subject position information, the second image and the first processing parameter are all associated with the image identifier of the first image; the processor 1510 can be used to determine the position information of the subject in the second image transmitted by the processor 1510 based on the first subject position information and the first processing parameter transmitted by the processor 1510.

[0182] In some embodiments of the present application, the first path is a path between the first processing engine and the application framework layer, and the first processing engine is used to detect the subject position information of the subject in the image; the processor 1510 can be specifically used to: transmit the first subject position information to the application framework layer through the first path; transmit the first subject position information to the application layer through the third path, and the third path is a path between the application framework layer and the application layer.

[0183] In some embodiments of the present application, the above-mentioned processor 1510 can be specifically used to: when the fourth image transmitted through the second path reaches the application framework layer, transmit the first N subject position information in the subject position information queue to the application layer through the third path, the subject position information in the subject position information queue is arranged in the time sequence of the subject position information arriving at the application framework layer, and the first N subject position information include the first subject position information, N is a positive integer; wherein the fourth image is the result image corresponding to the fifth image, and the fifth image is an image located before the first image in the image stream or is the first image.

[0184] In some embodiments of the present application, the above-mentioned second path is a path between the second processing engine and the application framework layer, and the second processing engine is used to perform image processing on the image; the above-mentioned processor 1510 can be specifically used to: transmit the second image and the first processing parameter to the application framework layer through the second path; when the fourth image reaches the application framework layer, transmit the first N processing parameters in the processing parameter queue to the application layer through the third path, and the first N processing parameters include the first processing parameter; when the second image reaches the application framework layer, transmit the second image to the application layer through the third path; wherein the processing parameters in the processing parameter queue are arranged in the time order when the processing parameters arrive at the application framework layer, and N is a positive integer.

[0185] In some embodiments of the present application, the processor 1510 is used to select the first subject position information from the subject position information in the application layer based on the image identifier associated with the second image, and select the first processing parameter from the processing parameters in the application layer before the processor 1510 determines the position information of the subject in the second image based on the first subject position information and the first processing parameter.

[0186] In the electronic device provided in the embodiment of the present application, in the process of processing the image stream, since the subject position information of the subject in each image can be transmitted through the first path, and the processing parameters corresponding to each image and the processed result image, such as the second image, can be transmitted through the second path; that is, the subject position information of the subject in the image does not need to be transmitted along with the processing parameters corresponding to the image, and the subject position information, result image and processing parameters of each image are all associated with the image identifier of the image, it can be ensured that the electronic device uses the correct subject position information to determine the position of the subject in the corresponding result image, thereby improving the accuracy of the electronic device in determining the position of the subject in the image.

[0187] It should be understood that in the embodiment of the present application, the input unit 1504 may include a graphics processor (Graphics Processing Unit, GPU) 15041 and a microphone 15042, and the graphics processor 15041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1506 may include a display panel 15061, and the display panel 15061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1507 includes a touch panel 15071 and at least one of other input devices 15072. The touch panel 15071 is also called a touch screen. The touch panel 15071 may include two parts: a touch detection device and a touch controller. Other input devices 15072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0188] The memory 1509 can be used to store software programs and various data. The memory 1509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1509 may include a volatile memory or a non-volatile memory, or the memory 1509 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1509 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0189] The processor 1510 may include one or more processing units; optionally, the processor 1510 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1510.

[0190] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0191] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0192] An embodiment of the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0193] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0194] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0195] It should be noted that, in this article, the term "comprises", "includes" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0196] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0197] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: In the process of processing an image stream, first subject position information of a first image in the image stream is transmitted through a first path, and a second image and a first processing parameter corresponding to the first image are transmitted through a second path, wherein the first image is any image in the image stream; wherein the second image is a result image obtained by processing the first image according to the first processing parameter; and the first subject position information, the second image and the first processing parameter are all associated with an image identifier of the first image; Based on the first subject position information and the first processing parameter, position information of a subject in the second image is determined.

2. The method according to claim 1, characterized in that The first path is a path between the first processing engine and the application framework layer, and the first processing engine is used to detect subject position information of a subject in an image; The transmitting the first subject position information of the first image in the image stream through the first path includes: transmitting the first subject location information to the application framework layer through the first path; The first subject position information is transmitted to the application layer through a third path, and the third path is a path between the application framework layer and the application layer.

3. The method according to claim 2, characterized in that The transmitting the first subject position information to the application layer through the third path includes: In the case where the fourth image transmitted through the second path reaches the application framework layer, the first N subject position information in the subject position information queue is transmitted to the application layer through the third path, the subject position information in the subject position information queue is arranged in the time sequence of the subject position information arriving at the application framework layer, the first N subject position information includes the first subject position information, and N is a positive integer; The fourth image is a result image corresponding to the fifth image, and the fifth image is an image located before the first image in the image stream or is the first image.

4. The method according to claim 3, characterized in that The second path is a path between the second processing engine and the application framework layer, and the second processing engine is used to perform image processing on the image; The transmitting the second image and the first processing parameter corresponding to the first image through the second path includes: transmitting the second image and the first processing parameter to the application framework layer through the second path; When the fourth image reaches the application framework layer, transmitting first N processing parameters in the processing parameter queue to the application layer through the third path, wherein the first N processing parameters include the first processing parameter; When the second image reaches the application framework layer, transmitting the second image to the application layer through the third path; The processing parameters in the processing parameter queue are arranged in the order of time when the processing parameters arrive at the application framework layer, and N is a positive integer.

5. The method according to claim 4, characterized in that Before determining the position information of the subject in the second image based on the first subject position information and the first processing parameter, the method further includes: Based on the image identifier associated with the second image, the first subject position information is selected from the subject position information in the application layer, and the first processing parameter is selected from the processing parameters in the application layer.

6. An image processing device, characterized in that: The device comprises: a transmission module, configured to transmit, during processing of an image stream, first subject position information of a first image in the image stream through a first path, and transmit a second image and a first processing parameter corresponding to the first image through a second path, wherein the first image is any image in the image stream; wherein the second image is a result image obtained by processing the first image according to the first processing parameter; and the first subject position information, the second image and the first processing parameter are all associated with an image identifier of the first image; A determination module is used to determine the position information of the subject in the second image transmitted by the transmission module based on the first subject position information transmitted by the transmission module and the first processing parameter.

7. The device according to claim 6, characterized in that The first path is a path between the first processing engine and the application framework layer, and the first processing engine is used to detect subject position information of a subject in an image; The transmission module is specifically used for: transmitting the first subject location information to the application framework layer through the first path; The first subject position information is transmitted to the application layer through a third path, and the third path is a path between the application framework layer and the application layer.

8. The device according to claim 7, characterized in that The transmission module is specifically used for: In the case where the fourth image transmitted through the second path reaches the application framework layer, the first N subject position information in the subject position information queue is transmitted to the application layer through the third path, the subject position information in the subject position information queue is arranged in the time sequence of the subject position information arriving at the application framework layer, the first N subject position information includes the first subject position information, and N is a positive integer; The fourth image is a result image corresponding to the fifth image, and the fifth image is an image located before the first image in the image stream or is the first image.

9. The device according to claim 7, characterized in that The second path is a path between the second processing engine and the application framework layer, and the second processing engine is used to perform image processing on the image; The transmission module is specifically used for: transmitting the second image and the first processing parameter to the application framework layer through the second path; When the fourth image reaches the application framework layer, transmitting first N processing parameters in the processing parameter queue to the application layer through the third path, wherein the first N processing parameters include the first processing parameter; When the second image reaches the application framework layer, transmitting the second image to the application layer through the third path; The processing parameters in the processing parameter queue are arranged in the order of time when the processing parameters arrive at the application framework layer, and N is a positive integer.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 5 are implemented.

Images