Shooting method and electronic equipment

Abstract

The invention provides a shooting method and electronic equipment, and relates to the technical field of terminals.The method comprises the steps that in response to an image obtaining request of an application program, an original image is obtained from an image sensor; processing a target area in the original image to obtain an analysis stream image, the image quality of the target area in the analysis stream image being superior to the image quality of the target area in the original image; and sending the analysis stream image to the application program, wherein the analysis stream image is used for the application program to execute an image analysis service. According to the technical scheme provided by the invention, the efficiency of providing the image with the higher image quality for the application program is improved, the interaction frequency generated by the application program for obtaining the image with the higher image quality is reduced, and the efficiency of executing the image analysis service by the application program is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a shooting method and electronic equipment. Background Art

[0002] Currently, electronic devices may include cameras and other applications. Other applications may wake up the camera, and the camera may capture images and provide the images to other applications. Other applications may perform image analysis services based on the images, such as scanning codes, etc. Among them, the quality of the image is directly related to the execution effect of the image analysis service.

[0003] In the prior art, the camera can send images to other applications, and the other applications perform image analysis services based on the received images. If the image analysis service fails to be executed, the image quality of the image may be relatively low, so the camera control parameters such as the focal length of the camera can be changed. The camera obtains a new image again according to the changed camera control parameters and provides the new image to other applications. The other applications try to perform the image analysis service again based on the new image, and so on, and so on, until the image analysis service is successfully executed. Since the camera and other applications need to interact multiple times to obtain the image used to perform the image analysis service, the efficiency of other applications in performing the image analysis service is extremely low. Summary of the invention

[0004] In view of this, the present application provides a shooting method and an electronic device, which can improve the quality of images provided to an application and improve the efficiency of the application in performing business.

[0005] In order to achieve the above-mentioned objectives, in a first aspect, an embodiment of the present application provides a shooting method, which is applied to an electronic device, comprising: in response to an image acquisition request of an application of the electronic device, acquiring an original image from an image sensor; processing a target area in the original image to obtain an analysis stream image, wherein the image quality of the target area in the analysis stream image is better than the image quality of the target area in the original image; sending the analysis stream image to the application, wherein the analysis stream image is used by the application to perform image analysis services.

[0006] In an embodiment of the present application, in response to an image acquisition request of an application, an original image can be acquired from an image sensor, and the target area in the original image can be optimized to obtain an analysis stream image. Since the original image is an image acquired from an image sensor, the original image includes more pixel information. Therefore, by optimizing the target area in the original image, these more pixel information can be fully utilized, so that the image quality of the target area in the analysis stream image is better than the image quality of the target area in the original image, that is, the analysis stream image obtained by the optimization process has a higher image quality. Then, providing the analysis stream image to the application improves the efficiency of providing the application with images with higher image quality, reduces the number of interactions generated by the application to obtain images with higher image quality, and improves the efficiency of the application in performing image analysis services.

[0007] The image analysis service may be a service that includes analyzing and processing an image, and the application may execute the image analysis service to obtain service information. A better image quality of an image may mean that the image includes more meaningful features, such as color, shape, and texture, so that objects or other targets included in the image can be identified faster and more accurately; on the contrary, a poor image quality of an image may mean that the image includes fewer meaningful features, such as color, shape, and texture. For example, the image quality of the target area in the analysis stream image is better than the image quality of the target area in the original image, which may mean that the target area in the analysis stream image includes less noise, has a higher contrast, is clearer, and so on.

[0008] In addition, the target area may be at least a portion of the original image. When the target area is smaller, the time required to process the target area in the original image to obtain the analysis stream image is shorter, and the efficiency of providing the image to the application and the efficiency of the application performing the image analysis service is higher.

[0009] In some embodiments, the method further includes: the display of the electronic device displays a first interface, the first interface includes a preview stream image, and the preview stream image is the original image or an image obtained by processing the original image.

[0010] By displaying the preview stream image, the user can preview the current shooting process, which is convenient for the user to control the camera system. The processing of the original image may include the aforementioned optimization processing and / or downsampling of the target area. When the displayed image is the same as the image sent to the application, the image previewed by the user can be synchronized with the image used by the application to perform the image analysis service. What the user sees is what the application gets, which improves the reliability of the interaction between the user and the application.

[0011] In some embodiments, after displaying the first interface, the method further includes: the display of the electronic device displays a second interface, the second interface includes the preview stream image and business information, and the business information is information obtained by the application performing the image analysis business based on the analysis stream image.

[0012] Among them, since the business information is the information generated by the application program performing the image analysis business based on the analysis stream image, the higher the similarity between the analysis stream image and the preview stream image, the higher the matching degree between the business information displayed in the second interface and the preview stream image, and the process of switching from the first interface to the second interface is smoother, further reducing the user's perception of the process of obtaining the analysis stream image. In some embodiments, the application program can send business information and the display position corresponding to the business information to the display, and the display position can be determined based on the analysis stream image. If the display program displays the business information in the second interface based on the display position, the higher the similarity between the analysis stream image and the preview stream image, the smaller the difference between the relative position of the display position in the analysis stream image and the relative position of the display position in the preview stream image, thereby reducing the user's perception of the process of obtaining the analysis stream image and improving the user experience.

[0013] In some embodiments, the preview stream image is the original image after downsampling, and the method further includes: downsampling the original image to obtain the downsampled original image; processing the target area in the original image to obtain the analysis stream image, including: acquiring a local image in the target area from the original image; optimizing and downsampling the local image to obtain a new local image; replacing the target area of ​​the downsampled original image with the new local image to obtain the analysis stream image.

[0014] Among them, downsampling, also called downsampling, can be understood as reducing the image and reducing the number of pixels included in the image.

[0015] In the embodiment of the present application, on the one hand, for the original image obtained from the image sensor, the direct original image can be downsampled and the downsampled original image can be displayed, so as to reduce the difference between the image previewed by the user and the image seen by the user through the glasses, and reduce the user's perception of the image optimization process. On the other hand, for the original image obtained from the image sensor, a local image can be obtained from the target area in the original image, and the local image can be optimized and downsampled to obtain a new local image with higher image quality, and the target area in the downsampled original image can be replaced with the new local image to obtain an analysis stream image, so the target area of ​​the analysis stream image and the target area of ​​the preview stream image include similar image content, and the image quality of the target area of ​​the analysis stream image is higher, which not only enables the application to improve the efficiency and reliability of executing the image analysis business based on the analysis stream image with higher image quality, but also reduces the difference between the analysis stream image provided to the application and the preview stream image displayed, which is convenient for the subsequent display of the business information corresponding to the image analysis business, thereby improving the user experience.

[0016] In some implementations, the image analysis service includes a code scanning service, an object recognition service, a human body recognition service, or a text recognition service.

[0017] The code scanning service may include identifying whether a graphic code (such as a barcode and a QR code) is included in an image, identifying the code information carried by the graphic code, and displaying the code information. When the code information is a network address, the page indicated by the network address may be displayed. Accordingly, the service information may be used to indicate whether at least one of the graphic code, the code information, and the page indicated by the code information is recognized.

[0018] The object recognition service may include identifying whether there is a specific object in the image, or identifying multiple objects included in the image. In some embodiments, the object recognition service may also include further operations based on the object recognition results. For example, based on the identified object, continue to search and return the search results. For example, if shoes are identified from the image, you can further search for stores selling the shoes or similar items from the e-commerce mall. Another example is rendering and displaying corresponding visual effects based on the identified objects, such as in AR, VR and mixed reality (MR) scenes, based on the identified preset reference objects, displaying virtual animation effects, etc. Correspondingly, the business information can be used to indicate whether a specific object is recognized, marking of the recognized object, search results and at least one of the visual effects.

[0019] The human body recognition service may include identifying whether a specific human body is included in an image, or identifying multiple human bodies included in an image. For example, in a face verification scenario, it may be possible to identify whether a human face is included in an image; in a beauty photo shooting scenario, it may be possible to identify a human body and display the beautified human body. Accordingly, the service information may be used to indicate at least one of whether a human body is recognized and the visual effect used to beautify the human body.

[0020] The text recognition service may include recognizing text included in an image. In some embodiments, text corresponding to the recognized text may be generated for user storage. In some embodiments, the recognized text may be translated or annotated. Accordingly, the service information may be used to indicate whether it is text, the specific content of the recognized text, at least one of translation and annotation.

[0021] It is understandable that, in actual applications, the manner of executing the image analysis service and the specific content included in the service information can be determined by the application developer and are not limited to the above-mentioned services.

[0022] In some implementations, the method further includes: identifying preset image content included in the original image; and determining an area where the preset image content is located as the target area.

[0023] It can be identified whether the original image includes the image feature corresponding to the preset image content. If the original image includes the image feature, it can be determined that the original image includes the preset image, and then the area where the preset image content is located is determined as the target area. In some embodiments, the preset image content may include specific objects, text or human bodies such as QR codes, flowers, animals and commodities. For example, the preset image content includes a QR code, and the image feature corresponding to the QR code may include three positioning frames. Then it can be identified whether the original image includes three positioning frames. If so, it can be determined that the original image includes the QR code, and the area where the three positioning frames are located is the target area. For another example, the preset image content includes a face, and the image feature corresponding to the face may include facial features. Then it can be identified whether the original image includes facial features. If so, it can be determined that the original image includes a face, and the area where the face is located is the target area. It can be understood that the preset image content and the image features corresponding to the preset image content can be obtained by setting in advance.

[0024] In some embodiments, the target area may be an area of ​​a preset position, preset size, and preset shape. For example, the target area may be a rectangular area of ​​a preset size centered at the center of the original image. It is understood that in actual applications, the target area may also be an area of ​​other positions, other sizes, or other shapes.

[0025] In some embodiments, the processing of the target area in the original image includes at least one of the following: performing image restoration processing on the target area, the image restoration processing including at least one of denoising processing, deblurring processing and geometric correction processing; replacing a first graphic code included in the target area with a second graphic code, the second graphic code being a graphic code generated based on the code information carried by the first image code.

[0026] Among them, performing image restoration processing on the target area can make the meaningful features included in the target area more prominent; and directly replacing the first graphic code of the target area with the newly generated second graphic code by reversely generating the graphic code, the newly generated second graphic code is clearer, that is, the target area has a higher image quality, and because the second graphic code is a graphic code generated based on the code information carried by the first image code, the code information carried by the first graphic code and the second graphic code can be the same, reducing the adverse effects of replacing the graphic code on subsequent image analysis services.

[0027] In some implementations, the processing the target area in the original image includes: if the application belongs to a program set, processing the target area in the original image, and the program set includes at least one application.

[0028] Among them, when the application is a preset application, the target area in the original image can be processed, so as to provide the application with an analysis stream image with higher image quality, so as to improve the efficiency of the preset application in performing image analysis business; when the application is not a preset application, the image quality provided to the application may not be improved to reduce system energy consumption.

[0029] In a second aspect, an embodiment of the present application provides an electronic device, the electronic device comprising an application and a camera hardware abstraction layer (HAL);

[0030] The camera HAL is used to respond to an image acquisition request of the application, acquire an original image from an image sensor, process a target area in the original image, obtain an analysis stream image, wherein the image quality of the target area in the analysis stream image is better than the image quality of the target area in the original image, and send the analysis stream image to the application;

[0031] The application is used to perform image analysis services based on the analysis stream image.

[0032] In some implementations, the electronic device includes an application layer and a HAL, the application layer includes the application, and the HAL includes the camera HAL.

[0033] In some embodiments, the electronic device further comprises a display;

[0034] The camera HAL is further used to send a preview stream image to the display, where the preview stream image is the original image or an image obtained by processing the original image;

[0035] The display is used to display a first interface, and the first interface includes the preview stream image.

[0036] In some embodiments, the application is further used to perform the image analysis service based on the analysis stream image, obtain the service information, and send the service information to the display; the display is further used to display a second interface after displaying the first interface, and the second interface includes the preview stream image and the service information.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, the memory being used to store a computer program; the processor being used to execute any one of the methods described in the first aspect above when calling the computer program.

[0038] In a fourth aspect, an embodiment of the present application provides a chip system, which includes a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement any method described in the first aspect above.

[0039] The chip system may be a single chip or a chip module composed of multiple chips.

[0040] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in any one of the above-mentioned first aspects is implemented.

[0041] In a sixth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes any one of the methods described in the first aspect above.

[0042] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0044] Figure 2 A software structure block diagram of an electronic device provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of a shooting scene provided in an embodiment of the present application;

[0047] Figure 5 A flowchart of a shooting method provided in an embodiment of the present application;

[0048] Figure 6 A schematic diagram of a shooting interface provided in an embodiment of the present application;

[0049] Figure 7 A flowchart of another shooting method provided by an embodiment of the present application;

[0050] Figure 8 A schematic diagram of a method for processing an image provided by an embodiment of the present application;

[0051] Fig. 9 A schematic diagram of another method for processing an image provided by an embodiment of the present application;

[0052] Fig.10 A flowchart of another shooting method provided by an embodiment of the present application;

[0053] Fig.11 A flowchart of another shooting method provided by an embodiment of the present application;

[0054] Fig.12 A flowchart of another shooting method provided by an embodiment of the present application;

[0055] Fig.13 A flowchart of another shooting method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The shooting method provided in the embodiments of the present application can be applied to electronic devices such as mobile phones, cameras, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0057] Figure 1 1 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may include a processor 110, a memory 120, a communication module 130, a camera 140, and the like.

[0058] The processor 110 may include one or more processing units, and the memory 120 is used to store program codes and data. In an embodiment of the present application, the processor 110 may execute computer-executable instructions stored in the memory 120 to control and manage the actions of the electronic device 100. For example, the processor 110 may include an application processor (application processor, AP), a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a video codec, etc. Different processing units may be independent devices or integrated in one or more processors.

[0059] In some implementations, processor 110 may include one or more interfaces.

[0060] In some embodiments, the interface may include a mobile industry processor interface (MIPI). The MIPI interface may be used to connect the processor 110 to peripheral devices such as the camera 140. The MIPI interface includes a camera serial interface (CSI), and the processor 110 and the camera 140 communicate through the CSI interface to implement the shooting function of the electronic device 100.

[0061] In some embodiments, the interface may include a general-purpose input / output (GPIO) interface. The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal or as a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 to the camera 140.

[0062] The communication module 130 can be used for communication between various internal modules of the electronic device 100, or communication between the electronic device 100 and other external electronic devices. Exemplarily, if the electronic device 100 communicates with other electronic devices via a wired connection, the communication module 130 may include an interface, such as a USB interface. The USB interface may be an interface that complies with USB standard specifications, specifically a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as remote control devices.

[0063] Alternatively, the communication module 130 may include an audio device, a radio frequency circuit, a Bluetooth chip, a wireless fidelity (Wi-Fi) chip, a near-field communication technology (NFC) module, etc., and can achieve interaction between the electronic device 100 and other electronic devices in a variety of different ways.

[0064] The camera 140 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format.

[0065] Optionally, the electronic device 100 may further include a display 150, which may display images or videos in a human-computer interaction interface. In some embodiments, the electronic device 100 implements a display function through a GPU, a display 150, and an application processor. The GPU is a microprocessor for image processing, which connects the display 150 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information. In some embodiments, the electronic device 100 may implement a shooting function through an ISP, a camera 140, a video codec, a GPU, a display 150, and an application processor.

[0066] The ISP is used to process the data fed back by the camera 140. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 140.

[0067] In some embodiments, the electronic device 100 may further include a power supply, which may be used to supply power to various components of the electronic device 100 including the processor 110 .

[0068] It should be understood that, in addition to Figure 1 In addition to the various components or modules listed in the figure, the embodiment of the present application does not specifically limit the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware.

[0069] Figure 2 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0070] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0071] The application layer can include a series of application packages.

[0072] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0073] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0074] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0075] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0076] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0077] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0078] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0079] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0080] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0081] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0082] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0083] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0084] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as openGL ES), 2D graphics engine (such as SGL), etc.

[0085] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0086] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0087] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0088] A 2D graphics engine is a drawing engine for 2D drawings.

[0089] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0090] Please refer to Figure 3 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes an application, a camera system and a display system. The display system can be omitted.

[0091] The application can be the aforementioned Figure 2 Any application in the application layer in the system can control the camera to obtain images from the camera system and perform image analysis based on the images. It can also control the display system to display a human-computer interaction interface to the user based on the display system and interact with the user.

[0092] Among them, the image analysis service may be a service that includes analyzing and processing images, and the application may execute the image analysis service to obtain service information. In some embodiments, the image analysis service may include a code scanning service, an object recognition service, a human body recognition service, or a text recognition service. In some embodiments, the service information may be used to indicate the analysis behavior, analysis process, or analysis result of the image analysis service. It is understandable that in actual applications, the manner in which the image analysis service is executed and the specific content included in the service information may be determined by the developer of the application, and is not limited to the above-mentioned services.

[0093] The code scanning service may include identifying whether a graphic code (such as a barcode and a QR code) is included in an image, identifying the code information carried by the graphic code, and displaying the code information. When the code information is a network address, the page indicated by the network address may be displayed. Accordingly, the service information may be used to indicate whether at least one of the graphic code, the code information, and the page indicated by the code information is recognized.

[0094] The object recognition service may include identifying whether there is a specific object in the image, or identifying multiple objects included in the image. In some embodiments, the object recognition service may also include further operations based on the object recognition results. For example, based on the recognized object, continue to search and return the search results. For example, if shoes are recognized from the image, you can further search for stores selling the shoes or similar items from the e-commerce mall. Another example is rendering and displaying corresponding visual effects based on the recognized objects, such as displaying virtual animation effects based on the recognized preset reference objects in AR, VR and MR scenes. Accordingly, the business information can be used to indicate whether a specific object is recognized, marking of the recognized object, search results and at least one of the visual effects.

[0095] The human body recognition service may include identifying whether a specific human body is included in an image, or identifying multiple human bodies included in an image. For example, in a face verification scenario, it may be possible to identify whether a human face is included in an image; in a beauty photo shooting scenario, it may be possible to identify a human body and display the beautified human body. Accordingly, the service information may be used to indicate at least one of whether a human body is recognized and the visual effect used to beautify the human body.

[0096] The text recognition service may include recognizing text included in an image. In some embodiments, text corresponding to the recognized text may be generated for user storage. In some embodiments, the recognized text may be translated or annotated. Accordingly, the service information may be used to indicate whether it is text, the specific content of the recognized text, at least one of translation and annotation.

[0097] The camera system may include a camera application, a camera service, a camera HAL, a camera driver, and camera hardware, or a combination of one or more of the above.

[0098] The camera app can be located in Figure 2 The camera application can directly interact with the user, receive user operations such as preview, photo taking and video recording, and display the image and camera status information sent back by the camera system.

[0099] The camera service can be located at Figure 2The camera service is a link between the upper and lower layers, including providing corresponding interface services to applications such as camera applications, and interacting directly with HAL. In some embodiments, the camera service sends multiple data streams provided by the camera HAL to different software or hardware.

[0100] The camera HAL can be located in Figure 2 The kernel layer in the camera HAL. The camera HAL can connect the bottom-layer camera driver and camera hardware based on the upper-layer camera framework. The camera HAL defines a standard interface called by the camera service and ensures the normal operation of the camera hardware. The camera HAL can connect camera hardware and camera drivers from different manufacturers to the camera system. In some embodiments, the camera HAL can process the original image returned by the bottom layer and input the processed image to the upper layer. In some embodiments, the camera HAL can be used to shunt the image so that multiple software or hardware in the electronic device 100 can obtain the image. For example, the camera HAL can shunt an image into an analysis stream image and a preview stream image. The preview stream image can be sent to the display 150 for display, and the analysis stream image can be sent to the application for data analysis and processing. In some embodiments, the camera HAL can downsample the original image returned by the bottom layer. The downsampled image can include less pixel information and a smaller size, which can reduce the amount of data for subsequent image processing. In some embodiments, the camera HAL can identify the original image to determine whether the original image has preset image content.

[0101] The camera driver can be located in Figure 2 The camera driver may provide an interface for calling the camera hardware. In some embodiments, the camera driver may manage the camera hardware and maintain the underlying driver business logic. In some embodiments, the camera driver may return the image processing status to the user. In some embodiments, the camera driver may enumerate the various device capabilities of the hardware device so that the user can use these device capabilities.

[0102] The camera hardware is at the bottom layer of the camera system and is the physical implementation part of the camera system. In some embodiments, the camera hardware may include a lens, an image sensor, an ISP, and may also include auxiliary modules such as a focus motor, a flash, a filter, and an aperture.

[0103] The display system may include the aforementioned Figure 1 The display and Figure 2 The display driver in is used to display the human-computer interaction interface, for example, to provide a preview interface when the camera system is taking pictures; and to provide the application interface of the application when the application is running.

[0104] It should be understood that, in addition to Figure 3 In addition to the various components or modules listed in the figure, the embodiments of the present application do not specifically limit the structure of the electronic device. In other embodiments of the present application, the electronic device may also include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0105] Please refer to Figure 4 , below, taking the code scanning business as an example, the workflow of the image analysis business and electronic equipment is illustrated.

[0106] While the user is watching TV, the TV displays a QR code 410 inviting the user to participate in the TV program. The user can pick up the phone and click the scan button of the code scanning application (third-party application) in the phone. When the user clicks the scan button, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into raw input events (including touch coordinates, timestamp of the touch operation, and other information). The raw input events are stored in the kernel layer. The application framework layer obtains the raw input events from the kernel layer and identifies the input events to wake up the camera application. The camera application calls the relevant interface services of the camera service, starts the camera driver through the camera HAL, and captures images through the camera hardware.

[0107] The image captured by the camera hardware is a raw image, which can be transmitted to the camera HAL through the camera driver. The camera HAL can perform downsampling and shunting on the raw image. The camera service or camera application can send the preview stream image obtained by shunting to the display, and the display can display the shooting interface 420, which includes the preview stream image, including the prompt information "Identify QR code / flowers / animals / commodities", so as to prompt the user to perform one or more services based on the captured image. The user can preview the shooting process through the shooting interface 420 and adjust the shooting mode and other operations. The camera service or camera application can also send the analysis stream image obtained by shunting to the code scanning application.

[0108] The code scanning application identifies whether the analysis stream image includes a QR code. If it is determined that the analysis stream includes a QR code, the display is instructed to display another shooting interface 430. The other shooting interface 430 still displays a circular mark 440 based on the shooting interface 420, thereby prompting the user that the QR code has been recognized. The code scanning application can also continue to jump to the service interface 450 based on the code information. The service interface 450 can include a participation page for the TV program, and the user can participate in the TV program on the participation page.

[0109] Please refer to Figure 5 , which is a flow chart of a shooting method provided in an embodiment of the present application. Figure 5 The method shown can be explained in Figure 4 The workflow of the camera system in the scenario shown.

[0110] S501: A camera system application receives an image acquisition request from an application.

[0111] In some implementations, the application may request the camera system to acquire an image when the image analysis service is triggered. For example, the application may request the camera system to acquire an image when it detects that the code scanning function is turned on.

[0112] S502: The camera system acquires a raw image from an image sensor.

[0113] S503: The camera system downsamples the original image to obtain a preview stream image.

[0114] Among them, downsampling the original image can reduce the size of the original image.

[0115] For example, the effective pixels of the original image may be 4096*3072, and the effective pixels of the preview stream image may be 1902*108. The preview stream image includes fewer effective pixels than the original image, which is convenient for previewing.

[0116] For another example, the original image includes a QR code, the size of the QR code is 234*234, and the original image is downsampled by 2 times, and the size of the QR code in the preview stream image is 117*117. The QR code in the preview stream image loses 3 / 4 of the pixel information compared to the QR code in the original image.

[0117] S504: The camera system splits the preview stream image, provides the preview stream image to the display system, and provides the analysis stream image to the application.

[0118] By providing the preview stream image to the display system, the display system can display the preview stream image on the display, providing the user with a preview of the picture currently captured by the camera system, such as Figure 4 The shooting interface 420 in the figure makes it easy for the user to control the camera system.

[0119] By splitting the preview stream image, multiple data streams can be obtained, and the content and quality included in the multiple data streams are consistent. By splitting the multiple data streams, images can be provided to multiple software modules or hardware modules, so that multiple software modules and hardware modules can perform different services based on the acquired images.

[0120] In some implementations, the camera system may replicate the preview stream image to obtain an analysis stream image.

[0121] In some implementations, the analysis stream image can be used in applications such as code scanning to perform image analysis services.

[0122] In some embodiments, if the application fails to perform an image analysis service based on the analysis stream image, such as being unable to recognize a QR code from the analysis stream image in a code scanning service, the camera control parameters may be changed to trigger steps similar to or identical to S501-S504 again to continue acquiring images until the image analysis service is successfully performed.

[0123] In the above Figure 5 In the shooting method shown, the camera system needs to downsample the original image captured by the image sensor and send the preview stream image obtained by the downsampling to the application. Since the preview stream image loses a large amount of pixel information, the performance of the application in executing the business will be reduced. If the application fails to perform the image analysis business based on a certain frame of image, it is often necessary to control the camera system multiple times to change the camera control parameters and obtain images multiple times. From sending the camera control parameters to re-acquiring the image based on the camera control parameters, it will cause the application to take too long to execute the image analysis business, thereby reducing the efficiency of the application and user experience.

[0124] During the shooting process, affected by factors such as shooting distance, ambient light, and shooting angle, the image quality of the original image obtained by the image sensor itself may be relatively poor, and the preview stream image obtained by the application performing the image analysis service will lose a large amount of pixel information, which makes it difficult for the application to recognize the QR code from the preview stream image. The application needs to control the camera multiple times to obtain more frames of images to continue to try to perform the image analysis service, including multiple zooms in long-distance scenes and multiple aperture adjustments in dark light scenes. Therefore, the reliability and efficiency of the application in performing the image analysis service are very low.

[0125] For example, Figure 6 As shown, when S501-S504 is executed for the first time, the camera system provides to send the 0th frame image 421 to the application and the display system, which takes 0.866 seconds. Since the shooting angle is relatively biased, the 0th frame image 421 does not include a complete QR code, and the application cannot recognize the QR code in the 0th frame image 421. Therefore, S501-S504 is executed again to request the camera system to reacquire the image, which includes requesting the camera system to change control parameters such as focal length or lens. The camera system reacquires the image based on the new control parameters and sends the image to the application.

[0126] When the camera system continues to send the 11th frame image 422 to the application and display system, it takes 1.309 seconds. Since the 11th frame image 422 is relatively clear, the size of the QR code therein is too small, and the application still cannot recognize the QR code, the application executes S501-S504 again to request the camera system to change the control parameters and reacquire the image, including requesting the camera system to increase the focal length.

[0127] When the camera system continues to send the 26th frame image 423 to the application and the display system, it takes 1.893 seconds. However, the size of the QR code in the 26th frame image 423 is still small, and the application still cannot recognize the QR code, so the application executes S501-S504 again to request the camera system to change the control parameters and reacquire the image, including requesting the camera system to increase the focal length.

[0128] When the camera system continues to send the 64th frame image 424 to the application and display system, it takes 3.424 seconds. The application successfully recognizes the QR code from the 64th frame image 424 and performs an image analysis service based on the QR code.

[0129] It can be seen that the camera system needs to obtain and provide 64 or more frames of images to the application so that the application can complete the code scanning service. Then, it will take 3 seconds or even longer for the application to complete the code scanning service.

[0130] To solve at least part of the above technical problems, the embodiment of the present application provides another shooting method. The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0131] The shooting method provided in the embodiment of the present application can be applied to the aforementioned Figure 1 The electronic device in the embodiment can be specifically applied to a camera system in the electronic device. In some implementations, the shooting method can be applied to a camera HAL in the camera system.

[0132] Please refer to Figure 7 , is a flow chart of a shooting method provided in an embodiment of the present application. It should be noted that the method is not based on Figure 7 The specific order described below is a limitation, and it should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0133] S701 , in response to an image acquisition request of an application program, acquiring an original image from an image sensor.

[0134] The image acquisition request may be used to request to acquire an image from a camera system.

[0135] In some embodiments, the application may send an image acquisition request to the camera system when the image analysis service is triggered. In some embodiments, the image analysis service may include a code scanning service, an object recognition service, a human body recognition service, or a text recognition service. It is understood that in actual applications, the image analysis service is not limited thereto.

[0136] In some implementations, the image acquisition request may carry at least one camera control parameter. In some implementations, the camera control parameter may include focal length, aperture, or shutter.

[0137] In some embodiments, the application may be any application in an electronic device. In some embodiments, the application may be an application that can perform image analysis services based on images, such as identifying specific objects such as QR codes, flowers, plants, animals, and commodities included in an image, text translation, face verification, beauty photography, AR, VR, and MR.

[0138] In some embodiments, the raw image may be a raw image file directly obtained from an image sensor. In some embodiments, the data format of the raw image may be a raw image format (RAW image format, RAW). The RAW image may be an unedited image with internal colors in a wide color gamut.

[0139] S702, processing the target area in the original image to obtain an analysis flow image, wherein the image quality of the target area in the analysis flow image is better than the image quality of the target area in the original image.

[0140] As can be seen from the above, the original image is an image obtained from the image sensor, and the original image includes more pixel information. By optimizing the target area in the original image, these more pixel information can be fully utilized, and the optimization process can obtain an image with higher image quality. Among them, the image quality of an image is better, which can mean that the image includes more meaningful features, such as color, shape, and texture, so that the object or other target included in the image can be recognized faster and more accurately; on the contrary, the image quality of an image is poor, which can mean that the image includes fewer meaningful features, such as color, shape, and texture.

[0141] In some embodiments, the target area may be an area including preset image content. It may be identified whether the original image includes an image feature corresponding to the preset image content. If the original image includes the image feature, it may be determined that the original image includes the preset image, and then the area where the preset image content is located is determined as the target area. In some embodiments, the preset image content may include specific objects such as QR codes, flowers, animals, and commodities, text, or a human body.

[0142] For example, the preset image content includes a QR code, and the image features corresponding to the QR code may include three positioning boxes. Then it can be identified whether the original image includes three positioning boxes. If so, it can be determined that the original image includes a QR code, and the area where the three positioning boxes are located is the target area.

[0143] For another example, the preset image content includes a face, and the image features corresponding to the face may include facial feature points. Then it can be identified whether the original image includes facial feature points. If so, it can be determined that the original image includes a face, and the area where the face is located is the target area.

[0144] It is understandable that the preset image content and the image features corresponding to the preset image content can be obtained by presetting. It is also understandable that the preset image content can also include more or less image content, and the method of identifying whether the original image includes the preset image content is also not limited to the above-mentioned method.

[0145] In some embodiments, the target area may be an area of ​​a preset position, preset size, and preset shape. For example, the target area may be a rectangular area of ​​a preset size centered at the center of the original image. It is understood that in actual applications, the target area may also be an area of ​​other positions, other sizes, or other shapes.

[0146] It should be noted that the target area may be at least a part of the original image. When the target area is smaller, the duration of executing S702 is shorter, and the efficiency of providing the image to the application and the efficiency of the application performing the image analysis service is higher.

[0147] In some embodiments, the optimization process of the target area in the original image may include performing image restoration process on the target area through an image restoration algorithm. In some embodiments, the image restoration process may include at least one of denoising, deblurring, and geometric correction. Of course, in practical applications, the image restoration process may include more or less processing methods that can improve image quality.

[0148] In some embodiments, the target area of ​​the original image includes a first graphic code, and the optimization processing of the target area in the original image may include obtaining code information carried by the first graphic code, generating a second graphic code based on the code information (i.e., reversely generating the graphic code), and replacing the first graphic code of the target area with the second graphic code, wherein the code information carried by the second graphic code is the same as the code information carried by the first graphic code. Since the second graphic code is newly generated, the image quality of the second graphic code is higher than that of the first graphic code, and since the code information carried by the first graphic code and the second graphic code is the same, the adverse impact on subsequent image analysis services caused by replacing the graphic code can be reduced.

[0149] In some embodiments, the graphic code may include a QR code or a barcode. It is understood that in practical applications, the graphic code may also include other graphics that can carry code information. It is also understood that the embodiments of the present application do not limit the specific content of the code information carried by the graphic code. For example, the code information may include text information or a network address.

[0150] In some embodiments, see Figure 8 The schematic diagram of the method for processing an image shown can directly optimize the target area in the original image to obtain an analysis flow image.

[0151] In some embodiments, see Fig. 9 The schematic diagram of the method for processing an image shown in the figure can obtain a local image of a target area from an original image, optimize the local image to obtain a new local image, replace the target area of ​​the original image with the new local image, and obtain an analysis stream image, wherein the image quality of the new local image is better than the image quality of the local image before processing. By first obtaining a local image of a target area from an original image and then continuing to optimize the local image, the amount of data required for the optimization process can be reduced, the efficiency of the optimization process can be improved, more pixel information in the original image can be retained, and the success rate of subsequent application programs performing image analysis services can be improved.

[0152] In some embodiments, if the local image of the target area includes a first graphic code, code information carried by the first graphic code may be obtained, and a new local image may be generated based on the code information, wherein the new local image includes a second graphic code.

[0153] In some implementations, the duration of executing S702 may be shorter than the time interval for the camera system to acquire two frames of raw images, so as to ensure that multiple frames of continuous raw images can be processed.

[0154] S703: Send the analysis stream image to the application program, where the analysis stream image is used by the application program to perform image analysis service.

[0155] Since the original image acquired from the image sensor can include more pixel information, more pixel information can be utilized when processing the target area in the original image, so that the image quality of the analysis stream image is better than that of the original image at least in the target area. Therefore, sending the analysis stream image to the application improves the efficiency of providing the application with images with higher image quality, reduces the number of interactions generated by the application to obtain images with higher image quality, and improves the efficiency of the application in performing image analysis services.

[0156] In an embodiment of the present application, in response to an image acquisition request of an application, an original image can be acquired from an image sensor, and the target area in the original image can be optimized to obtain an analysis stream image. Since the original image is an image acquired from an image sensor, the original image includes more pixel information. Therefore, by optimizing the target area in the original image, these more pixel information can be fully utilized, so that the image quality of the target area in the analysis stream image is better than the image quality of the target area in the original image, that is, the analysis stream image obtained by the optimization process has a higher image quality. Then, providing the analysis stream image to the application improves the efficiency of providing the application with images with higher image quality, reduces the number of interactions generated by the application to obtain images with higher image quality, and improves the efficiency of the application in performing image analysis services.

[0157] Please refer to Fig.10 , which is a flow chart of a shooting method provided in an embodiment of the present application. Fig.10 The method shown can be Figure 7 The detailed description of the method shown in FIG. 1 is used to determine whether the optimization operation needs to be performed before the original image is optimized. It should be noted that the method does not Fig.10 The specific order described below is a limitation, and it should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0158] S1001, in response to an image acquisition request from an application, an original image is acquired from an image sensor. Thereafter, S1002 or S1003 is executed.

[0159] Among them, the method of executing S1001 can be the same as or similar to the method of executing S701, and will not be described in detail here.

[0160] S1002, determine whether the application is a preset application. If yes, execute S1003 or S1004, otherwise execute S1006.

[0161] When the application is a preset application, the following S1004-S1005 can be executed to provide the application with an analysis stream image with higher image quality, so as to improve the efficiency of the preset application in performing image analysis business; when the application is not a preset application, the image quality provided to the application may not be improved to reduce system energy consumption.

[0162] In some implementations, it can be determined whether the application belongs to a preset program set. If the application identifier belongs to the preset program set, the application is determined to be a preset application. Otherwise, the application is determined not to be a preset application. The program set may include at least one application, such as an application identifier of the application. In some implementations, the program set may also be referred to as a whitelist.

[0163] In some embodiments, the application identifier of the application can be used to identify the application. In some embodiments, the application identifier of the application can include the name of the activity (activity component) of the application. It is understandable that in actual applications, the application identifier of the application can also include other information that can identify the application.

[0164] In some embodiments, the application identifier of the application may be determined based on the image acquisition request. In some embodiments, the application may send the application identifier of the application when sending the image acquisition request. It is understandable that in actual applications, the application identifier of the application may also be obtained by other means.

[0165] In some implementations, S1002 may be omitted.

[0166] S1003, determining whether the original image includes the preset image content, if yes, executing S1004, otherwise executing S1006.

[0167] When the original image includes preset image content, the following S1004-S1005 can be executed to provide the application with an analysis stream image with higher image quality, so as to improve the efficiency of the application in performing image analysis services; when the original image does not include preset image content, the image quality provided to the application may not be improved, so as to reduce system energy consumption.

[0168] In some embodiments, image recognition may be performed on at least a portion of the original image to determine whether the original image contains preset image content. In some embodiments, it may be determined whether the original image includes a graphic code, and if so, S1004 is executed, otherwise S1006 is executed.

[0169] In some implementations, S1003 may be omitted.

[0170] S1004, optimizing the target area in the original image to obtain an optimized original image, wherein the image quality of the target area in the optimized original image is better than the image quality of the target area in the original image.

[0171] Among them, the method of executing S1004 can be the same as or similar to the method of executing S702, and will not be described in detail here.

[0172] S1005: Send an analysis stream image to the application program, where the analysis stream image is the original image after optimization processing.

[0173] S1006: Send an analysis stream image to the application, where the analysis stream image is the original image.

[0174] It should be noted that the method of executing S1005 and S1006 may be the same as or similar to the method of executing S703.

[0175] In an embodiment of the present application, before providing an image to an application, it can be determined whether the application is a preset application and / or whether the original image includes preset image content. Therefore, when it is determined whether the application is a preset application and / or when it is determined that the original image includes preset image content, an analysis stream image with higher image quality is provided to the application to improve the efficiency of the preset application in performing image analysis services. In other cases, the image quality provided to the application is not improved, thereby saving system energy consumption.

[0176] Please refer to Fig.11 , which is a flow chart of a shooting method provided in an embodiment of the present application. Fig.11 The method shown can be Figure 7 The method described in detail can realize sending the downsampled image to the application. It should be noted that this method does not Fig.11 The specific order described below is a limitation, and it should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0177] S1101 , in response to an image acquisition request of an application program, acquiring an original image from an image sensor.

[0178] Among them, the method of executing S1101 can be the same as or similar to the method of executing S701, and will not be described in detail here.

[0179] S1102, optimizing the target area in the original image to obtain an optimized original image, wherein the image quality of the target area in the optimized original image is better than the image quality of the target area in the original image.

[0180] It should be noted that the method of executing S1102 may be the same as or similar to the method of executing S702 described above, and will not be described in detail here.

[0181] S1103, down-sampling the optimized original image to obtain an analysis flow image.

[0182] Since the original image is directly obtained from the image sensor, the size of the original image after optimization is also relatively large. When the application performs image analysis, the size of the input image may be required to be smaller than the size of the image obtained by the image sensor. Therefore, in order to ensure the reliability of the application in performing image analysis, the original image after optimization can be downsampled. That is, the analysis stream image can be the original image after optimization and downsampling.

[0183] Among them, downsampling, also called downsampling, can be understood as reducing the image and reducing the number of pixels included in the image. In some embodiments, a pixel can be taken every at least one pixel, and the obtained multiple pixels are used as a new image. In some embodiments, the image is divided into multiple windows, the average of the pixels in each window is used as a new pixel, and the pixels taken from the multiple windows are used as a new image. It is understood that in practical applications, downsampling can be achieved by other means.

[0184] S1104, sending the analysis flow image to the application.

[0185] The method of executing S1104 may be the same as or similar to the method of executing S703, and will not be described in detail here.

[0186] S1105, displaying a preview stream image.

[0187] The preview stream image may be sent to the display so that the display displays the preview stream image. By displaying the preview stream image, the user can preview the current shooting process, which is convenient for the user to control the camera system.

[0188] In some embodiments, the preview stream image may be an original image or an image obtained by processing the original image, wherein the processing of the original image may include the aforementioned optimization processing and / or downsampling of the target area. In some embodiments, the analysis stream image obtained in S1103 may be shunted to obtain a preview stream image. When the displayed image is the same as the image sent to the application, the image previewed by the user can be synchronized with the image used by the application to perform the image analysis service, and what the user sees is what the application obtains, thereby improving the reliability of the interaction between the user and the application.

[0189] In some implementations, S1105 may be omitted, that is, the preview stream image may not be displayed.

[0190] In an embodiment of the present application, since the original image is acquired directly from the image sensor, the size of the original image after optimization processing is also relatively large, so the original image after optimization processing can be downsampled to obtain a smaller analysis stream image, and the downsampled analysis stream image is sent to the application, thereby improving the problem that the application cannot perform image analysis services due to the large size of the image.

[0191] Please refer to Fig.12 , which is a flow chart of a shooting method provided in an embodiment of the present application. Fig.12 The method shown can be used for Figure 7 The detailed description of the method shown in the figure can downsample the original image obtained from the image sensor in advance, so as to provide an image preview while sending the image to the application. Fig.12 The specific order described below is a limitation, and it should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0192] S1201, in response to an image acquisition request of an application, an original image is acquired from an image sensor. Afterwards, S1202 and S1204 may be executed.

[0193] Among them, the method of executing S1201 can be the same as or similar to the method of executing S701, and will not be described in detail here.

[0194] S1202, down-sample the original image to obtain a down-sampled original image.

[0195] Among them, the method of executing S1202 can be the same as or similar to the method of executing S1103, and will not be described in detail here.

[0196] S1203: Display the preview stream image.

[0197] The preview stream image may be a down-sampled original image.

[0198] It should be noted that S1203 may also be omitted.

[0199] S1204, obtaining a local image of the target area from the original image.

[0200] S1205, optimizing and down-sampling the local image to obtain a new local image, wherein the image quality of the new local image is better than the image quality of the local image before optimization.

[0201] In some implementations, the local image may be optimized first, and then the optimized local image may be downsampled to obtain a new local image.

[0202] Since the local image is a partial image from the original image, and the original image includes more pixel information, optimizing the local image first can make use of more pixel information output by the image sensor and improve the image quality of the new local image.

[0203] S1206: Replace the target area in the downsampled original image with a new local image to obtain an analysis flow image.

[0204] In some embodiments, the downsampled original image generated in S1202 can be shunted to obtain the downsampled original image generated in S1202, and then the target area in the obtained downsampled original image is replaced with a new local image. Then, when the displayed preview stream image is also the downsampled original image, the image previewed by the user can be more similar to the image used by the application to perform image analysis services, thereby improving the reliability of the user's interaction with the application.

[0205] Among them, the method of executing S1204-S1206 can refer to the relevant description in executing S702, which will not be repeated here.

[0206] S1207, sending the analysis flow image to the application.

[0207] Among them, the method of executing S1207 can be the same as or similar to the method of executing S703, and will not be described in detail here.

[0208] In some embodiments, the interface including the preview stream image displayed in S1203 may be a first interface. When the application performs image analysis based on the analysis stream image to obtain business information, the application may send the business information to the display, and the display displays a second interface, which may include the preview stream image and the business information. Among them, since the business information is information generated by the application performing image analysis based on the analysis stream image, the higher the similarity between the analysis stream image and the preview stream image, the higher the matching degree between the business information displayed in the second interface and the preview stream image, and the process of switching from the first interface to the second interface is smoother, further reducing the user's perception of the process of obtaining the analysis stream image. In some embodiments, the application may send business information and a display position corresponding to the business information to a display. The display position may be determined based on an analysis stream image. If the display displays the business information in a second interface based on the display position, the higher the similarity between the analysis stream image and the preview stream image, the smaller the difference between the relative position of the display position in the analysis stream image and the relative position of the display position in the preview stream image. This reduces the user's perception of the process of obtaining the analysis stream image and improves the user experience.

[0209] For example, the first interface includes the aforementioned Figure 4 The second interface includes the shooting interface 420 described above. The image 1 in the shooting interface 420 may be an image obtained through S1202, and the image 2 obtained by the code scanning application may be an image obtained through S1204-S1207. The difference between the image 1 displayed in the shooting interface 420 and the image 2 obtained by the code scanning application may only include the image quality of the target area, such as the target area in the image 1 includes a relatively blurred QR code, and the target area in the image 2 includes a relatively clear QR code. When the code scanning application successfully performs the code scanning service based on the image 2, the display position of the circular mark 440 may be determined as the center of the target area based on the image 2. The display may add the circular mark 440 to the display position of the image 1 included in the shooting interface 420 to obtain another shooting interface 430. Since the difference between the image 1 and the image 2 only includes the image quality of the target area, in the other shooting interface 430, the circular mark 440 is also located at the center of the target area. When the user sees the other shooting interface 430, it can be determined that the QR code of the target area has been successfully identified.

[0210] In some embodiments, the duration of executing S1204-S1206 may be less than the time interval between two frames of original images acquired by the image sensor, thereby ensuring that multiple frames of original images acquired by the image sensor can be processed, and the time when the analysis stream image is provided to the application in S1207 is the same as the time when the preview stream image is displayed in S1203, thereby reducing the user's perception of the image optimization processing process, facilitating the continued display of business information of the image analysis business, and improving the user experience.

[0211] In some implementations, the preview stream image in the first interface may also be the original image acquired in S1201 or the analysis stream image generated in S1206. That is, the preview stream image may be the original image or an image obtained by processing the original image.

[0212] In the embodiment of the present application, on the one hand, for the original image obtained from the image sensor, the direct original image can be downsampled and the downsampled original image can be displayed, so as to reduce the difference between the image previewed by the user and the image seen by the user through the glasses, and reduce the user's perception of the image optimization process. On the other hand, for the original image obtained from the image sensor, a local image can be obtained from the target area in the original image, and the local image can be optimized and downsampled to obtain a new local image with higher image quality, and the target area in the downsampled original image can be replaced with the new local image to obtain an analysis stream image, so the target area of ​​the analysis stream image and the target area of ​​the preview stream image include similar image content, and the image quality of the target area of ​​the analysis stream image is higher, which not only enables the application to improve the efficiency and reliability of executing the image analysis business based on the analysis stream image with higher image quality, but also reduces the difference between the analysis stream image provided to the application and the preview stream image displayed, which is convenient for the subsequent display of the business information corresponding to the image analysis business, thereby improving the user experience.

[0213] Please refer to Fig.13 , is a flow chart of a shooting method provided in an embodiment of the present application. It should be noted that the method is not based on Fig.13 The specific order described below is a limitation, and it should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0214] S1301, the application sends an image acquisition request to the camera system.

[0215] S1302: The camera system acquires a raw image from an image sensor.

[0216] S1303: The camera system downsamples the original image to obtain a downsampled original image.

[0217] S1304: The camera system acquires a local image from the target area of ​​the original image.

[0218] In some implementations, it may be determined whether to optimize the original image. If yes, S1305 is executed, otherwise the downsampled original image is sent to the application. The determination of whether to optimize the original image may include at least one of the aforementioned steps S1002 and S1003.

[0219] S1305, the camera system optimizes the local image to obtain an optimized local image.

[0220] S1306: The camera system downsamples the optimized local image to obtain a new local image.

[0221] S1307: The camera system sends the preview stream image to the display system, and sends the analysis stream image to the application.

[0222] The preview stream image may be a downsampled original image, and the analysis stream image may be obtained by replacing a target area of ​​the downsampled original image with a new local image.

[0223] It should be noted that the method of executing S1301-S1307 can refer to the relevant descriptions in the aforementioned embodiments, and will not be described in detail here.

[0224] The display system can display a preview stream image, thereby allowing the user to preview the currently captured image.

[0225] The application can perform image analysis services based on the analysis stream image. In some embodiments, if the application successfully performs the image service, the application can send service information to the display system, and the display system can display the service information. In some embodiments, if the application cannot perform the image analysis service based on the analysis stream image, the application can request the camera system to obtain a new original image again.

[0226] Will Fig.13 and Figure 5 By comparing the methods shown, we can see that Fig.13 The method shown has at least the following advantages:

[0227] Fig.13 The method shown improves the image quality of the analysis stream image sent to the application, improves the problem of the application repeatedly sending camera control parameters to the camera system, and improves the efficiency and reliability of the application in performing image analysis services. For example, in the code scanning scenario, the application does not need to control the camera system multiple times to change the camera control parameters such as the focal length, which reduces the code scanning delay and the success rate of the code scanning.

[0228] Fig.13 The method shown does not modify the preview stream image sent to the display system, reducing the user's perception of the image optimization process, so that a consistent viewing experience can be provided to the user regardless of whether the image is optimized before being sent to the application.

[0229] Fig.13 The method shown can reduce the difference between the analysis stream image and the preview stream image, ensuring accurate display of business information when performing image analysis business in subsequent steps. It does not require application developers to modify the application and has high compatibility.

[0230] Based on the same inventive concept, an embodiment of the present application further provides an electronic device. The electronic device comprises: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method described in the above method embodiment when calling the computer program.

[0231] The electronic device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0232] Based on the same inventive concept, an embodiment of the present application further provides a chip system, which includes a processor coupled to a memory, and the processor executes a computer program stored in the memory to implement the method described in the above method embodiment.

[0233] The chip system may be a single chip or a chip module composed of multiple chips.

[0234] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in the above method embodiment is implemented.

[0235] The embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the electronic device implements the method described in the above method embodiment when executing the computer program product.

[0236] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0237] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0238] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0239] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0240] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0241] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0242] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0243] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0244] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0245] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A shooting method, characterized in that: Used in electronic equipment, including: In response to an image acquisition request of an application of the electronic device, acquiring an original image from an image sensor; Processing the target area in the original image to obtain an analysis flow image, wherein the image quality of the target area in the analysis flow image is better than the image quality of the target area in the original image; The analysis stream image is sent to the application, where the analysis stream image is used by the application to perform image analysis service.

2. The method according to claim 1, characterized in that The method further comprises: The display of the electronic device displays a first interface, wherein the first interface includes a preview stream image, and the preview stream image is the original image or an image obtained by processing the original image.

3. The method according to claim 2, characterized in that After displaying the first interface, the method further includes: The display of the electronic device displays a second interface, where the second interface includes the preview stream image and service information, where the service information is information obtained by the application program performing the image analysis service based on the analysis stream image.

4. The method according to claim 2 or 3, characterized in that: The preview stream image is the original image after downsampling, and the method further includes: Downsampling the original image to obtain the downsampled original image; The step of processing the target area in the original image to obtain an analysis flow image includes: Acquire a local image in the target area from the original image; Optimizing and down-sampling the local image to obtain a new local image; The target area of ​​the downsampled original image is replaced by the new local image to obtain the analysis flow image.

5. The method according to any one of claims 1 to 4, characterized in that: The image analysis service includes a code scanning service, an object recognition service, a human body recognition service or a text recognition service.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Identifying preset image content included in the original image; The area where the preset image content is located is determined as the target area.

7. The method according to any one of claims 1 to 6, characterized in that: The processing of the target area in the original image includes at least one of the following: Performing image restoration processing on the target area, wherein the image restoration processing includes at least one of denoising processing, deblurring processing and geometric correction processing; The first graphic code included in the target area is replaced with a second graphic code, where the second graphic code is a graphic code generated based on the code information carried by the first image code.

8. The method according to any one of claims 1 to 7, characterized in that: The processing of the target area in the original image includes: If the application belongs to a program set, the target area in the original image is processed, and the program set includes at least one application.

9. An electronic device, characterized in that: The electronic device includes an application and a camera hardware abstraction layer HAL; The camera HAL is used to respond to an image acquisition request of the application, acquire an original image from an image sensor, process a target area in the original image, obtain an analysis stream image, wherein the image quality of the target area in the analysis stream image is better than the image quality of the target area in the original image, and send the analysis stream image to the application; The application is used to perform image analysis services based on the analysis stream image.

10. The electronic device according to claim 9, characterized in that: The electronic device includes an application layer and a HAL, the application layer includes the application, and the HAL includes the camera HAL.

11. The electronic device according to claim 9 or 10, characterized in that: The electronic device further comprises a display; The camera HAL is further used to send a preview stream image to the display, where the preview stream image is the original image or an image obtained by processing the original image; The display is used to display a first interface, and the first interface includes the preview stream image.

12. The electronic device according to claim 11, characterized in that: The application is further used to execute the image analysis service based on the analysis stream image, obtain the service information, and send the service information to the display; The display is further used to display a second interface after displaying the first interface, where the second interface includes the preview stream image and the business information.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method according to any one of claims 1 to 8 when calling the computer program.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

15. A computer program product, characterized in that When the computer program product is run on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 8.

Images