Image reading method, electronic equipment and computer readable storage medium

Abstract

The invention relates to the technical field of camera shooting, provides an image reading method, electronic equipment and a computer readable storage medium, and is applied to a camera shooting module. The method comprises the following steps: acquiring a plurality of feature pixel units, wherein the plurality of feature pixel units are a plurality of pixel units corresponding to a first feature region in a first image; reading out a plurality of first charge quantities of the plurality of feature pixel units according to a first mode, and reading out a plurality of second charge quantities of the plurality of non-feature pixel units according to a second mode to obtain a second image; wherein the plurality of non-feature pixel units are partial or all pixel units except the plurality of feature pixel units in a pixel array of the camera module, and a first ratio of the number of the plurality of first charge quantities to the number of the plurality of feature pixel units is equal to a first ratio of the number of the plurality of first charge quantities to the number of the plurality of feature pixel units; the first ratio is greater than a second ratio of the number of the plurality of second charge quantities to the number of the plurality of non-characteristic pixel units, and the first ratio is less than or equal to 1. According to the method, the power consumption is reduced while the image definition is improved.

Description

Technical Field

[0001] The present application relates to the field of camera technology, and in particular to an image reading method, an electronic device and a computer-readable storage medium. Background Art

[0002] With the rapid development of camera modules, the performance of camera modules is getting higher and higher, and the quality of the images taken is also getting higher and higher. Usually, people can install the camera module on the terminal device, and then call the camera module to shoot by operating the terminal device.

[0003] However, as the resolution of camera modules increases, the power consumption during shooting also increases. The increasing power consumption poses a greater challenge to the standby time of terminal devices. Summary of the invention

[0004] The present application provides an image reading method, device, chip, electronic device, computer-readable storage medium and computer program product, which can take into account the clarity of the image while reducing power consumption, extending the standby time and reducing heat.

[0005] In a first aspect, an image reading method is provided, which is applied to a camera module, and the method includes: acquiring a plurality of characteristic pixel units, wherein the plurality of characteristic pixel units are a plurality of pixel units corresponding to a first characteristic area in a first image; reading a plurality of first charge quantities of the plurality of characteristic pixel units according to a first mode, and reading a plurality of second charge quantities of a plurality of non-characteristic pixel units according to a second mode, so as to obtain a second image; wherein the plurality of non-characteristic pixel units are part or all of the pixel units in a pixel array of the camera module except the plurality of characteristic pixel units, and a first ratio of the number of the plurality of first charge quantities to the number of the plurality of characteristic pixel units is greater than a second ratio of the number of the plurality of second charge quantities to the number of the plurality of non-characteristic pixel units, and the first ratio is less than or equal to 1.

[0006] The multiple pixel units corresponding to the first feature area may be pixel units corresponding one-to-one to the pixel points in the first feature area, or may be pixel units of an entire row where the pixel units corresponding one-to-one to the pixel points in the first feature area are located.

[0007] The first image may be a preview image of the previous frame. The terminal device identifies the first feature area of ​​the first image. The importance level of the first feature area is higher than the importance level of the first non-feature area. For example, the first feature area may be an important area obtained by identifying the first image, and the first non-feature area may be a non-important area. For multiple feature pixel units corresponding to the first feature area with a high importance level, the charge amount may be read out in a full-image size output mode to obtain multiple first charge amounts, and then the multiple second charge amounts of the first non-feature area may be read out in a pixel-merged output mode. Afterwards, the terminal device generates a second image by amplifying, analog-to-digital converting, and preprocessing the charge amount.

[0008] Optionally, the multiple non-characteristic pixel units may be all pixel units in the pixel array of the camera module except for the multiple characteristic pixel units.

[0009] Optionally, the terminal device may also adopt an output mode with a small pixel merging size to read out multiple first charge quantities, and adopt an output mode with a large pixel merging size to read out multiple second charge quantities. The full-image size output mode can be regarded as an output mode with a pixel merging size of 1*1. Then, as long as the pixel units corresponding to the areas with high importance levels are read out, the charge quantities are read out using the output mode with a small pixel merging size, while for the pixel units corresponding to the areas with low importance levels, the charge quantities are read out using the output mode with a large pixel merging size. Then, the first ratio of the number of the multiple first charge quantities to the number of the multiple characteristic pixel units is greater than the second ratio of the number of the multiple second charge quantities to the number of the multiple non-characteristic pixel units.

[0010] Among them, when the first ratio is equal to 1, it means that the charge is read out by adopting the output mode of the full image size for multiple characteristic pixel units, and the charge is read out by adopting the output mode of pixel merging with a small pixel merging size and not 1*1 for multiple non-characteristic pixel units; when the first ratio is less than 1, it means that the charge is read out by adopting the output mode of pixel merging with a small pixel merging size and not 1*1 for multiple characteristic pixel units, and the charge is read out by adopting the output mode of pixel merging with a large pixel merging size for multiple non-characteristic pixel units.

[0011] Optionally, the multiple non-feature pixel units may be some of the pixel units other than the multiple feature pixel units in the pixel array of the camera module. Another part of the pixel units other than the multiple feature pixel units in the pixel array may also read out the charge using the first mode or the second mode. Optionally, another part of the pixel units other than the multiple feature pixel units in the pixel array may also read out the charge using a pixel merging size between the pixel merging sizes corresponding to the first mode and the second mode. It is understandable that the importance level of the area in the first image corresponding to another part of the pixel units other than the multiple feature pixel units in the pixel array is lower than the first feature area and higher than the second feature area.

[0012] The image reading method can identify the generated image, and after identifying the important area in the image, adopt the full-image size output mode for the pixel units corresponding to the important area, so as to ensure that the resolution of the important area of ​​the image will not be reduced and will not affect the user's browsing experience of the image; and adopt the pixel merging output mode for the pixel units corresponding to the non-important areas outside the important area, which can reduce power consumption, thereby extending the standby time and reducing heat, and is therefore more reasonable.

[0013] In some possible implementations, multiple first charge quantities are used to determine pixel values ​​of a second feature area in a second image, and multiple second charge quantities are used to determine pixel values ​​of a second non-feature area in the second image, where the second non-feature area is part or all of the area in the second image except the second feature area, and the resolution of the second feature area is higher than the resolution of the second non-feature area.

[0014] Since the terminal device uses a first mode with a small pixel merging size to read out the first charge amount of multiple feature pixel units corresponding to the first feature area, and uses a second mode with a large pixel merging size to read out the second charge amount of multiple non-feature pixel units corresponding to the first non-feature area, the second feature area corresponding to the first feature area in the generated second image has high resolution and high definition, and has low resolution and low definition for the second non-feature area corresponding to the first non-feature area. This image reading method can ensure that the clarity of important areas of the image will not be reduced, does not affect the user's experience of browsing the image, and can also appropriately reduce power consumption, so it is more reasonable.

[0015] In some possible implementations, any first charge quantity corresponds to M characteristic pixel units, and any second charge quantity corresponds to N adjacent non-characteristic pixel units, where M and N are both natural numbers, and M is less than N.

[0016] In some possible implementations, M is 1.

[0017] In some possible implementations, N adjacent non-feature pixel units are located in the same row of a pixel array; or, N adjacent non-feature pixel units are located in the same column of the pixel array; or, N adjacent non-feature pixel units are distributed in a square shape, where N is the square of a natural number that is not 1.

[0018] It should be noted that each first charge is the charge of M characteristic pixel units. When M is a natural number greater than 1, the M characteristic pixel units are adjacent characteristic pixel units. The charge amounts sensed by these M characteristic pixel units are superimposed to output a first charge. When M is 1, each characteristic pixel unit reads out a corresponding first charge. Each second charge is the charge of N non-characteristic pixel units. N is a natural number greater than 1, and the N non-characteristic pixel units are adjacent non-characteristic pixel units. The charge amounts sensed by these N non-characteristic pixel units are superimposed to output a second charge.

[0019] Optionally, the N adjacent non-feature pixel units may be non-feature pixel units arranged in sequence in the same row of the pixel array, or non-feature pixel units arranged in sequence in the same column of the pixel array, or adjacent non-feature pixel units distributed in a square, such as 2*2 (N is 4) distribution, 3*3 (N is 9) distribution, 4*4 (N is 16) distribution, etc. Optionally, when M is not 1, the M adjacent non-feature pixel units may be non-feature pixel units arranged in sequence in the same row of the pixel array, or non-feature pixel units arranged in sequence in the same column of the pixel array, or adjacent non-feature pixel units distributed in a square, such as 2*2 (M is 4) distribution, 3*3 (M is 9) distribution, 4*4 (M is 16) distribution, etc. As long as M is less than N.

[0020] In some possible implementations, acquiring multiple characteristic pixel units includes: identifying a first image to obtain a first characteristic region; and acquiring multiple characteristic pixel units corresponding to the first characteristic region.

[0021] In some possible implementations, identifying a first image and obtaining a first feature area includes: using a preset recognition model to identify the first image to obtain the first feature area, where the first feature area is a region where a main object in the first image is located; or, in response to a user's focus operation, determining that the region where a focus object indicated by the focus operation is located is the first feature area.

[0022] Optionally, the first feature area may be an area where a main object in the first image is located. The terminal device uses a preset recognition model to recognize the main object and determines the area where the main object is located as the first feature area.

[0023] Optionally, the terminal device may also use the area where the focus object is located as the first feature area based on the focus operation of the user clicking the focus object.

[0024] The terminal device may obtain a first feature region in the first image by identifying the first image, and then determine a feature pixel unit for generating a pixel value in the first feature region based on the first feature region.

[0025] In some possible implementations, the camera module includes a depth of field camera, which identifies the first image and obtains a first feature area, including: in the depth of field mode, determining through the depth of field camera the area where the first object is located as the first feature area, and the first object is the object in the first image that is closest to the camera module.

[0026] Based on the depth of field camera identifying the area where the nearest object is located as the first feature area, a high-resolution output mode can be automatically adopted for the first nearest object, while a low-resolution readout mode is adopted for other areas, thereby reading out the image without the need for manual operation by the user, with a high degree of automation.

[0027] In some possible implementations, each row of pixel units in the camera module corresponds to a row driving circuit, and any row driving circuit is used to drive the corresponding row of pixel units; multiple characteristic pixel units are pixel units in multiple characteristic pixel unit rows where pixel points of the first characteristic area are distributed, and multiple non-characteristic pixel units are pixel units in multiple non-characteristic pixel unit rows, and there are no identical pixel unit rows in the multiple non-characteristic pixel unit rows and the multiple characteristic pixel unit rows.

[0028] In a camera module where pixel units are refreshed in whole rows, each row of the pixel array corresponds to a row driving circuit, which is used to drive a corresponding row of pixel units to read out the charge.

[0029] Then the terminal device determines that the pixel unit row corresponding to the pixel points of the first characteristic area is a characteristic pixel unit row, and the pixel units in the characteristic pixel unit row may partially correspond to the pixel points of the first characteristic area, and partially correspond to the pixel points outside the first characteristic area in the same row as the pixel points of the first characteristic area. The pixel units in the characteristic pixel unit row are multiple characteristic pixel units, and the terminal device reads out multiple first charge quantities according to the first mode for the pixel units in the characteristic pixel unit row, that is, multiple characteristic pixel units.

[0030] The terminal device may determine that a pixel cell row without characteristic pixel cells is a non-characteristic pixel cell row, and read out multiple second charge amounts according to a second mode for the pixel cells in the non-characteristic pixel cell row, that is, multiple non-characteristic pixel cells.

[0031] This method is applied in the camera module with full-row refresh. It can control the areas with different importance levels to read out the charge according to different output modes in units of pixel units, taking into account both clarity and power consumption, and is highly reasonable.

[0032] In some possible implementations, each row of pixel units in the camera module corresponds to a row driving circuit, and any row driving circuit is used to drive the corresponding row of pixel units; the multiple feature pixel units are pixel units in multiple feature pixel unit rows and some non-feature pixel unit rows, the some non-feature pixel unit rows include at least one non-feature pixel unit row adjacent to the multiple feature pixel unit rows, the some non-feature pixel unit rows are some pixel unit rows in the multiple non-feature pixel unit rows, and the multiple non-feature pixel unit rows also include other non-feature pixel unit rows; the multiple non-feature pixel units are pixel units in other non-feature pixel unit rows, and there are no identical pixel unit rows in the multiple non-feature pixel unit rows and the multiple feature pixel unit rows.

[0033] In a camera module where pixel units are refreshed in whole rows, each row of the pixel array corresponds to a row driving circuit, which is used to drive a corresponding row of pixel units to read out the charge.

[0034] The terminal device can also use the pixel units in at least one row of non-feature pixel unit rows adjacent to the multiple feature pixel unit rows among the multiple non-feature pixel unit rows as feature pixel units, and use the first mode to read out the corresponding first charge amount, so as to ensure that even if slight shaking occurs during the shooting process, the resolution of the important area will not be affected, thereby ensuring the clarity of the important area in the subsequent image output. The other non-feature pixel unit rows use the second mode to read out the corresponding second charge amount. There are no identical pixel unit rows among the multiple non-feature pixel unit rows and the multiple feature pixel unit rows, that is, the multiple non-feature pixel unit rows and the multiple feature pixel unit rows do not overlap.

[0035] In some possible implementations, each pixel unit in the camera module corresponds to a unit driving circuit, and any unit driving circuit is used to drive a corresponding pixel unit; the multiple feature pixel units are multiple pixel units corresponding to the pixel points of the first feature area, and the multiple non-feature pixel units are multiple pixel units corresponding to the pixel points of the first non-feature area. The first non-feature area is an area in the first image corresponding to the multiple non-feature pixel units, and there are no identical pixel units among the multiple feature pixel units and the multiple non-feature pixel units.

[0036] Optionally, the first non-feature area is an area in the first image corresponding to pixel values ​​output by a plurality of non-feature pixel units.

[0037] In a camera module in which each pixel unit is refreshed independently, each pixel unit of the pixel array corresponds to a driving circuit, and the row driving circuit is used to drive a corresponding pixel unit to read out the charge amount.

[0038] In a camera module where each pixel unit is refreshed independently, the terminal device does not need to use the same output mode to read the charge according to the entire row. Instead, it determines the output mode for each pixel unit based on the importance level of the corresponding area, making the division of images read out by different output modes more accurate, further reducing power consumption.

[0039] In some possible implementations, each pixel unit in the camera module corresponds to a row driving circuit, and any row driving circuit is used to drive a corresponding pixel unit; the multiple feature pixel units are multiple pixel units corresponding to pixel points of the first feature area and part of the pixel points of the first non-feature area, and the pixel points of part of the first non-feature area are pixel points adjacent to the pixel points of the first feature area, and there are no identical pixel units among the multiple feature pixel units and the multiple non-feature pixel units.

[0040] In a camera module in which each pixel unit is refreshed independently, each pixel unit of the pixel array corresponds to a driving circuit, and the row driving circuit is used to drive a corresponding pixel unit to read out the charge amount.

[0041] The terminal device may also use some of the pixel units adjacent to or next to the multiple characteristic pixel units among the multiple non-characteristic pixel units as characteristic pixel units, and read out the corresponding first charge amount using the first mode. Some of the adjacent pixel units may be pixel units corresponding to the pixel points of the first non-characteristic area, or may not be pixel units corresponding to the first non-characteristic area. The pixel unit corresponding to the pixel point mentioned here means that the charge amount sensed by the pixel unit is used to generate the pixel value of the pixel point.

[0042] This ensures that even if slight shaking occurs during shooting, the resolution of important areas will not be affected, ensuring the clarity of important areas in the subsequent images.

[0043] In some possible implementations, the multiple non-feature pixel units are some pixel units in the pixel array of the camera module except for the multiple feature pixel units, and the pixel array also includes multiple sub-feature pixel units. The method also includes: reading out multiple third charge quantities of the multiple sub-feature pixel units according to a third mode, a third ratio of the number of the multiple third charge quantities to the number of the multiple sub-feature pixel units is greater than the second ratio and less than the first ratio, and the multiple third charge quantities are used to determine the pixel value of the secondary feature area in the second image; wherein the resolution of the secondary feature area is higher than the resolution of the second non-feature area, and lower than the resolution of the second feature area.

[0044] The terminal device recognizes the first image and can also obtain three areas of different importance levels, including the first feature area with the highest importance level and the first non-feature area with the lowest importance level, as well as the first feature area. The pixel unit corresponding to the pixel point of the first feature area is a sub-feature pixel unit. The terminal device reads out multiple third charge quantities of multiple sub-feature pixel units according to the third mode. The multiple third charge quantities are used to generate pixel values ​​of the pixel points in the second feature area in the second image corresponding to the first feature area. It should be noted that the pixel merging size of the third mode is larger than that of the first mode and smaller than that of the second mode. Therefore, the resolution in the second feature area is higher than that in the second non-feature area, and lower than that in the second feature area.

[0045] In some possible implementations, the first image also includes multiple other areas, each of the multiple other areas has a different importance level, the multiple other areas and the first feature area and the first non-feature area are different areas, the first non-feature area is an area corresponding to multiple non-feature pixel units in the first image, the first area and the second area are any two different areas in the multiple other areas, the first area has a higher importance level than the second area, and the first area and the second area have higher importance levels than the first non-feature area and lower than the first feature area; the method also includes: reading out multiple fourth charge amounts of the pixel unit corresponding to the first area according to the fourth mode, and reading out multiple fifth charge amounts of the pixel unit corresponding to the second area according to the fifth mode; wherein, A fourth ratio of the number of multiple fourth charge quantities to the number of pixel units corresponding to the first area is greater than a fifth ratio of the number of multiple fifth charge quantities to the number of pixel units corresponding to the second area, and both the fourth ratio and the fifth ratio are greater than the second ratio and less than the first ratio; the multiple fourth charge quantities are used to determine the pixel value of the third area in the second image, and the multiple fifth charge quantities are used to determine the pixel value of the fourth area in the second image, the second non-feature area is a partial area of ​​the second image other than the second feature area, and the second image also includes at least the third area and the fourth area, the resolution of the third area is higher than the resolution of the fourth area, and the resolutions of the third area and the fourth area are both higher than the resolution of the second feature area, and lower than the resolution of the second non-feature area.

[0046] When the importance levels of different areas identified by the first image exceed three, in addition to the first feature area and the first non-feature area, at least the first area and the second area may be included. The importance levels of the first area and the second area are between the first feature area and the first non-feature area. The importance level of the first area is higher than that of the second area. For areas with high importance levels, the terminal device uses a mode with a small pixel merging size to read out the charge, and for areas with low importance levels, uses a mode with a large pixel merging size to read out the charge, so that in the obtained second image, the resolution of different areas is proportional to the importance level. In other words, the higher the importance level of the area, the higher the resolution and the higher the clarity; the lower the importance level of the area, the lower the resolution, the lower the clarity, and the lower the power consumption. This method can adapt to images of more importance levels, and the application is more flexible and reasonable.

[0047] In some possible implementations, the time difference between the generation time of the second image and the generation time of the first image is less than or equal to a preset time threshold.

[0048] The preset duration threshold can be a shorter duration, such as 50 milliseconds or 100 milliseconds. The time difference between the generation time of the second image and the generation time of the first image is less than or equal to the preset duration threshold, which can avoid the problem that the output mode and the object in the important area do not match when the second image is subsequently read out due to a large displacement of the camera module after the first image is generated, thereby ensuring the image quality.

[0049] In a second aspect, a camera module is provided, which includes a pixel array and is used to read out an image using any one of the methods described in the first aspect.

[0050] In a third aspect, an image reading device is provided, comprising a unit composed of software and / or hardware, wherein the unit is used to execute any one of the methods in the technical solution described in the first aspect.

[0051] In a fourth aspect, an embodiment of the present application provides a chip, comprising a processor; the processor is used to read and execute a computer program stored in a memory to execute any one of the methods in the technical solution described in the first aspect.

[0052] Optionally, the chip also includes a memory, and the memory is connected to the processor via a circuit or wire.

[0053] Further optionally, the chip also includes a communication interface.

[0054] In a fifth aspect, an electronic device is provided, comprising: a processor, a memory and an interface; the processor, the memory and the interface cooperate with each other so that the electronic device executes any one of the methods in the technical solution described in the first aspect.

[0055] In a sixth aspect, an electronic device is provided, the electronic device comprising any camera module in the technical solution described in the second aspect; or comprising any chip in the technical solution described in the fourth aspect.

[0056] In the seventh aspect, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the processor executes any one of the methods in the technical solution described in the first aspect.

[0057] In an eighth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is run on an electronic device, the electronic device executes any one of the methods in the technical solution described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a schematic diagram of the structure of a terminal device 100 provided in an embodiment of the present application; Figure 2 is a software structure block diagram of the terminal device 100 provided in an embodiment of the present application; Figure 3 is a schematic diagram of an example of a full-size output mode provided in an embodiment of the present application; Figure 4 is a schematic diagram of an output mode of a pixel merging mode provided in an embodiment of the present application; Figure 5 is a structural diagram of a camera module provided in an embodiment of the present application; Figure 6 This is a schematic diagram of an important area and a non-important area provided in an embodiment of the present application; Figure 7 This is a schematic diagram of pixel distribution in which different output modes are adopted in different areas provided in an embodiment of the present application; Figure 8 This is another example of pixel distribution schematic diagram of different regions using different pixel binning sizes provided in an embodiment of the present application; Fig. 9 is a schematic diagram of an example of areas including three levels of importance provided in an embodiment of the present application; Fig.10 It is a schematic diagram of pixel distribution using different pixel binning sizes for three important levels of areas under the control of a routine driving circuit provided in an embodiment of the present application; Fig.11 This is a schematic diagram of pixel distribution with different pixel binning sizes under the control of a single unit driving circuit provided in an embodiment of the present application; Fig.12This is a schematic diagram of pixel distribution using different pixel binning sizes for three important levels of regions under the control of a single unit driving circuit provided in an embodiment of the present application; Fig.13 is a flowchart of an image reading method provided in an embodiment of the present application; Fig.14 It is a structural schematic diagram of an image reading device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0060] In the following, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features.

[0061] The image reading method provided in the embodiments of the present application can be applied to terminal devices equipped with electronic camera modules, such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), cameras, etc. The embodiments of the present application do not impose any restrictions on the specific types of terminal devices.

[0062] For example, Figure 11 is a schematic diagram of the structure of an example of a terminal device 100 provided in an embodiment of the present application. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, a first antenna 1, a second antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0063] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0064] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0065] The controller may be the nerve center and command center of the terminal device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0066] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0067] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0068] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the terminal device 100.

[0069] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the terminal device 100.

[0070] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0071] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0072] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0073] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0074] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0075] The software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 100.

[0076] Figure 2 1 is a software structure diagram of the terminal device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer. The application layer can include a series of application packages.

[0077] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0078] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0079] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0080] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0081] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0082] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0083] The phone manager is used to provide communication functions of the terminal device 100, such as management of call status (including connection, disconnection, etc.).

[0084] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0085] The notification manager enables applications to display notification information in the status bar, which can be used to convey informational messages and disappear automatically after a short stay without user interaction.

[0086] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0087] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0088] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0089] The system library can include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0090] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0091] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0092] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0093] A 2D graphics engine is a drawing engine for 2D drawings.

[0094] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0095] For ease of understanding, the following embodiments of the present application will be described in detail with Figure 1 and Figure 2 Taking the terminal device with the shown structure as an example, the image acquisition method provided in the embodiment of the present application is specifically explained in combination with the accompanying drawings and application scenarios.

[0096] With the rapid development of camera modules, the resolution of camera modules is getting higher and higher, and the clarity of the images taken is also getting higher and higher. The camera module includes a pixel array formed by multiple pixel units. Specifically, when the user uses the terminal device to shoot, the pixel unit of the camera module is sensitive to light and generates electric charge. The camera module can read the charge of each pixel unit through the readout circuit, and these charges can be transmitted and processed in the form of electrical signals. After these charges are processed separately, the pixel value corresponding to each pixel can be generated to generate an image. When the resolution of the camera module is higher, it means that the number of pixel units in the camera module is more, then the number of charges of the pixel units that the readout circuit needs to read is more, and the power consumption is greater.

[0097] Here, the way in which the camera module reads out the charge amount of the charge generated by the pixel unit is called an output mode or a readout mode, and the output mode is used in the following description. Optionally, the camera module here may include the camera described above.

[0098] For high-resolution camera modules, the camera module can use the full-size output mode. In the full-size output mode, the readout circuit can read the sensed charge for each pixel unit separately, that is, the number of pixel values ​​obtained later is equal to the number of pixel units. For details, please refer to Figure 3 As shown, Figure 3 In the example, a pixel array includes 36 pixel units. Figure 3 In a, 1, 2, 3, ..., up to 36, each number represents a pixel unit, and these pixel units form a 6*6 pixel array. When the full-size output mode is used, the distribution of pixel values ​​of the obtained image can be seen as follows Figure 3 As shown in b in , the pixel value and the pixel unit correspond one to one. Optionally, the pixel value includes a color value and / or a brightness value. Figure 3 In b, Gr, Gb, B, and R represent the color values ​​corresponding to the pixel units at the corresponding positions, and these color values ​​can be the same or different. When the camera module adopts the full-size output mode, the resolution of the generated image is the highest. Therefore, the image clarity is high, but the power consumption is high.

[0099] The camera module can also use the pixel binning (Binning) output mode. In this output mode, the camera module can add the induced charges in multiple adjacent pixel units together through the readout circuit, and then read out in a pixel mode. When the pixel binning output mode is used, in the pixel unit such as Figure 4 An example of the distribution of pixel values ​​of the obtained image under the distribution shown in a in FIG. Figure 4 As shown in b. Figure 4 In b, 9 adjacent pixel units jointly output the same pixel value, and the number of pixel values ​​is less than the number of pixel units. These 9 adjacent pixel units are distributed in a square array, which can be called 3*3 pixel units. Such a pixel merging output mode can be called 3*3 Binning. Optionally, in the pixel merging output mode, it can also be that other numbers of adjacent pixel units jointly output the same pixel value, such as 2*2, 4*4 and other distribution forms, which are not limited in the embodiments of the present application. Among them, 2*2, 3*3, and 4*4 can all be called pixel merging sizes (Binning sizes). When the camera module adopts the pixel merging output mode, the resolution of the generated image is reduced. For example, when the pixel merging size adopted is 3*3, the resolution of the generated image is reduced to one-ninth compared to the full image size. Similarly, when the pixel merging size adopted is 2*2, the resolution of the generated image is reduced to one-quarter compared to the full image size. The pixel-merging output mode can increase the photosensitive area and improve the sensitivity to light in dim light, while reducing the amount of image data, which helps to achieve a higher frame rate and reduce power consumption, but it will reduce the overall clarity of the image.

[0100] Optionally, the full-image-size output mode may also be referred to as a pixel-binning output mode in which the pixel-binning size is 1*1.

[0101] The embodiment of the present application provides an image reading method, which can identify the generated image. After identifying the important area in the image, a full-image size output mode is adopted for the pixel units corresponding to the important area, which can ensure that the clarity of the important area of ​​the image will not be reduced and will not affect the user's experience of browsing the image; and a pixel merging output mode is adopted for the pixel units corresponding to the non-important areas outside the important area, which can reduce power consumption, thereby extending the standby time and reducing heat, and is therefore more reasonable.

[0102] Next, combined with the specific structure of the camera module, the specific principle of the camera module output is introduced. Figure 5As shown, the camera module includes: a pixel array, a row driving circuit, a row decoding controller, a column amplifier, a column decoding controller, a clock circuit and a control circuit. Among them, the pixel array includes pixel units arranged in an array. Each row of pixel units corresponds to a driving circuit, and the driving circuits of each row are connected together to convert the light signal generated by the photodiode in the pixel unit sensing the ambient light into an electrical signal by driving the MOS tube. After that, the electrical signal is output to the column amplifier for amplification under the control of the corresponding driving circuit (such as the row driving circuit). Optionally, the electrical signal of each column of pixel units is uniformly amplified. The amplified electrical signal can enter the output amplifier again, and after amplification again, it is converted into a digital signal through an analog-to-digital converter (A / D converter), and then after filtering, denoising and other processing by the preprocessing circuit, it is forwarded to the system chip (system on a chip, SOC) through the interface circuit using MIPI communication, and finally a complete image is generated and displayed on the screen. The clock circuit in the camera module is used to receive the clock signal sent by the SOC and realize the clock synchronization of the camera module based on the clock signal. The control circuit is used to control the row drive circuit to refresh the pixel units row by row, and to instruct the column decoder controller to control the column amplifier to read the charge generated by the pixel unit, that is, to receive the electrical signal of the pixel unit. The SOC also sends control instructions to the control circuit through the I2C bus to indicate the operation mode of the camera module.

[0103] That is to say, in the full-image size output mode, the control circuit controls the row driving circuit and the column driving circuit to select each pixel unit in turn and read out the corresponding charge; in the pixel merging output mode, the control circuit controls the row driving circuit and the column driving circuit to select each group of pixel units in turn according to the pre-set pixel merging size, and read out the total charge of all pixel units in each group of pixel units.

[0104] After understanding the structure and working principle of the camera module, the image reading method involved in the embodiment of the present application is described below.

[0105] First, the terminal device identifies the important area in the image. The method of identifying the important area may include the following identification methods.

[0106] Identification method 1: When a user uses a terminal device to shoot, after the shooting picture is stabilized, the terminal device can recognize the shot image. The image can be a preview image. The terminal device can use the intelligent scene recognition function (using a pre-trained AI recognition model for recognition) to identify the important area (i.e., the first feature area) in the image.

[0107] Optionally, the important area may be an area where a person in the image is located, and the unimportant area (eg, the first non-feature area) may be a background area other than the person.

[0108] Optionally, the important area may also be the area where the main object in the image is located, and the non-important area may be other areas except the important area. When the image is an image of a building, the main object may be the building in the image, then the important area may be the area where the building is located, and the non-important area may be other areas except the area where the building is located. When the image is a landscape image containing a mountain, the main object may be the mountain in the image, then the important area may be the area where the mountain is located, and the non-important area may be other areas except the area where the mountain is located.

[0109] Optionally, the AI ​​recognition model used by the terminal device to identify important areas in the image may be a saliency detection model. The saliency detection model can detect the most salient objects or the objects that are most likely to attract human visual attention. It should be noted that the AI ​​recognition model can be trained using training images labeled with a variety of subjects (also referred to as objects). In other words, the richer the types of labeled subjects involved in the training image, the more accurate the results of the subject cutout model detection. These subjects may include but are not limited to people, buildings, trees, objects (boxes, cars, etc.), mountains, and so on.

[0110] Optionally, the terminal device can also determine the importance levels of different types of objects by learning photos in the album taken by the user. When the AI ​​recognition model recognizes that there is an object with an existing importance level in the image, the important area of ​​the image can be determined according to the importance level corresponding to the object.

[0111] For example, among the photos taken by the user, the photos with people as the main object account for the largest proportion, for example, 70%, followed by photos with food as the main object, accounting for 15%, followed by photos with scenery as the main object, accounting for 5%, and the remaining 10% are other types of subjects. Then the terminal device can determine the importance level of different categories of objects in order from large to small according to the proportion. For example, the importance level of people is greater than that of food, the importance level of food is greater than that of scenery, and the importance level of scenery is greater than that of others. Based on this, when the terminal device recognizes that there are people in the image, it can determine that the area where the people are located is an important area, and the rest are non-important areas; if there are no people but food, the terminal device determines that the area where the food is located is an important area, and the rest are non-important areas; if there are no people and food but scenery, the terminal device determines that the area where the scenery is located is an important area, and the rest are non-important areas. Optionally, the terminal device can also determine that the area where the object with the highest importance level is located is an important area, and other areas are non-important areas when there are any combination of people, food and scenery, or recognize that people are important areas, food is a secondary important area, and scenery is a non-important area.

[0112] Figure 6 a in FIG. 1 shows an image including a person. The area where the person is located is the important area, i.e. Figure 6 The area inside the dotted line in b.

[0113] Identification method 2: When a user uses a terminal device to shoot, he can focus by clicking on the main shooting target in the camera preview interface. For example, if the user clicks on a face to focus, the terminal device can determine that the person whose face needs to be focused is the main shooting target, that is, the focus object. In response to the user's operation of selecting a focus object, the terminal device can identify the focus object selected by the user and identify the focus object as the main object, thereby obtaining an important area where the focus object is located in the preview image.

[0114] Identification method three: When the user uses the terminal device to shoot, the depth of field camera can also be used to identify objects that are close to the user. The terminal device can determine the area in the preview image where the close objects are located as the important area. In the preview image, other areas outside the important area can be considered as non-important areas.

[0115] Secondly, after the terminal device identifies the important area of ​​the image, it uses different output modes to read out the image for the pixel units corresponding to the important area and the non-important area.

[0116] After the terminal device identifies the important area of ​​the image, it can determine the pixel units corresponding to the pixel points in the important area, and then determine the important pixel units (ie, the characteristic pixel units) based on these pixel units.

[0117] Optionally, in a camera module that is refreshed by row, the terminal device may regard the row in the pixel array where the pixel unit corresponding to the important area is located as the important pixel unit row (i.e., the characteristic pixel unit row). Then, in the pixel array, other pixel unit rows except the important pixel unit row may be unimportant pixel unit rows (e.g., non-characteristic pixel unit rows). In the subsequent image generation process, the terminal device may use the full-image size output mode to read out the charge for the important pixel unit rows, and the pixel merging output mode to read out the charge for the unimportant pixel unit rows. In this way, the output image can take into account the clarity of the important areas of the image, and can also reduce power consumption by reducing the clarity of the unimportant areas.

[0118] For example, in a camera module that is refreshed by row, the terminal device is based on Figure 6 The distribution of important pixel unit rows and non-important pixel unit rows can be determined by referring to the image shown in Figure 7 Based on a. Figure 7 The pixel value distribution output by a in can be seen as follows Figure 7 As shown in b. Figure 7 As can be seen from b, the important pixel unit rows use the full-image size output mode, and the output pixel values ​​are dense, that is, the output image has high resolution and high definition; the unimportant pixel unit rows use the pixel merging output mode, and the output pixel values ​​are sparse, that is, the output image has low resolution and low definition. It should be noted that Figure 6 and Figure 7 The number of pixel units, the number of pixel unit rows, and the pixel binning size shown in the figure are examples and do not limit the embodiments of the present application.

[0119] Optionally, when the non-important pixel unit rows are not an integer multiple of the side length of the pixel merging size (for example, in a 3*3 pixel merging size, the side length is 3), that is, when the quotient of the number of non-important pixel unit rows divided by the side length of the pixel merging size is not an integer, the non-important pixel unit rows close to the important pixel unit rows can also be treated as important pixel unit rows and processed in accordance with the output mode of the full image size, and other non-important pixel unit rows far from the important pixel unit rows still use the pixel merging output mode, so that the clarity of the edge of the important area is high, ensuring that the clarity of the important area of ​​the image will not be reduced. The number of pixel unit rows processed in accordance with the pixel merging output mode can be an integer multiple of the side length of the pixel merging size.

[0120] Optionally, when the non-important pixel unit row is not an integer multiple of the side length of the pixel merging size, the important pixel unit row close to the non-important pixel unit row can also be treated as a non-important pixel unit row and processed according to the pixel merging output mode, and other important pixel unit rows far away from the non-important pixel unit row still use the full image size output mode, which can further reduce power consumption without significantly affecting the clarity of the important area. The number of pixel unit rows processed according to the pixel merging output mode can be an integer multiple of the side length of the pixel merging size.

[0121] Optionally, when the non-important pixel unit row is not an integer multiple of the side length of the pixel combination size, the non-important pixel unit row can also be read out according to the output mode of the pixel combination size, and the remaining non-important pixel unit rows that are less than the side length of the pixel combination size can be merged according to the remaining number of rows to read out the charge. For example, if there are 17 non-important pixel unit rows and the pixel combination size is 3*3, then 15 of the 17 non-important pixel unit rows are read out according to the size of 3*3, and the remaining 2 rows can directly read out the charge of the two rows of pixel units according to the row size of 2, that is, the charge is read out with the pixel combination size of 2*3 as 2*3.

[0122] It should be noted that if the number of columns of pixel units is not an integer multiple of the side length of the pixel merging size, the charge of the non-important pixel unit rows is read out according to the output mode of the pixel merging size, and the remaining pixel unit columns that are less than the side length of the pixel merging size are merged according to the remaining number of columns to read out the charge. For example, if there are 17 rows of non-important pixel units and the pixel merging size is 3*3, then 15 of the 17 rows of non-important pixel unit rows are read out according to the size of 3*3, and the remaining 2 rows can directly read out the charge of the two rows of pixel units according to the row size of 2, that is, the charge is read out with the pixel merging size of 2*3 as 2*3. For example, when the number of columns of pixel units is 47, the charge of 45 consecutive columns can be read out according to the size of 2*3 for the remaining 2 rows of pixel units, and the charge of the remaining two columns of pixel units can be read out according to the size of 2*2.

[0123] Optionally, in order to further reduce power consumption, the terminal device may also perform pixel merging at a smaller pixel merging size for important areas and perform pixel merging at a larger pixel merging size for unimportant areas, thereby balancing the clarity and power consumption of the output image. Figure 8As shown, the terminal device can use a smaller 3*3 pixel merging size for important pixel unit rows and a larger 5*5 pixel merging size for unimportant pixel unit rows, thereby further reducing power consumption. It should be noted that when the number of rows or columns of the pixel unit rows using pixel merging is not an integer multiple of the side length of the pixel merging size, the remaining pixel units that are less than the pixel merging size can be merged and output.

[0124] Optionally, the terminal device can also identify the less important areas (less important areas) when identifying the important areas using the above method. Then, in the image, all areas except the important areas and less important areas are unimportant areas. The terminal device can use the full-image size output mode for the pixel unit rows corresponding to the important areas, use a smaller pixel merging size for pixel unit rows corresponding to the less important areas (including less important pixel units) for pixel merging, and use a larger pixel merging size for pixel unit rows corresponding to the less important areas, thereby outputting an image with three resolutions. In this image, the most important area has the highest clarity, the less important area has the second highest clarity, and the less important area has the lowest clarity. Based on this, the terminal device can output more images of different clarity in a more fine-grained manner, and can more reasonably balance the clarity and power consumption of the output image.

[0125] For example, when an image such as Fig. 9 As shown in a in FIG. 1 , when there are multiple characters, the terminal device can identify the area where the person in the front is located as the important area, the area where the person in the back is located as the less important area, and the area of ​​the background as the unimportant area. Afterwards, the terminal device can output different areas in different output modes according to the importance level. The pixel distribution of the obtained image can be seen in FIG. Fig. 9 As shown in b.

[0126] Optionally, when the image includes a person, the important area may be the area where the person's face is located, the less important area may be the area where the body, hair, etc. other than the person's face are located, and the unimportant area may be the background. Optionally, when the image includes a person and a landscape, the important area may be the area where the person is located, the less important area may be the building, object, etc. near the person, and the unimportant area may be the distant background.

[0127] Optionally, the terminal device can also identify more areas of different importance levels according to the specific content in the image, and thus use different pixel binning sizes for processing according to the different importance levels. The embodiment of the present application does not limit the number of importance levels of different areas in the image, and can also divide the importance levels in a more fine-grained manner, for example, including four, five, six, or other more areas of different importance levels. Specifically, the higher the importance level, the larger the pixel binning size, the closer the output mode is to the full image size, the higher the clarity, and the higher the power consumption; the lower the importance level, the larger the pixel binning size, the lower the clarity, and the lower the power consumption.

[0128] Optionally, when using recognition method 1 to determine the important area, the area where the object related to the important area is located can be a secondary important area, and other backgrounds can be non-important areas. For example, if the face of a person is an important area, the secondary important area can be the area where the person's body is located, and the non-important area can be the background of the person; if the area where the person is located is an important area, the secondary important area can be the vehicle the person is riding in, the racket the person is holding, and the guardrail the person is leaning on, and the non-important area can be the background that does not include the important area and the secondary important area.

[0129] When using identification method 3 to determine important areas, after the depth of field camera identifies the closest object, the area where the closest object is located is determined as the important area, and other objects of importance levels are set in turn according to the distance. The closer the object is, the higher the importance level of the area it is in; the farther the object is, the lower the importance level of the area it is in.

[0130] It should be noted that when the number of importance levels is greater, the terminal device can determine the importance level according to the degree of relevance to the important area. The higher the degree of relevance to the important area, the higher the importance level and the smaller the pixel binning size; the lower the degree of relevance to the important area, the lower the importance level and the larger the pixel binning size.

[0131] The above-mentioned method of distinguishing output modes is based on a camera module in which pixel units are refreshed according to entire rows. When the pixel units in the camera module can be refreshed one by one, that is, each pixel unit has a separate unit driving circuit for controlling the readout of the charge sensed by the pixel unit. In this case, the terminal device does not need to execute different output modes according to the entire row, but can read out the image according to different output modes pixel by pixel.

[0132] Specifically, when the pixel units in the camera module can be refreshed one by one, Figure 6 After the image shown in a in the figure is recognized, we get Fig.11The important area and the non-important area are shown in a in FIG. Among them, the important area corresponds to the important pixel unit, and the non-important area corresponds to the non-important pixel unit. The pixel merging method of the obtained image and the distribution of the generated pixel values ​​can be referred to as Fig.11 As shown in b. Fig.11 In b, the pixel units corresponding to the area inside the dotted line are read out using the output mode of the full image size, and can also be read out using the output mode of pixel merging with a small pixel merging size. The pixel units corresponding to the area outside the dotted line are read out using the output mode of pixel merging with a large pixel merging size. It should be noted that adjacent pixel units that are less than the pixel merging size are directly merged and read out, that is, each read-out charge is the sum of the charges of adjacent pixel units that are less than or equal to the pixel merging size. Among them, the adjacent pixel units that are less than or equal to the pixel merging size are the side lengths of the rows whose row sizes are all less than or equal to the pixel merging size, or the side lengths of the columns whose column sizes are all less than or equal to the pixel merging size.

[0133] Optionally, if the camera module is refreshed based on a pixel unit according to a separate pixel unit, when it is identified that there are important areas, less important areas and non-important areas in the image, for example Fig.12 As shown in a in the figure. Then the pixel merging method of the obtained image and the distribution of the generated pixel values ​​can be seen as follows Fig.12 As shown in b. Fig.12 In b, the important area is read out using the output mode of the full image size, the less important area is read out using the output mode of a small pixel merging size, and the unimportant area is read out using the output mode of a large pixel merging size.

[0134] Optionally, when the identified image regions correspond to more importance levels, the terminal device reads out the charge amount according to the importance levels, from the important region with the highest importance level to the non-important region with the lowest importance level, respectively, according to the pixel binning size from small to large. Optionally, the important region with the highest importance level can be read out in a full-image size output mode or in a pixel binning output mode with a small pixel binning size; other regions are read out by increasing the pixel binning size in sequence according to the importance level.

[0135] Optionally, after identifying the important area, the terminal device can also expand 1 to 2 or more pixel units in the opposite direction of the important area (the direction of the non-important area, or the direction of the area with the next level of importance) as the updated important pixel units based on the important pixel units corresponding to the important area, and then use the full-image size output mode for the updated important pixel units, and read out the pixel units in other areas using the pixel merging output mode. Optionally, when there are multiple importance levels, the pixel units corresponding to each importance level can be expanded by 1 to 2 or more pixel units toward the area with the next level of importance or a lower importance level to execute the readout mode corresponding to the pixel merging size corresponding to the importance level. That is, for the identified areas of different importance levels, a certain area is expanded in the direction of the area with the lower importance level to execute the output mode with a smaller pixel merging size. This ensures that even if slight shaking occurs during the shooting process, the resolution of the important area will not be affected, ensuring the clarity of the important area in the subsequent image output.

[0136] Fig.13 : is a flowchart of an example of an image reading method provided in an embodiment of the present application. The method comprises: S1301. Acquire a plurality of characteristic pixel units, where the plurality of characteristic pixel units are a plurality of pixel units corresponding to a first characteristic region in a first image.

[0137] S1302. Read out multiple first charge quantities of multiple characteristic pixel units according to a first mode, and read out multiple second charge quantities of multiple non-characteristic pixel units according to a second mode to obtain a second image; wherein the multiple non-characteristic pixel units are part or all of the pixel units in the pixel array of the camera module except the multiple characteristic pixel units, and a first ratio of the number of the multiple first charge quantities to the number of the multiple characteristic pixel units is greater than a second ratio of the number of the multiple second charge quantities to the number of the multiple non-characteristic pixel units, and the first ratio is less than or equal to 1.

[0138] The multiple pixel units corresponding to the first feature area may be pixel units corresponding one-to-one to the pixel points in the first feature area, or may be pixel units of an entire row where the pixel units corresponding one-to-one to the pixel points in the first feature area are located.

[0139] The first image may be a preview image of the previous frame. The terminal device identifies the first feature area of ​​the first image. The importance level of the first feature area is higher than the importance level of the first non-feature area. For example, the first feature area may be an important area obtained by identifying the first image, and the first non-feature area may be a non-important area. For multiple feature pixel units corresponding to the first feature area with a high importance level, the charge amount may be read out in a full-image size output mode to obtain multiple first charge amounts, and then the multiple second charge amounts of the first non-feature area may be read out in a pixel-merged output mode. Afterwards, the terminal device generates a second image by amplifying, analog-to-digital converting, and preprocessing the charge amount.

[0140] Optionally, the multiple non-characteristic pixel units may be all pixel units in the pixel array of the camera module except for the multiple characteristic pixel units.

[0141] Optionally, the terminal device may also adopt an output mode with a small pixel merging size to read out multiple first charge quantities, and adopt an output mode with a large pixel merging size to read out multiple second charge quantities. The full-image size output mode can be regarded as an output mode with a pixel merging size of 1*1. Then, as long as the pixel units corresponding to the areas with high importance levels are read out, the charge quantities are read out using the output mode with a small pixel merging size, while for the pixel units corresponding to the areas with low importance levels, the charge quantities are read out using the output mode with a large pixel merging size. Then, the first ratio of the number of the multiple first charge quantities to the number of the multiple characteristic pixel units is greater than the second ratio of the number of the multiple second charge quantities to the number of the multiple non-characteristic pixel units.

[0142] Among them, when the first ratio is equal to 1, it means that the charge is read out by adopting the output mode of the full image size for multiple characteristic pixel units, and the charge is read out by adopting the output mode of pixel merging with a small pixel merging size and not 1*1 for multiple non-characteristic pixel units; when the first ratio is less than 1, it means that the charge is read out by adopting the output mode of pixel merging with a small pixel merging size and not 1*1 for multiple characteristic pixel units, and the charge is read out by adopting the output mode of pixel merging with a large pixel merging size for multiple non-characteristic pixel units.

[0143] Optionally, the multiple non-feature pixel units may be some of the pixel units other than the multiple feature pixel units in the pixel array of the camera module. Another part of the pixel units other than the multiple feature pixel units in the pixel array may also read out the charge using the first mode or the second mode. Optionally, another part of the pixel units other than the multiple feature pixel units in the pixel array may also read out the charge using a pixel merging size between the pixel merging sizes corresponding to the first mode and the second mode. It is understandable that the importance level of the area in the first image corresponding to another part of the pixel units other than the multiple feature pixel units in the pixel array is lower than the first feature area and higher than the second feature area.

[0144] The image reading method can identify the generated image, and after identifying the important area in the image, adopt the full-image size output mode for the pixel units corresponding to the important area, so as to ensure that the resolution of the important area of ​​the image will not be reduced and will not affect the user's browsing experience of the image; and adopt the pixel merging output mode for the pixel units corresponding to the non-important areas outside the important area, which can reduce power consumption, thereby extending the standby time and reducing heat, and is therefore more reasonable.

[0145] In some embodiments, multiple first charge quantities are used to determine pixel values ​​of a second feature area in a second image, and multiple second charge quantities are used to determine pixel values ​​of a second non-feature area in the second image, where the second non-feature area is part or all of the area in the second image except the second feature area, and the resolution of the second feature area is higher than the resolution of the second non-feature area.

[0146] Since the terminal device uses a first mode with a small pixel merging size to read out the first charge amount of multiple feature pixel units corresponding to the first feature area, and uses a second mode with a large pixel merging size to read out the second charge amount of multiple non-feature pixel units corresponding to the first non-feature area, the second feature area corresponding to the first feature area in the generated second image has high resolution and high definition, and has low resolution and low definition for the second non-feature area corresponding to the first non-feature area. This image reading method can ensure that the clarity of important areas of the image will not be reduced, does not affect the user's experience of browsing the image, and can also appropriately reduce power consumption, so it is more reasonable.

[0147] In some embodiments, any first charge quantity corresponds to M characteristic pixel units, and any second charge quantity corresponds to N adjacent non-characteristic pixel units, where M and N are both natural numbers, and M is less than N.

[0148] In some embodiments, M is 1.

[0149] In some embodiments, N adjacent non-feature pixel units are located in the same row of the pixel array; or, N adjacent non-feature pixel units are located in the same column of the pixel array; or, N adjacent non-feature pixel units are distributed in a square shape, where N is the square of a natural number that is not 1.

[0150] It should be noted that each first charge is the charge of M characteristic pixel units. When M is a natural number greater than 1, the M characteristic pixel units are adjacent characteristic pixel units. The charge amounts sensed by these M characteristic pixel units are superimposed to output a first charge. When M is 1, each characteristic pixel unit reads out a corresponding first charge. Each second charge is the charge of N non-characteristic pixel units. N is a natural number greater than 1, and the N non-characteristic pixel units are adjacent non-characteristic pixel units. The charge amounts sensed by these N non-characteristic pixel units are superimposed to output a second charge.

[0151] Optionally, the N adjacent non-feature pixel units may be non-feature pixel units arranged in sequence in the same row of the pixel array, or non-feature pixel units arranged in sequence in the same column of the pixel array, or adjacent non-feature pixel units distributed in a square, such as 2*2 (N is 4) distribution, 3*3 (N is 9) distribution, 4*4 (N is 16) distribution, etc. Optionally, when M is not 1, the M adjacent non-feature pixel units may be non-feature pixel units arranged in sequence in the same row of the pixel array, or non-feature pixel units arranged in sequence in the same column of the pixel array, or adjacent non-feature pixel units distributed in a square, such as 2*2 (M is 4) distribution, 3*3 (M is 9) distribution, 4*4 (M is 16) distribution, etc. As long as M is less than N.

[0152] In some embodiments, acquiring multiple characteristic pixel units includes: identifying a first image to obtain a first characteristic region; and acquiring multiple characteristic pixel units corresponding to the first characteristic region.

[0153] In some embodiments, identifying a first image and obtaining a first feature area includes: using a preset recognition model to identify the first image to obtain the first feature area, the first feature area being the area where a main object in the first image is located; or, in response to a user's focus operation, determining that the area where a focus object indicated by the focus operation is located is the first feature area.

[0154] Optionally, the first feature area may be an area where a main object in the first image is located. The terminal device uses a preset recognition model to recognize the main object and determines the area where the main object is located as the first feature area.

[0155] Optionally, the terminal device may also use the area where the focus object is located as the first feature area based on the focus operation of the user clicking the focus object.

[0156] The terminal device may obtain a first feature region in the first image by identifying the first image, and then determine a feature pixel unit for generating a pixel value in the first feature region based on the first feature region.

[0157] In some embodiments, the camera module includes a depth of field camera, identifies the first image, and obtains a first feature area, including: in depth of field mode, determining through the depth of field camera that the area where the first object is located is the first feature area, and the first object is the object in the first image that is closest to the camera module.

[0158] Based on the depth of field camera identifying the area where the nearest object is located as the first feature area, a high-resolution output mode can be automatically adopted for the first nearest object, while a low-resolution readout mode is adopted for other areas, thereby reading out the image without the need for manual operation by the user, with a high degree of automation.

[0159] In some embodiments, each row of pixel units in the camera module corresponds to a row driving circuit, and any row driving circuit is used to drive the corresponding row of pixel units; multiple feature pixel units are pixel units in multiple feature pixel unit rows where pixel points of the first feature area are distributed, and multiple non-feature pixel units are pixel units in multiple non-feature pixel unit rows, and there are no identical pixel unit rows in the multiple non-feature pixel unit rows and the multiple feature pixel unit rows.

[0160] In a camera module where pixel units are refreshed in whole rows, each row of the pixel array corresponds to a row driving circuit, which is used to drive a corresponding row of pixel units to read out the charge.

[0161] Then the terminal device determines that the pixel unit row corresponding to the pixel points of the first characteristic area is a characteristic pixel unit row, and the pixel units in the characteristic pixel unit row may partially correspond to the pixel points of the first characteristic area, and partially correspond to the pixel points outside the first characteristic area in the same row as the pixel points of the first characteristic area. The pixel units in the characteristic pixel unit row are multiple characteristic pixel units, and the terminal device reads out multiple first charge quantities according to the first mode for the pixel units in the characteristic pixel unit row, that is, multiple characteristic pixel units.

[0162] The terminal device may determine that a pixel cell row without characteristic pixel cells is a non-characteristic pixel cell row, and read out multiple second charge amounts according to a second mode for the pixel cells in the non-characteristic pixel cell row, that is, multiple non-characteristic pixel cells.

[0163] This method is applied in the camera module with full-row refresh. It can control the areas with different importance levels to read out the charge according to different output modes in units of pixel units, taking into account both clarity and power consumption, and is highly reasonable.

[0164] In some embodiments, each row of pixel units in the camera module corresponds to a row driving circuit, and any row driving circuit is used to drive the corresponding row of pixel units; the multiple feature pixel units are pixel units in multiple feature pixel unit rows and some non-feature pixel unit rows, the some non-feature pixel unit rows include at least one row of non-feature pixel unit rows adjacent to the multiple feature pixel unit rows, the some non-feature pixel unit rows are some pixel unit rows in the multiple non-feature pixel unit rows, and the multiple non-feature pixel unit rows also include other non-feature pixel unit rows; the multiple non-feature pixel units are pixel units in other non-feature pixel unit rows, and there are no identical pixel unit rows in the multiple non-feature pixel unit rows and the multiple feature pixel unit rows.

[0165] In a camera module where pixel units are refreshed in whole rows, each row of the pixel array corresponds to a row driving circuit, which is used to drive a corresponding row of pixel units to read out the charge.

[0166] The terminal device can also use the pixel units in at least one row of non-feature pixel unit rows adjacent to the multiple feature pixel unit rows among the multiple non-feature pixel unit rows as feature pixel units, and use the first mode to read out the corresponding first charge amount, so as to ensure that even if slight shaking occurs during the shooting process, the resolution of the important area will not be affected, thereby ensuring the clarity of the important area in the subsequent image output. The other non-feature pixel unit rows use the second mode to read out the corresponding second charge amount. There are no identical pixel unit rows among the multiple non-feature pixel unit rows and the multiple feature pixel unit rows, that is, the multiple non-feature pixel unit rows and the multiple feature pixel unit rows do not overlap.

[0167] In some embodiments, each pixel unit in the camera module corresponds to a unit driving circuit, and any unit driving circuit is used to drive a corresponding pixel unit; the multiple feature pixel units are multiple pixel units corresponding to the pixel points of the first feature area, and the multiple non-feature pixel units are multiple pixel units corresponding to the pixel points of the first non-feature area. The first non-feature area is an area in the first image corresponding to the multiple non-feature pixel units, and there are no identical pixel units among the multiple feature pixel units and the multiple non-feature pixel units.

[0168] Optionally, the first non-feature area is an area in the first image corresponding to pixel values ​​output by a plurality of non-feature pixel units.

[0169] In a camera module in which each pixel unit is refreshed independently, each pixel unit of the pixel array corresponds to a driving circuit, and the row driving circuit is used to drive a corresponding pixel unit to read out the charge amount.

[0170] In a camera module where each pixel unit is refreshed independently, the terminal device does not need to use the same output mode to read the charge according to the entire row. Instead, it determines the output mode for each pixel unit based on the importance level of the corresponding area, making the division of images read out by different output modes more accurate, further reducing power consumption.

[0171] In some embodiments, each pixel unit in the camera module corresponds to a row driving circuit, and any row driving circuit is used to drive a corresponding pixel unit; the multiple characteristic pixel units are multiple pixel units corresponding to the pixel points of the first characteristic area and part of the pixel points of the first non-characteristic area, and the pixel points of part of the first non-characteristic area are pixel points adjacent to the pixel points of the first characteristic area, and there are no identical pixel units among the multiple characteristic pixel units and the multiple non-characteristic pixel units.

[0172] In a camera module in which each pixel unit is refreshed independently, each pixel unit of the pixel array corresponds to a driving circuit, and the row driving circuit is used to drive a corresponding pixel unit to read out the charge amount.

[0173] The terminal device may also use some of the pixel units adjacent to or next to the multiple characteristic pixel units among the multiple non-characteristic pixel units as characteristic pixel units, and read out the corresponding first charge amount using the first mode. Some of the adjacent pixel units may be pixel units corresponding to the pixel points of the first non-characteristic area, or may not be pixel units corresponding to the first non-characteristic area. The pixel unit corresponding to the pixel point mentioned here means that the charge amount sensed by the pixel unit is used to generate the pixel value of the pixel point.

[0174] This ensures that even if slight shaking occurs during shooting, the resolution of important areas will not be affected, ensuring the clarity of important areas in the subsequent images.

[0175] In some embodiments, the multiple non-feature pixel units are some pixel units in the pixel array of the camera module except for the multiple feature pixel units, and the pixel array also includes multiple sub-feature pixel units. The method also includes: reading out multiple third charge quantities of the multiple sub-feature pixel units according to a third mode, a third ratio of the number of the multiple third charge quantities to the number of the multiple sub-feature pixel units is greater than the second ratio and less than the first ratio, and the multiple third charge quantities are used to determine the pixel value of the secondary feature area in the second image; wherein the resolution of the secondary feature area is higher than the resolution of the second non-feature area, and lower than the resolution of the second feature area.

[0176] The terminal device recognizes the first image and can also obtain three areas of different importance levels, including the first feature area with the highest importance level and the first non-feature area with the lowest importance level, as well as the first feature area. The pixel unit corresponding to the pixel point of the first feature area is a sub-feature pixel unit. The terminal device reads out multiple third charge quantities of multiple sub-feature pixel units according to the third mode. The multiple third charge quantities are used to generate pixel values ​​of the pixel points in the second feature area in the second image corresponding to the first feature area. It should be noted that the pixel merging size of the third mode is larger than that of the first mode and smaller than that of the second mode. Therefore, the resolution in the second feature area is higher than that in the second non-feature area, and lower than that in the second feature area.

[0177] In some embodiments, the first image also includes multiple other areas, and the importance levels of each of the multiple other areas are different. The multiple other areas and the first feature area and the first non-feature area are different areas. The first non-feature area is an area corresponding to multiple non-feature pixel units in the first image. The first area and the second area are any two different areas in the multiple other areas. The importance level of the first area is higher than that of the second area. The importance levels of the first area and the second area are higher than the first non-feature area and lower than the first feature area. The method also includes: reading out multiple fourth charge amounts of the pixel unit corresponding to the first area according to the fourth mode, and reading out multiple fifth charge amounts of the pixel unit corresponding to the second area according to the fifth mode; wherein, multiple A fourth ratio of the number of fourth charge quantities to the number of pixel units corresponding to the first area is greater than a fifth ratio of the number of multiple fifth charge quantities to the number of pixel units corresponding to the second area, and both the fourth ratio and the fifth ratio are greater than the second ratio and less than the first ratio; the multiple fourth charge quantities are used to determine the pixel value of the third area in the second image, and the multiple fifth charge quantities are used to determine the pixel value of the fourth area in the second image, the second non-feature area is a partial area of ​​the second image other than the second feature area, and the second image also includes at least the third area and the fourth area, the resolution of the third area is higher than the resolution of the fourth area, and the resolutions of the third area and the fourth area are both higher than the resolution of the second feature area, and lower than the resolution of the second non-feature area.

[0178] When the importance levels of different areas identified by the first image exceed three, in addition to the first feature area and the first non-feature area, at least the first area and the second area may be included. The importance levels of the first area and the second area are between the first feature area and the first non-feature area. The importance level of the first area is higher than that of the second area. For areas with high importance levels, the terminal device uses a mode with a small pixel merging size to read out the charge, and for areas with low importance levels, uses a mode with a large pixel merging size to read out the charge, so that in the obtained second image, the resolution of different areas is proportional to the importance level. In other words, the higher the importance level of the area, the higher the resolution and the higher the clarity; the lower the importance level of the area, the lower the resolution, the lower the clarity, and the lower the power consumption. This method can adapt to images of more importance levels, and the application is more flexible and reasonable.

[0179] In some embodiments, the time difference between the generation time of the second image and the generation time of the first image is less than or equal to a preset time threshold.

[0180] The preset duration threshold can be a shorter duration, such as 50 milliseconds or 100 milliseconds. The time difference between the generation time of the second image and the generation time of the first image is less than or equal to the preset duration threshold, which can avoid the problem that the output mode and the object in the important area do not match when the second image is subsequently read out due to a large displacement of the camera module after the first image is generated, thereby ensuring the image quality.

[0181] The above describes in detail an example of the method provided by the present application. It is understandable that, in order to implement the above functions, the corresponding device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0182] The present application can divide the image reading device into functional modules according to the above method example. For example, each function can be divided into each functional module, or two or more functions can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0183] Fig.14The schematic diagram of the structure of an image reading device provided by the present application is shown. The device 1400 includes: The acquisition module 1401 is used to acquire a plurality of characteristic pixel units, where the plurality of characteristic pixel units are a plurality of pixel units corresponding to a first characteristic region in a first image.

[0184] The output module 1402 is used to read out multiple first charge quantities of multiple characteristic pixel units according to a first mode, and to read out multiple second charge quantities of multiple non-characteristic pixel units according to a second mode to obtain a second image; wherein the multiple non-characteristic pixel units are part or all of the pixel units in the pixel array of the camera module except the multiple characteristic pixel units, and a first ratio of the number of the multiple first charge quantities to the number of the multiple characteristic pixel units is greater than a second ratio of the number of the multiple second charge quantities to the number of the multiple non-characteristic pixel units, and the first ratio is less than or equal to 1.

[0185] The specific manner in which the device 1400 executes the image reading method and the beneficial effects produced can be found in the relevant description of the method embodiment, which will not be repeated here.

[0186] The present application also provides an electronic device, including the above processor. The electronic device provided by this embodiment can be Figure 1 The terminal device 100 shown is used to perform the above-mentioned image reading method. In the case of an integrated unit, the terminal device may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the terminal device, for example, it can be used to support the terminal device to execute the steps performed by the display unit, the detection unit and the processing unit. The storage module can be used to support the terminal device to execute and store program codes and data, etc. The communication module can be used to support the communication between the terminal device and other devices.

[0187] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other terminal devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0188] In one embodiment, when the processing module is a processor and the storage module is a memory, the terminal device involved in this embodiment may be a Figure 1 Device of the structure shown.

[0189] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the image reading method described in any of the above embodiments.

[0190] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the image reading method in the above-mentioned embodiment.

[0191] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0192] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, the replaced units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0193] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0195] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image reading method, characterized in that: Applied to a camera module, the method comprises: Acquire a plurality of characteristic pixel units, where the plurality of characteristic pixel units are a plurality of pixel units corresponding to a first characteristic region in the first image; Reading out a plurality of first charge quantities of the plurality of characteristic pixel units according to a first mode, and reading out a plurality of second charge quantities of the plurality of non-characteristic pixel units according to a second mode, so as to obtain a second image; Among them, the multiple non-characteristic pixel units are part or all of the pixel units in the pixel array of the camera module except the multiple characteristic pixel units, and a first ratio of the number of the multiple first charge quantities to the number of the multiple characteristic pixel units is greater than a second ratio of the number of the multiple second charge quantities to the number of the multiple non-characteristic pixel units, and the first ratio is less than or equal to 1.

2. The method according to claim 1, characterized in that: The multiple first charge quantities are used to determine the pixel value of the second feature area in the second image, and the multiple second charge quantities are used to determine the pixel value of the second non-feature area in the second image, the second non-feature area is part or all of the area in the second image except the second feature area, and the resolution of the second feature area is higher than the resolution of the second non-feature area.

3. The method according to claim 1 or 2, characterized in that: Any one of the first charge amounts corresponds to M characteristic pixel units, and any one of the second charge amounts corresponds to N adjacent non-characteristic pixel units, where M and N are both natural numbers, and M is less than N.

4. The method according to claim 3, characterized in that M is 1.

5. The method according to claim 3, characterized in that: The N adjacent non-feature pixel units are located in the same row of the pixel array; or, The N adjacent non-feature pixel units are located in the same column of the pixel array; or, The N adjacent non-feature pixel units are distributed in a square shape, where N is the square of a natural number that is not 1.

6. The method according to claim 1 or 2, characterized in that: The step of obtaining a plurality of characteristic pixel units comprises: recognizing the first image to obtain the first characteristic region; The plurality of characteristic pixel units corresponding to the first characteristic region are obtained.

7. The method according to claim 6, characterized in that The identifying the first image to obtain the first feature area includes: The first image is recognized by using a preset recognition model to obtain the first characteristic region, where the first characteristic region is a region where a main object in the first image is located; or In response to a focus operation by a user, an area where a focus object indicated by the focus operation is located is determined as the first feature area.

8. The method according to claim 6, characterized in that The camera module includes a depth-of-field camera, and the identifying the first image to obtain the first feature area includes: In the depth of field mode, the area where the first object is located is determined by the depth of field camera as the first feature area, and the first object is the object in the first image that is closest to the camera module.

9. The method according to claim 1 or 2, characterized in that: Each row of pixel units in the camera module corresponds to a row driving circuit, and any row driving circuit is used to drive a corresponding row of pixel units; The multiple characteristic pixel units are pixel units in multiple characteristic pixel unit rows where the pixel points of the first characteristic area are distributed, and the multiple non-characteristic pixel units are pixel units in multiple non-characteristic pixel unit rows. There are no identical pixel unit rows between the multiple non-characteristic pixel unit rows and the multiple characteristic pixel unit rows.

10. The method according to claim 1 or 2, characterized in that: Each row of pixel units in the camera module corresponds to a row driving circuit, and any row driving circuit is used to drive a corresponding row of pixel units; The plurality of characteristic pixel units are pixel units in the plurality of characteristic pixel unit rows and some non-characteristic pixel unit rows, the some non-characteristic pixel unit rows include at least one non-characteristic pixel unit row adjacent to the plurality of characteristic pixel unit rows, the some non-characteristic pixel unit rows are some pixel unit rows in the plurality of non-characteristic pixel unit rows, and the plurality of non-characteristic pixel unit rows also include other non-characteristic pixel unit rows; The plurality of non-feature pixel units are pixel units in the other non-feature pixel unit rows, and there is no identical pixel unit row between the plurality of non-feature pixel unit rows and the plurality of feature pixel unit rows.

11. The method according to claim 2, characterized in that Each pixel unit in the camera module corresponds to a unit driving circuit, and any one of the unit driving circuits is used to drive a corresponding pixel unit; The multiple feature pixel units are multiple pixel units corresponding to the pixel points of the first feature area, the multiple non-feature pixel units are multiple pixel units corresponding to the pixel points of the first non-feature area, the first non-feature area is the area in the first image corresponding to the multiple non-feature pixel units, and there are no identical pixel units among the multiple feature pixel units and the multiple non-feature pixel units.

12. The method according to claim 11, characterized in that Each pixel unit in the camera module corresponds to a row driving circuit, and any row driving circuit is used to drive a corresponding pixel unit; The multiple feature pixel units are multiple pixel units corresponding to the pixel points of the first feature area and part of the pixel points of the first non-feature area, the pixel points of the part of the first non-feature area are pixel points adjacent to the pixel points of the first feature area, and there are no identical pixel units among the multiple feature pixel units and the multiple non-feature pixel units.

13. The method according to claim 2, characterized in that The plurality of non-characteristic pixel units are some pixel units in the pixel array of the camera module except the plurality of characteristic pixel units, the pixel array also includes a plurality of sub-characteristic pixel units, and the method further includes: reading out a plurality of third charge quantities of the plurality of sub-feature pixel units according to a third mode, wherein a third ratio of the number of the plurality of third charge quantities to the number of the plurality of sub-feature pixel units is greater than the second ratio and less than the first ratio, and the plurality of third charge quantities are used to determine a pixel value of a secondary feature area in the second image; The resolution of the second characteristic region is higher than that of the second non-characteristic region, and lower than that of the second characteristic region.

14. The method according to claim 2, characterized in that The first image also includes a plurality of other regions, each of which has a different importance level, the plurality of other regions and the first characteristic region and the first non-characteristic region are different regions, the first non-characteristic region is a region corresponding to the plurality of non-characteristic pixel units in the first image, the first region and the second region are any two different regions among the plurality of other regions, the importance level of the first region is higher than that of the second region, and the importance levels of the first region and the second region are higher than the first non-characteristic region and lower than the first characteristic region; The method further comprises: Reading out a plurality of fourth charge quantities of the pixel units corresponding to the first area according to a fourth mode, and reading out a plurality of fifth charge quantities of the pixel units corresponding to the second area according to a fifth mode; wherein a fourth ratio of the number of the plurality of fourth charge quantities to the number of pixel units corresponding to the first region is greater than a fifth ratio of the number of the plurality of fifth charge quantities to the number of pixel units corresponding to the second region, and the fourth ratio and the fifth ratio are both greater than the second ratio and less than the first ratio; The multiple fourth charge quantities are used to determine the pixel value of the third area in the second image, and the multiple fifth charge quantities are used to determine the pixel value of the fourth area in the second image. The second non-feature area is a partial area of ​​the second image excluding the second feature area. The second image also includes at least the third area and the fourth area. The resolution of the third area is higher than that of the fourth area. The resolutions of the third area and the fourth area are both higher than that of the second feature area, and lower than that of the second non-feature area.

15. The method according to claim 1 or 2, characterized in that: The time difference between the generation time of the second image and the generation time of the first image is less than or equal to a preset time threshold.

16. A camera module, characterized in that: The camera module includes a pixel array, and the camera module is used to read out images using the method described in any one of claims 1 to 15.

17. An electronic device, characterized in that: include: Processors, memory, and interfaces; The processor, the memory and the interface cooperate with each other so that the electronic device executes the method according to any one of claims 1 to 15; or, Comprising any camera module as described in claim 16.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 15.

Images