An image processing method, apparatus and electronic device

Abstract

The application provides an image processing method, device and electronic equipment. First image data of a first image size is obtained, and then, after an image output size for a target application is obtained, the first image data is processed into second image data of a second image size with a length-width ratio of 1:1 according to the image output size, so that the second image data has a certain proportional relationship with at least part of the first image data, and the second image size is greater than the output image size. Then, the second image data is processed to obtain third image data with the image output size, and the third image data is output.

Description

Technical Field

[0001] This application relates primarily to the field of electronic equipment applications, and more specifically to an image processing method, apparatus, and electronic equipment. Background Technology

[0002] In scenarios where cameras are used on electronic devices such as tablets, the image size captured directly by the camera often differs from the required output image size. This necessitates cropping and / or rotating the captured image to obtain the desired output image size.

[0003] For example, if the screen is rotated during the camera activation process of an application running on an electronic device, the entire application interface, including the image output frame, will rotate with the screen. Figure 1 The scene diagram shown can be referenced if you need the image display orientation within the image output box to remain unchanged. Figure 2 The image processing method shown involves cropping an image (which may have an image size of 4000×3000) to the desired size (such as 1920×1080), rotating it to the desired display orientation, cropping out invalid areas, and then stretching the resulting image to the desired output size.

[0004] It can be seen that, in Figure 2 In the image processing methods shown, the rotation and cropping operation of the image will reduce the field of view of the image and lose more image content; the image stretching operation will not only consume too many processing resources, but also change the focal length of the image and reduce the quality of the output image. Summary of the Invention

[0005] To address the above problems, this application provides the following technical solution:

[0006] This application proposes an image processing method, the method comprising:

[0007] Obtain first image data with a first image size, and the image output size for the target application;

[0008] Based on the image output size, the first image data is processed to obtain second image data with a second image size; wherein at least a portion of the first image data and the second image data have a proportional relationship, the aspect ratio of the second image size is 1:1, and the second image size is larger than the image output size;

[0009] The second image data is processed to obtain third image data having the image output size;

[0010] Output the third image data.

[0011] Optionally, processing the first image data according to the image output size to obtain second image data with a second image size includes:

[0012] If the aspect ratio of the first image size is 1:1, determine the first side length of the image output size; the first side length refers to the maximum side length in the image output size.

[0013] Based on the first side length of the image output size, the first image data is scaled to obtain second image data with a second image size.

[0014] Optionally, scaling the first image data according to the first side length of the image output size to obtain second image data with a second image size includes:

[0015] Based on the first side length of the image output size, determine the scaling ratio for the first image data;

[0016] The first image data is scaled according to the scaling ratio to obtain second image data with a second image size.

[0017] Optionally, scaling the first image data according to the first side length of the image output size to obtain second image data with a second image size includes:

[0018] Based on the first side length of the image output size, a candidate image size that matches the first side length is selected from a plurality of preset candidate image sizes as the second image size; the aspect ratio of the plurality of candidate image sizes is 1:1.

[0019] The first image data is scaled according to the second image size to obtain second image data with the second image size.

[0020] Optionally, processing the first image data according to the image output size to obtain second image data with a second image size includes:

[0021] If the aspect ratio of the first image size is not 1:1, determine the second side length of the first image size; the second side length refers to the minimum side length of the first image size, and the second side length is greater than the maximum side length of the image output size;

[0022] The first image data is cropped according to the second side length of the first image size to obtain a fourth image data with a third image size; the aspect ratio of the third image size is 1:1.

[0023] The fourth image data is determined to be second image data with a second image size, or the fourth image data is scaled according to the first side length of the image output size to obtain second image data with a second image size; the first side length refers to the maximum side length in the image output size.

[0024] Optionally, processing the first image data according to the image output size to obtain second image data with a second image size includes:

[0025] If the aspect ratio of the first image size is not 1:1, determine the first side length of the image output size; the first side length refers to the maximum side length in the image output size.

[0026] Based on the first side length of the image output size, a cropped equal side length for the first image data is obtained; the cropped equal side length is greater than the first side length.

[0027] The first image data is cropped according to the stated equal side length to obtain a fifth image data with a fourth image size; the aspect ratio of the fourth image size is 1:1.

[0028] The fifth image data is determined as second image data with a second image size, or the fifth image data is scaled according to the first side length of the image output size to obtain second image data with a second image size; the first side length refers to the maximum side length in the image output size.

[0029] Optionally, if the first image data is acquired by the image acquisition device of the electronic device when the electronic device is in a first display state, the method further includes:

[0030] Once the electronic device is determined to switch from the first display state to the second display state, it is triggered to enter the first image processing mode and execute the step of processing the first image data according to the image output size to obtain second image data with the second image size.

[0031] Optionally, processing the second image data to obtain third image data having the image output size includes:

[0032] The second image data is rotated to obtain a sixth image data having the second display state; the sixth image data is the same as the second image data.

[0033] Based on the image output size, the sixth image data is cropped to obtain the third image data with the image output size.

[0034] This application also proposes an image processing apparatus, the apparatus comprising:

[0035] The first image data acquisition module is used to acquire first image data having a first image size;

[0036] Image output size acquisition module, used to obtain the image output size for the target application;

[0037] The second image data acquisition module is used to process the first image data according to the image output size to obtain second image data with a second image size; wherein at least a portion of the image data of the first image data has a proportional relationship with the second image data, the aspect ratio of the second image size is 1:1, and the second image size is larger than the image output size;

[0038] The third image data acquisition module is used to process the second image data to obtain third image data having the image output size;

[0039] The third image data output module is used to output the third image data.

[0040] This application also proposes an electronic device, the electronic device comprising:

[0041] communication devices;

[0042] An image acquisition device is used to acquire first image data of a first image size;

[0043] Image processing apparatus for loading and executing at least one set of computer instructions, including:

[0044] Obtain first image data with a first image size, and the image output size for the target application;

[0045] Based on the image output size, the first image data is processed to obtain second image data with a second image size; wherein at least a portion of the first image data and the second image data have a proportional relationship, the aspect ratio of the second image size is 1:1, and the second image size is larger than the image output size;

[0046] The second image data is processed to obtain third image data having the image output size;

[0047] Output the third image data.

[0048] This application also proposes a computer-readable storage medium storing at least one set of computer instructions that can be loaded and executed by a processor to implement the image processing method proposed in this application. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0050] Figure 1 A schematic diagram of an electronic device screen rotation scenario;

[0051] Figure 2 This is a schematic diagram of an image processing method applicable to scenarios where the screen of an electronic device rotates.

[0052] Figure 3 This is a schematic flowchart of an optional embodiment of the image processing method proposed in this application;

[0053] Figure 4 This is a schematic diagram of an optional application architecture applicable to the image processing method proposed in this application;

[0054] Figure 5 This is a schematic diagram of an optional scenario for the first image data processing process in the image processing method proposed in this application;

[0055] Figure 6 This is a schematic diagram of another optional scenario for the first image data processing process in the image processing method proposed in this application;

[0056] Figure 7 This is a schematic flowchart of an optional embodiment two of the image processing method proposed in this application;

[0057] Figure 8 This is a schematic flowchart of an optional embodiment three of the image processing method proposed in this application;

[0058] Figure 9 This is a schematic flowchart of an optional embodiment four of the image processing method proposed in this application;

[0059] Figure 10 This is a schematic flowchart of an optional embodiment five of the image processing method proposed in this application;

[0060] Figure 11 This is a schematic flowchart of an optional embodiment six of the image processing method proposed in this application;

[0061] Figure 12 This is a schematic diagram of an electronic device screen rotation scenario applicable to the image processing method proposed in this application;

[0062] Figure 13 This is a schematic diagram of another optional embodiment of the image processing method proposed in this application;

[0063] Figure 14 This is a schematic diagram of an optional embodiment of the image processing apparatus proposed in this application;

[0064] Figure 15 This is a schematic diagram of an optional embodiment of an electronic device suitable for the image processing method proposed in this application;

[0065] Figure 16 This is a schematic diagram of another alternative embodiment of an electronic device suitable for the image processing method proposed in this application. Detailed Implementation

[0066] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] Reference Figure 3 This is a flowchart illustrating an optional embodiment of the image processing method proposed in this application. This method can be applied to electronic devices, such as tablet computers, smartphones, and other terminal devices. Figure 3 As shown, the image processing method may include:

[0068] Step S31: Obtain first image data with a first image size, and image output size for the target application;

[0069] In this application embodiment, an image acquisition device (such as a camera) of an electronic device can be activated to acquire images. The directly acquired image data is recorded as the first image data. The first image size of the first image data can be the physical size of the image acquisition element (i.e., the image sensor) of the image acquisition device (i.e., the size of the image acquisition area). Therefore, the first image size can be determined based on the physical size of the image acquisition element. This application does not limit the value of the first image size.

[0070] In different application scenarios of image output, the required image size of the output image may be different. It can be determined according to the application requirements of the image to be output. Therefore, when the target application running on the electronic device or other devices connected to it needs to output the image acquired by the image acquisition device, the image output size required by the target application for the output image can be determined first. It can be one or more different sizes. This application does not limit the target application type and the value of the image output size of the required output image, but can be determined as appropriate.

[0071] For example, refer to Figure 4 The diagram illustrates an optional application architecture applicable to the image processing method proposed in this application. The target application running on the electronic device can communicate with the abstraction layer (Camera HAL) of the image acquisition device via a framework (a software development language that provides a framework for software development; this application can use its software delivery layer for data transmission). Accordingly, the desired image output size can be sent to the framework and then to the Camera HAL layer, enabling the image processing device to execute the image processing method proposed in this application. It should be understood that the final image obtained by the image processing device can be sent from the Camera HAL layer to the framework and then to the target application, where the received image is displayed on the application interface of the target application. The implementation process can be referred to the corresponding description below.

[0072] Step S32: Based on the image output size, process the first image data to obtain second image data with a second image size; at least a portion of the image data of the first image data has a proportional relationship with the second image data, the aspect ratio of the second image size is 1:1, and the second image size is larger than the image output size;

[0073] In light of the problems described in the background section above, this application proposes to reduce image loss and improve output image quality, especially in applications requiring image rotation during acquisition. Specifically, it proposes obtaining square second image data (i.e., second image data with an aspect ratio of 1:1) from the first image data, and then performing subsequent processing on this second image data. It should be noted that to avoid excessive resource consumption due to subsequent stretching of the second image data and reduced output image quality due to changes in the image focal length, the second image size needs to be larger than the output image size. This application does not impose any specific numerical limit on the second image size.

[0074] Based on the above description of the first image data, its first image size is limited by the image acquisition area size of the electronic device's image acquisition device. This means the aspect ratio of the first image may be 1:1, i.e., the first image is a square image. Alternatively, the aspect ratio may not be 1:1; for example, the first image may be a rectangular image. This application does not impose any limitation on the first image size. Therefore, for first image data of different shapes (i.e., different aspect ratios), different processing methods can be used to process the first image data into second image data that is square (i.e., a second image size with an aspect ratio of 1:1).

[0075] In some embodiments, when the aspect ratio of the first image size is not 1:1, such as Figure 5 As shown, the first image data can be processed by cropping to obtain image data with an aspect ratio of 1:1. Then, to reduce system power consumption and improve image processing speed, this 1:1 aspect ratio image data can be further scaled (e.g., a scale operation) to obtain second image data of the second image size. Of course, if the first image size has an aspect ratio of 1:1, it can be directly scaled to obtain second image data with the same aspect ratio. This application does not limit the scaling method for 1:1 aspect ratio image data; interpolation scaling algorithms can be used, but are not limited to them.

[0076] Therefore, after processing the first image data, the resulting second image data has a proportional relationship with a portion of the first image data (such as the image data with an aspect ratio of 1:1 after the above cropping process) or all of the first image data. This proportional relationship can be the scaling ratio of the above scaling process, such as a 1:1 ratio, or a non-1:1 ratio such as 1:2 or 3:4. This ratio can be determined comprehensively based on factors such as the size of the first image, the required image output size, and processing speed and system power consumption. This application does not limit the value of the above proportional relationship.

[0077] Step S33: Process the second image data to obtain third image data with the image output size;

[0078] Step S34: Output the third image data.

[0079] Based on the above description of the image output size, after obtaining the second image data of the square second image, it can be further processed to obtain the third image data of the required image output size. This can be determined based on the image output size and the changes in the display state (such as the display orientation on the screen of an electronic device) between the required output third image data and the acquired first image data. This application does not limit the implementation method of step S33.

[0080] Optional, in such Figure 1 In the scenario shown, where the screen rotation is required, the second image data can be rotated first, such as... Figure 6 The scene illustration shows that the second image data needs to be copied and written to the corresponding pixel positions. Since the aspect ratio of this second image data is 1:1, no invalid areas will be generated after overall rotation. Therefore, no cropping or interpolation processing is needed, and no loss of field of view will occur. This is relative to... Figure 2 The rotation cropping method in the image processing method shown in this embodiment greatly reduces the processing time, avoids image loss in this processing stage, and helps to ensure the quality of the output image.

[0081] After obtaining the rotated image data, such as Figure 6 As shown, since its image size is still larger than the image output size, it can be cropped into third image data of the required image output size without performing a stretching and enlargement process. This avoids the problems of stretching and enlargement consuming processing resources, increasing processing time, and causing image distortion due to stretching, thus ensuring the image quality of the output image and improving the image processing speed.

[0082] It should be understood that, in distinction from Figure 1 In the landscape / portrait screen rotation application scenarios shown, if the display state of the electronic device screen remains unchanged, it is not necessary to rotate the second image data. The second image data can be directly cropped into third image data of the image output size. The implementation process is not detailed in this application. Furthermore, when the target application requires images of different image output sizes, the third image data for each image output size can be implemented according to, but is not limited to, the methods described above. This application is not detailed in this application.

[0083] Referring to the above Figure 4 In the application architecture shown, after the processing device obtains the third image data according to the image processing method proposed in this application, it can send the data to the software transmission layer of the framework, which will then transmit it to the target application in the upper layer to output the corresponding third image. The implementation process of the output of the third image data is not described in detail in this application, but can be determined as appropriate.

[0084] In summary, in the embodiments of this application, after obtaining the first image data of the first image size and the image output size for the target application, the first image data can be processed into second image data with a second image size having an aspect ratio of 1:1 based on the image output size. This ensures that at least a portion of the image data of the second image data has a certain proportional relationship with the first image data, and the second image size is larger than the output image size, minimizing image loss. Subsequent processing of the second image data yields third image data with the image output size, which does not reduce the field of view or require stretching, thus avoiding excessive processing resources and changes in the focal length of the image that would reduce the output image quality. This ensures the image quality of the output third image data and improves image processing efficiency.

[0085] Reference Figure 7 This is a flowchart illustrating an optional embodiment two of the image processing method proposed in this application. This embodiment describes an optional refinement method of the image processing method proposed above. In an application scenario where the aspect ratio of the image acquisition area of ​​the image acquisition device in the electronic device executing this image processing method is 1:1, such as... Figure 7 As shown, the image processing method may include:

[0086] Step S71: Obtain first image data with a first image size, and image output size for the target application;

[0087] The implementation process of step S71 can be referred to the description of the corresponding part of the above embodiment, and will not be described in detail here.

[0088] Step S72: Determine the first side length of the image output size; the first side length refers to the maximum side length in the image output size.

[0089] Based on the above description of the image acquisition device, the aspect ratio of the first image data obtained by the processing device is 1:1. In some embodiments, when it is determined that the first image size is larger than the image output size, the first image data can be directly determined as second image data with a second image size. In this case, the second image size is the same as the first image size, and the first image data can be directly processed into third image data with the image output size, maximizing the quality of the output image. However, when the difference between the first image size and the image output size is large, the processing method of determining the first image data as second image data in this embodiment consumes more system power and also affects the image processing speed.

[0090] To improve the above problems, this application proposes to scale the first image data (in this application, this can be scaling down, i.e. downsampling, and the implementation method is not limited) to obtain second image data of the second image size, ensuring that the aspect ratio of the second image size is still 1:1, but the second image size is smaller than the first image size. Compared with the processing method described in the above embodiments, this can reduce system power consumption and improve image processing speed.

[0091] Based on this, in order to ensure that the second image size of the scaled second image data is greater than the image output size, that is, the side length of the second image size is greater than the maximum side length of the image output size, this embodiment can first determine the maximum side length of the image output size of the required output image, which can be denoted as the first side length of the image output size for ease of description. Then, the first image data can be scaled according to the first side length to obtain the second image data. The scaling process includes, but is not limited to, the implementation methods described in the following steps of this embodiment.

[0092] Step S73: Determine the scaling ratio for the first image data based on the first side length of the image output size;

[0093] Step S74: Scale the first image data according to the scaling ratio to obtain second image data with the second image size;

[0094] Following the above analysis, after obtaining the equal side length of the first image size of the first image data and the maximum side length of the image output size required by the target application (i.e., the first side length), a scaling ratio for the first image data can be obtained. This ensures that the equal side length of the second image size of the scaled second image data is greater than the first side length, reducing image loss in subsequent processing of the second image data. Since the image effects of the second image data processed by different scaling ratios differ, this application can determine the scaling ratio for the first image data based on multiple factors such as the electronic device's requirements for system power consumption, image processing speed, and image output effect. This application does not impose any limitation on the value of this scaling ratio.

[0095] Optionally, the method for obtaining the scaling ratio described above can be based on multiple dimensions of requirements, such as the system power consumption of the electronic device, image processing speed, and image output effect (i.e., image quality, which can be characterized by parameters such as image resolution or image data integrity). Artificial intelligence algorithms such as machine learning / deep learning can be used to process image data of different image sizes at different scaling ratios to determine the scaling ratio that meets the requirements of different dimensions, and the experimental results can be recorded. Based on this, step S73 above can combine the experimental results to obtain the scaling ratio that meets the image processing requirements of the target application, but it is not limited to this method of obtaining the scaling ratio.

[0096] In some embodiments, this application can determine the equal side length of the scaled second image data based on the first side length of the image output size, such as any side length greater than the first side length. To reduce system power consumption and improve processing speed, the first side length can be selected as the equal side length of the second image data. Alternatively, the equal side length can be determined by combining a preset processing error value (which can be a small value; this application does not limit its value). For example, the first side length and the processing error value are added together to obtain the equal side length of the second image data, and the ratio of the equal side lengths of the first and second image data is determined as the scaling ratio. Of course, after obtaining the equal side length of the second image data, the second image data can also be scaled directly according to the equal side length. The implementation process is not described in detail in this application.

[0097] For example, suppose the first image size is 4000×4000 (which represents the number of pixels, and the above side length also refers to the number of pixels on the corresponding side of the image), and the target application requires an image output size of 1920×1080, with a first side length of 1920 pixels. Following the method described above, it is desirable to process and obtain a second image data of 2000×2000. Thus, the scaling ratio for the first image data can be determined to be 1:2 (e.g., the number of pixels in the second image data: the number of pixels in the first image data).

[0098] Based on the above analysis, in order to improve the output image quality, the scaling ratio determined for the first image data in the above example can also be 3:4, thereby obtaining a second image data of 3000×3000. In some embodiments, where higher output image quality is required, the first image data of 4000×4000 can be directly determined as the second image data.

[0099] Step S75: Process the second image data to obtain third image data with the image output size;

[0100] Step S76: Output the third image data.

[0101] The implementation process of steps S75 and S76 can be referred to the description of the corresponding part of the context embodiment, and will not be described in detail here.

[0102] In some embodiments, such as the example above, if the target application also requires third image data with an image output size of 1280×720, that is, if the target application needs multiple images with different image output sizes, the second image data with an aspect ratio of 1:1 can still be obtained according to the method described above. After obtaining the second image data of a second image according to the above method, multiple third image data with different image output sizes can be output according to the methods described in the corresponding parts of steps S76 and S75. The processing of the third image data of each image output size is similar and will not be described in detail in this application.

[0103] Optionally, in order to obtain a third image data with a required output size of 1280×720, and to reduce system power consumption, this application may further scale the second image data obtained by the above scaling process according to the method described above, such as obtaining image data with an output size of 1300×1300 or 1500×1500, and then process it to obtain a third image data with an output size of 1280×720. This application does not limit the implementation method of obtaining different third image data with different output sizes from the same first image data, and can be determined as appropriate.

[0104] Reference Figure 8 This is a flowchart illustrating an optional embodiment three of the image processing method proposed in this application. This embodiment can describe another optional refined implementation method of the image processing method proposed above. In an application scenario where the aspect ratio of the image acquisition area of ​​the image acquisition device in the electronic device executing this image processing method is 1:1, such as... Figure 8 As shown, the image processing method may include:

[0105] Step S81: Obtain first image data with a first image size, and image output size for the target application;

[0106] Step S82: Determine the first side length of the image output size; the first side length refers to the maximum side length in the image output size.

[0107] The implementation process of steps S81 and S82 can be referred to the description of the corresponding parts of the above embodiments, and will not be repeated in this embodiment.

[0108] Step S83: Based on the first side length of the image output size, select a candidate image size that matches the first side length from a plurality of preset candidate image sizes as the second image size; the aspect ratio of these plurality of candidate image sizes is 1:1.

[0109] Step S84: Scale the first image data according to the second image size to obtain second image data with the second image size;

[0110] In the image scaling process proposed in this application, image processing templates with different candidate image sizes of 1:1 aspect ratio can be pre-determined based on the first image size, i.e., the size of the image acquisition area of ​​the image acquisition device, and the size of the image typically output in the actual application of electronic devices. Such templates include multiple candidate image sizes such as 1500×1500, 2000×2000, 2500×2500, 3000×3000, and 3500×3500. For the same image data to be processed, the larger the candidate image size used, the better the output image effect, but the higher the system power consumption and the slower the processing speed. Therefore, this application can flexibly select the required candidate image size according to, but not limited to, the method described above.

[0111] Optionally, taking the example of the target application requiring a third image data with an output size of 1920×1080, according to the above method, in order to minimize system power consumption and improve processing speed, a candidate image size of 2000×2000 can be selected as the second image size. Then, the first image can be scaled directly based on this, such as using an interpolation scaling algorithm to downsample the first image data to obtain the second image data of 2000×2000. Of course, other second image data with different output sizes can also be obtained. The implementation process is similar, and this application will not provide detailed examples of each one.

[0112] In another possible implementation, after obtaining the second image size according to the above method, the scaling ratio for the first image data can be determined based on the ratio between the second image size and the first image size. Then, the first image data is interpolated and scaled according to the scaling ratio to obtain the second image data of the second image size. The interpolation and scaling process is not described in detail in this application, but can be determined based on the operation principle of the interpolation and scaling algorithm.

[0113] Step S85: Process the second image data to obtain third image data with image output size;

[0114] Step S86: Output the third image data.

[0115] The implementation process of steps S85 and S86 can be referred to the description of the corresponding part of the context embodiment, and will not be described in detail in this embodiment.

[0116] In summary, when the aspect ratio of the first image data obtained is 1:1, the first image data can be scaled according to the maximum side length of the image output size. In this embodiment, downsampling can be used to reduce the size of the second image data, thereby reducing the system power consumption of subsequent processing and ensuring a high processing speed. Since the aspect ratio of the second image size is still 1:1 and is greater than the maximum side length of the image output size, the second image data can be directly cropped to obtain a smaller third image data output, eliminating the need to stretch the second image data. This reduces the consumption of processing resources during the process and improves the image processing speed.

[0117] Reference Figure 9 This is a flowchart illustrating an optional embodiment four of the image processing method proposed in this application. This embodiment describes another optional refined implementation of the image processing method proposed above. In application scenarios where the aspect ratio of the image acquisition area of ​​the image acquisition device in the electronic device executing this image processing method is not 1:1, such as... Figure 9 As shown, the image processing method may include:

[0118] Step S91: Obtain first image data with a first image size, and image output size for the target application;

[0119] Step S92: Determine the second side length of the first image size; the second side length refers to the minimum side length of the first image size, and the second side length is greater than the maximum side length of the image output size;

[0120] Step S93: Crop the first image data according to the second side length of the first image size to obtain a fourth image data with a third image size; the aspect ratio of the third image size is 1:1.

[0121] When the aspect ratio of the first image data is not 1:1, in order to obtain the second image data of the square second image, the first image data can be cropped to obtain the fourth image data of the square, and then scaled to obtain the second image data of the required second image size; or, the fourth image data of the third image size with an aspect ratio of 1:1 obtained by cropping the first image data can be directly determined as the second image data. This application does not limit the method of obtaining the second image data in this scenario.

[0122] In the cropping process of the first image data, to ensure that the third image size of the cropped fourth image data is larger than the image output size, the first image data can be cropped based on the maximum side length of the image output size. In practical applications, the minimum side length of the first image size is usually larger than the maximum side length of the image output size. This application can also directly crop the first image data based on the minimum side length of the first image size, such as directly cropping the first image data into a square fourth image data according to the minimum side length.

[0123] For example, if the first image size of the first image data is 4000×3000, the first image data can be cropped into a fourth image data of 3000×3000 according to the method described above; of course, it can also be cropped into a fourth image data of 2500×2500, or even directly cropped into a fourth image data of 2000×2000 (which can be applied to scenarios where it is the second image size), etc. Since the larger the third image size of the cropped fourth image data is, the better the image effect of the subsequent output image, but the greater the system power consumption and the slower the processing speed, the third image size of the cropped fourth image data can be determined by combining these multiple dimension requirements in conjunction with the description of the corresponding part of the above embodiment. The implementation process will not be described in detail in this application.

[0124] Step S94: Based on the first side length of the image output size, the fourth image data is scaled to obtain second image data with a second image size; the first side length refers to the maximum side length in the image output size.

[0125] In order to reduce system power consumption and improve processing speed, when the difference between the third image size and the image output size of the fourth image data is large, the fourth image data can be further scaled to obtain second image data with a smaller second image size. The scaling process in step S94 can be implemented by referring to any of the methods described in the above embodiments, and will not be described in detail in this application embodiment.

[0126] For example, in conjunction with the image processing method described above, in a scenario where a third image data with an output size of 1920×1080 is required, after cropping the first image data of 4000×3000 to a fourth image data of 3000×3000, it can be further scaled to a second image data of 2000×2000. The implementation process can refer to the scaling size of 3:2 described in the above embodiment, or select a second image size of 2000×2000 and then perform interpolation scaling on the fourth image data, etc. The implementation process will not be described in detail in this embodiment.

[0127] Step S95: Process the second image data to obtain third image data with the image output size;

[0128] Step S96: Output the third image data.

[0129] The implementation process of steps S95 and S96 can be referred to the description of the corresponding part of the context embodiment, and will not be described in detail in this embodiment.

[0130] Reference Figure 10 This is a flowchart illustrating an optional embodiment five of the image processing method proposed in this application. This embodiment describes another optional refined implementation of the image processing method proposed above. In application scenarios where the aspect ratio of the image acquisition area of ​​the image acquisition device in the electronic device executing this image processing method is not 1:1, such as... Figure 10 As shown, the image processing method may include:

[0131] Step S101: Obtain first image data with a first image size, and image output size for the target application;

[0132] Step S102: Determine the first side length of the image output size; the first side length refers to the maximum side length in the image output size.

[0133] Step S103: Based on the first side length of the image output size, obtain the cropped equal side length for the first image data; the cropped equal side length is greater than the first side length.

[0134] Step S104: The first image data is cropped according to the equal side length to obtain the fifth image data with the fourth image size; the aspect ratio of the fourth image size is 1:1.

[0135] In conjunction with the description of the relevant parts of the above embodiments, in order to ensure that the fourth image size of the fifth image data after cropping the first image data is greater than the image output size, this embodiment can also determine the equal side length of the cropped first image data based on the maximum side length of the image output size, and denoted as the cropped equal side length. For example, in the scenario where a third image data with an output size of 1920×1080 is required, the cropped equal side length of the first image data can be determined to be 2500×2500, but it is not limited to this value. It can be determined based on multiple dimensions such as system power consumption, processing speed and image processing effect.

[0136] Step S105: Based on the first side length of the image output size, the fifth image data is scaled to obtain second image data with a second image size; the first side length refers to the maximum side length in the image output size.

[0137] Regarding the implementation process of step S105, the scaling processing method described in the above embodiment can be referred to. For example, the scaling ratio can be determined by the ratio between the cropped side length and the equal side length of the required square second image data, and then the scaling processing can be performed accordingly. Alternatively, the second image size can be selected directly from the candidate image sizes, and then the scaling processing can be performed. The implementation process will not be elaborated in the embodiment.

[0138] In some other embodiments, after obtaining the fifth image data of square size according to the above cropping method, the fifth image data can also be directly determined as the second image data with the second image size. The implementation process can be referred to the description of the corresponding part of the above embodiments.

[0139] Step S106: Process the second image data to obtain third image data with image output size;

[0140] Step S107: Output the third image data.

[0141] The implementation process of steps S106 and S107 can be referred to the description of the corresponding part of the context embodiment, and will not be described in detail in this embodiment.

[0142] Reference Figure 11 This is a flowchart illustrating an optional embodiment six of the image processing method proposed in this application. Combined with the detailed implementations of the image processing method described in the preceding embodiments, this embodiment describes the implementation process of the image processing method in application scenarios where the electronic device's screen rotates between portrait and landscape orientations. For the corresponding processing procedures in various detailed scenarios where the image acquisition area size of the electronic device is 1:1 or not 1:1, please refer to the descriptions of the corresponding detailed embodiments above; these will not be detailed here. (Refer to...) Figure 12 The scene diagram shown illustrates that the image processing method may include:

[0143] Step S111: Obtain first image data with a first image size and an image output size for the target application; the first image data is acquired by the image acquisition device of the electronic device when the electronic device is in a first display state.

[0144] In the embodiments of this application, such as Figure 12 The diagram shows a scenario of screen rotation on an electronic device. The first display state can be the portrait state of the electronic device. At this time, the target application running on the electronic device starts the image acquisition device to acquire images. After obtaining the first image data with the first image size, the electronic device maintains the first display state and can enter the first image processing mode. According to the processing method described above, the third image data with the required image output size is obtained and the third image data is output on the application interface of the target application.

[0145] Optionally, while the electronic device remains in the first display state, it can also be triggered to enter the second image processing mode, directly cropping and / or scaling the first image data to obtain the third image data of the required output size. It is not necessary to process the second image data into a square and then further crop it into the third image data of the required size. This process also does not involve stretching, thus ensuring the image processing speed.

[0146] Step S112: Determine that the electronic device switches from the first display state to the second display state, and trigger the electronic device to enter the first image processing mode;

[0147] Step S113: In the first image processing mode, the first image data is processed according to the image output size to obtain second image data with a second image size; at least a portion of the image data of the first image data has a proportional relationship with the second image data, the aspect ratio of the second image size is 1:1, and the second image size is larger than the image output size;

[0148] Following the above analysis, if the electronic device rotates during the process of the target application outputting image data acquired by the image acquisition device, such as... Figure 12 As shown, when rotating from portrait mode (as described in the first display state) to landscape mode (as described in the second display state), in order to ensure that the display direction of the image data remains unchanged, during the synchronous rotation of the target application's interface and the captured image with the screen display state, it is necessary to rotate the image captured by the image acquisition device to achieve the following: Figure 12 The display effect shown.

[0149] Based on this, in order to solve the problems caused by the processing method described in the background section when the electronic device switches from the first display state to the second display state, this application can process the first image data into second image data of square size. The implementation process can be referred to the description of the corresponding part of the above embodiment, and will not be described in detail here.

[0150] In some other embodiments, regardless of the display state of the electronic device or any changes in its display state, the image processing method proposed in the above embodiments of this application can be used to process the acquired first image data to obtain second image data of square size. That is, the electronic device has an image processing mode. Therefore, when it is determined that the electronic device switches from the first display state to the second display state, there is no need to switch the image processing mode. The first image data can be processed directly according to the image output size to obtain second image data with the second image size. The implementation process can be referred to the description of the corresponding part of the above embodiments.

[0151] Step S114: Rotate the second image data to obtain a sixth image data with a second display state; the sixth image data is the same as the second image data.

[0152] As described above regarding the screen rotation scenario of the electronic device in this application, in order to ensure that the display direction of the output image is consistent with the display direction of the output image before rotation, after obtaining the second image data of square size according to the above processing method when the first image data is collected as the screen rotates, the second image data can be rotated first to adjust its image display direction, such as... Figure 6 As shown, this ensures that when the application interface of the target application rotates with the screen, or when the camera, which is the first image data source, rotates with the screen, the display direction of the output image is consistent with the display direction of the output image in the first display state, achieving the desired effect. Figure 12 The image output effect shown.

[0153] Step S115: Based on the image output size, the sixth image data is cropped to obtain the third image data with the image output size;

[0154] Step S116: Output the third image data.

[0155] Reference Figure 6 The illustrated scenario shows that this application rotates the square-sized second image data. It only requires copying the second image data and adjusting the display orientation to obtain the sixth image data. No interpolation or cropping is needed, thus avoiding a reduction in the field of view during this process. Subsequently, the obtained sixth image data is directly cropped to the desired output size as the third image data, without changing the image focal length. Figure 2 The processing method shown can quickly achieve image processing with reduced processing resources, thus improving image processing efficiency.

[0156] For example, suppose the first image data has a first image size of 4000×3000, and the target application requires third image data with sizes of 1920×1080 and 1280×720, such as... Figure 13The illustrated scenario, following the processing method described above, yields a second image data of size 2000×2000. This second image data is then converted to a sixth image data of size 2000×2000. Subsequently, based on the desired output image size, it can be cropped into third image data of sizes 1920×1080 and 1280×720, respectively. These third image data are then sent to the upper-layer target application via the framework transport layer. It should be understood that if multiple third image data sizes are required for the same or different target applications, the acquired first image data can be processed using the image processing method described above. The implementation process is similar, and will not be detailed here.

[0157] Reference Figure 14 This is a schematic diagram of an optional embodiment of the image processing apparatus proposed in this application. This apparatus can be applied to the aforementioned electronic devices, such as... Figure 14 As shown, the image processing apparatus may include:

[0158] The first image data acquisition module 141 is used to acquire first image data having a first image size;

[0159] Image output size acquisition module 142 is used to obtain the image output size for the target application;

[0160] The second image data acquisition module 143 is used to process the first image data according to the image output size to obtain second image data with a second image size; wherein at least a portion of the image data of the first image data has a proportional relationship with the second image data, the aspect ratio of the second image size is 1:1, and the second image size is larger than the image output size;

[0161] The third image data acquisition module 144 is used to process the second image data to obtain third image data having the image output size;

[0162] The third image data output module 145 is used to output the third image data.

[0163] Optionally, if the aspect ratio of the first image size is 1:1, the second image data acquisition module 143 may include:

[0164] The first determining unit is used to determine the first side length of the image output size; the first side length refers to the maximum side length in the image output size.

[0165] The first scaling processing unit is used to scale the first image data according to the first side length of the image output size to obtain second image data with a second image size.

[0166] In one possible implementation, the first scaling processing unit may include:

[0167] The second determining unit is used to determine the scaling ratio for the first image data based on the first side length of the image output size;

[0168] The second scaling processing unit is used to scale the first image data according to the scaling ratio to obtain second image data with a second image size.

[0169] In another possible implementation, the first scaling processing unit may also include:

[0170] The selection unit is used to select, based on the first side length of the image output size, a candidate image size that matches the first side length from a plurality of preset candidate image sizes as the second image size; the aspect ratio of the plurality of candidate image sizes is 1:1.

[0171] The third scaling processing unit is used to scale the first image data according to the second image size to obtain second image data with the second image size.

[0172] Optionally, if the aspect ratio of the first image size is not 1:1, the second image data acquisition module 143 may include:

[0173] The third determining unit is used to determine the second side length of the first image size; the second side length refers to the minimum side length of the first image size, and the second side length is greater than the maximum side length of the image output size;

[0174] The first cropping processing unit is used to crop the first image data according to the second side length of the first image size to obtain fourth image data with a third image size; the aspect ratio of the third image size is 1:1.

[0175] The fourth determining unit is used to determine the fourth image data as second image data having a second image size, or...

[0176] The fourth scaling processing unit is used to scale the fourth image data according to the first side length of the image output size to obtain second image data with a second image size; the first side length refers to the maximum side length in the image output size.

[0177] In some other embodiments, if the aspect ratio of the first image size is not 1:1, the second image data acquisition module 143 may include:

[0178] The fifth determining unit is used to determine the first side length of the image output size; the first side length refers to the maximum side length in the image output size.

[0179] A cropping equal-side-length obtaining unit is used to obtain a cropping equal-side-length for the first image data based on a first side length of the image output size; the cropping equal-side-length is greater than the first side length.

[0180] The fifth scaling processing unit is used to perform cropping processing on the first image data according to the cropping equal side length to obtain the fifth image data with the fourth image size; the aspect ratio of the fourth image size is 1:1.

[0181] The sixth determining unit is used to determine the fifth image data as second image data having a second image size, or...

[0182] The sixth scaling processing unit is used to scale the fifth image data according to the first side length of the image output size to obtain second image data with a second image size; the first side length refers to the maximum side length in the image output size.

[0183] Based on the image processing apparatus described in the embodiments above, if the first image data is acquired by the image acquisition device of the electronic device when the electronic device is in a first display state, the apparatus may further include:

[0184] The first image processing mode triggering module is used to determine that the electronic device switches from the first display state to the second display state, triggering the electronic device to enter the first image processing mode. The second image data acquisition module 143 processes the first image data according to the image output size to obtain second image data with the second image size.

[0185] Based on this, the aforementioned third image data acquisition module 144 may include:

[0186] A rotation processing unit is used to rotate the second image data to obtain a sixth image data having the second display state; the sixth image data is the same as the second image data.

[0187] The second cropping processing unit is used to crop the sixth image data according to the image output size to obtain third image data with the image output size.

[0188] It should be noted that the various modules and units in the above-mentioned device embodiments can all be stored as program modules in the memory of a computer device. The processor of the computer device executes the program modules stored in the memory to achieve the corresponding functions. The functions achieved by each program module and its combination, as well as the technical effects achieved, can be referred to the description of the corresponding part of the above-mentioned method embodiments, which will not be repeated in this embodiment.

[0189] This application also provides a computer-readable storage medium on which at least one set of computer instructions can be stored. This set of computer instructions can be called and loaded by a processor to implement the image processing method described in the above embodiments. The implementation process can be referred to the description of the corresponding part of the above method embodiments.

[0190] Reference Figure 15 This is a schematic diagram of an optional embodiment of an electronic device suitable for the image processing method proposed in this application. The electronic device may include: a communication device 151, an image acquisition device 152, and an image processing device 153, wherein:

[0191] The number of communication device 151, image acquisition device 152, and image processing device 15 can each be at least one, and they can be connected to each other via a bus or other communication methods to realize data transmission. This application only uses... Figure 9 The connection method shown is used as an example for explanation. This bus may include an address bus, a data bus, a control bus, etc. Figure 15 Not all of them are shown; they can be determined based on actual needs, and this application will not elaborate on them.

[0192] The communication module 151 may include a communication module capable of data interaction using a wireless communication network, such as a WIFI module, a 5G / 6G (fifth-generation mobile communication network / sixth-generation mobile communication network) module, a GPRS module, etc., to realize communication between the electronic device and other terminal devices and servers. It should be understood that the communication module 151 may include a communication interface for data interaction between internal components of the electronic device, such as a USB interface, a serial / parallel port, or other multimedia interfaces. This application does not limit the specific content included in the communication module 151.

[0193] The image acquisition device 152 can be used to acquire first image data of a first image size. In this embodiment, the image acquisition device 152 can be an image sensor, and the size of the sensing area, i.e., the image acquisition area, is the first image size. This application does not describe in detail the structure of the image acquisition device 152 and its image acquisition principle.

[0194] The image processing apparatus 153 can be used to load and execute at least one set of computer instructions to implement the image processing method proposed in this application. The implementation process can be referred to the description of the corresponding part of the above method embodiment, which will not be described in detail here.

[0195] In this embodiment, the image processing device 153 may include a combination of multiple hardware circuits such as a microprocessor, microcontroller, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), programmable logic device (PLD), image signal processing circuit, or other suitable hardware, firmware, and / or combinations of hardware and software. This application does not limit the hardware composition structure of the image processing device 153 of the electronic device, and it can be determined in conjunction with the requirements of the image processing method described above.

[0196] In some other embodiments proposed in this application, such as Figure 16 As shown, the image processing device 153 may include a first processor, a second processor, a third processor, and an image transmission circuit. The first processor can be used to implement image signal processing. After receiving the first image data and the image output size, it can process the first image data according to the image output size to obtain second image data with a second image size.

[0197] Optionally, the first processor can be an ISP (Image Signal Processing) Raw processor, which can communicate with the image acquisition device via, but is not limited to, a MIPI bus, and its configuration can be determined based on the physical layer deployment structure of the electronic device. In practical applications, the first processor can also perform one or more preprocessing methods on the first image data, such as format conversion (e.g., converting RGB format image data to YUV format image data), noise reduction, and calibration. Then, the preprocessed image data is processed according to the image processing method described above to obtain second image data of the second image size. The implementation process is not detailed in this application.

[0198] In applications where the screen of an electronic device rotates, the second processor can convert and process the second image data to obtain the sixth image data. The implementation process can be referred to the description of the corresponding part of the above-described embodiment; this application does not elaborate on the processor structure for implementing image rotation. In applications without screen rotation, the second processor can directly forward the obtained second image data to the third processor. Of course, in this scenario, the first processor can also send the obtained second image data to the third processor, depending on the communication connection method of each processor in the image processing device.

[0199] The third processor can be used to process the received image data to obtain third image data with the required image output size. In practical applications, this third processor can be an ISP YUV process processor, which performs image color processing and cropping of the received image data according to the required image output size to obtain at least one channel of third image data with the required image output size. This application does not describe the processor process in detail.

[0200] The image transmission circuit can send at least one channel of third image data with the required image output size, obtained from the third processor, to the upper-layer target application and output the third image data. This image transmission circuit can be a transport layer within a framework architecture; its deployment and implementation methods are not detailed in this application.

[0201] It should be understood that, Figure 15 and Figure 16 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of this application. In practical applications, the electronic device may include more than Figure 15 and Figure 16 The application may include other components, or combinations of certain components, and may also include, as needed, speakers, microphones, various sensors (such as attitude sensors, position sensors, etc., to detect the display status of electronic devices), power modules, antennas, etc., which are not listed here.

[0202] Finally, it should be noted that, regarding the above embodiments, unless the context explicitly indicates an exception, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, and these steps and elements do not constitute an exclusive list; the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0203] In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more.

[0204] The terms used in this application, such as "first" and "second," are for descriptive purposes only, used to distinguish one operation, unit, or module from another, and do not necessarily require or imply any such actual relationship or order between these units, operations, or modules. Furthermore, they should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0205] Furthermore, the various embodiments in this specification are described in a progressive or parallel manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatuses and electronic devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to in the method section.

[0206] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, the method comprising: Obtain first image data having a first image size and the image output size of the target application, wherein the first image data is an image acquired by the image acquisition device of the electronic device when the electronic device is in a first display state; If the electronic device switches from the first display state to the second display state, it processes the first image data according to the image output size to obtain second image data with a second image size; wherein at least a portion of the first image data and the second image data have a proportional relationship, the aspect ratio of the second image size is 1:1, and the second image size is larger than the image output size; The second image data is rotated to obtain a sixth image data having the second display state; the sixth image data is the same as the second image data. Based on the image output size, the sixth image data is cropped to obtain third image data with the image output size; Output the third image data.

2. The method according to claim 1, wherein processing the first image data according to the image output size to obtain second image data having a second image size includes: If the aspect ratio of the first image size is 1:1, determine the first side length of the image output size; The first side length refers to the maximum side length in the image output size; Based on the first side length of the image output size, the first image data is scaled to obtain second image data with a second image size.

3. The method according to claim 2, wherein scaling the first image data according to the first side length of the image output size to obtain second image data with a second image size includes: Based on the first side length of the image output size, determine the scaling ratio for the first image data; The first image data is scaled according to the scaling ratio to obtain second image data with a second image size.

4. The method according to claim 2, wherein scaling the first image data according to the first side length of the image output size to obtain second image data with a second image size includes: Based on the first side length of the image output size, a candidate image size that matches the first side length is selected from a plurality of preset candidate image sizes as the second image size; the aspect ratio of the plurality of candidate image sizes is 1:

1. The first image data is scaled according to the second image size to obtain second image data with the second image size.

5. The method according to claim 1, wherein processing the first image data according to the image output size to obtain second image data having a second image size includes: If the aspect ratio of the first image size is not 1:1, determine the second side length of the first image size; The second side length refers to the minimum side length in the first image size, and the second side length is greater than the maximum side length of the image output size; The first image data is cropped according to the second side length of the first image size to obtain a fourth image data with a third image size; the aspect ratio of the third image size is 1:

1. The fourth image data is determined to be second image data with a second image size, or the fourth image data is scaled according to the first side length of the image output size to obtain second image data with a second image size; the first side length refers to the maximum side length in the image output size.

6. The method according to claim 1, wherein processing the first image data according to the image output size to obtain second image data having a second image size includes: If the aspect ratio of the first image size is not 1:1, determine the first side length of the image output size; The first side length refers to the maximum side length in the image output size; Based on the first side length of the image output size, a cropped equal side length for the first image data is obtained; the cropped equal side length is greater than the first side length. The first image data is cropped according to the stated equal side length to obtain a fifth image data with a fourth image size; the aspect ratio of the fourth image size is 1:

1. The fifth image data is determined as second image data with a second image size, or the fifth image data is scaled according to the first side length of the image output size to obtain second image data with a second image size; the first side length refers to the maximum side length in the image output size.

7. An image processing apparatus, the apparatus comprising: The first image data acquisition module is used to acquire first image data having a first image size, wherein the first image data is an image acquired by the image acquisition device of the electronic device when the electronic device is in a first display state. Image output size acquisition module, used to obtain the image output size of the target application; The second image data acquisition module is configured to, if the electronic device switches from the first display state to the second display state, process the first image data according to the image output size to obtain second image data with a second image size; wherein at least a portion of the first image data has a proportional relationship with the second image data, the aspect ratio of the second image size is 1:1, and the second image size is larger than the image output size; The third image data acquisition module is used to rotate the second image data to obtain a sixth image data having the second display state; the sixth image data is the same as the second image data; according to the image output size, the sixth image data is cropped to obtain a third image data having the image output size; The third image data output module is used to output the third image data.

8. An electronic device, the electronic device comprising: communication devices; An image acquisition device is used to acquire first image data of a first image size; Image processing apparatus for loading and executing at least one set of computer instructions, including: Obtain first image data having a first image size and the image output size of the target application, wherein the first image data is an image acquired by the image acquisition device of the electronic device when the electronic device is in a first display state; If the electronic device switches from the first display state to the second display state, it processes the first image data according to the image output size to obtain second image data with a second image size; wherein at least a portion of the first image data and the second image data have a proportional relationship, the aspect ratio of the second image size is 1:1, and the second image size is larger than the image output size; The second image data is rotated to obtain a sixth image data having the second display state; the sixth image data is the same as the second image data. Based on the image output size, the sixth image data is cropped to obtain third image data with the image output size; Output the third image data.

Images