Image content processing method and electronic device

Abstract

An image content processing method and an electronic device, used for improving the processing effect of image content processing of electronic devices, thereby improving the use experience of users. In the method, when a scenario involving image content processing is detected, the quality of an image used for image content processing is analyzed, and when it is detected that the quality of the image is low, prompt information can be displayed. In this way, guidance is performed on the basis of image quality, so that a user can be prompted to improve the photographing mode of a camera, thereby improving the quality of the image used for image content processing, and further improving the processing effect of image content processing.

Description

An image content processing method and electronic device

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411976268.9, filed on December 26, 2024, entitled "An Image Content Processing Method and Electronic Device", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of electronic equipment technology, and in particular to an image content processing method and an electronic device. Background Technology

[0004] With increasing globalization, image translation, as a technology that overcomes language barriers, is becoming increasingly important. Image translation refers to the technology of translating source language text in an image into target language text through layout analysis, character recognition, machine translation, and other techniques. This technology is widely used in scenarios such as multilingual learning, international travel, and business communication, helping people overcome language barriers and more easily access and understand information in images.

[0005] Electronic devices, as essential tools in users' daily lives, can facilitate image translation by using them as a medium, providing convenience for users to overcome language barriers. Currently, while image translation technology on electronic devices can meet basic communication needs across languages, it suffers from limitations in image quality when translating images from real-time photos or historical images, resulting in suboptimal translation quality.

[0006] In addition, similar problems exist in scenarios that rely on image content recognition based on taking photos, such as image search, image text recognition, and image recognition, which are similar to image translation. Summary of the Invention

[0007] This application provides an image content processing method and electronic device to improve the effect of image content recognition based on images obtained from a shooting scene, thereby improving the accuracy of image content processing. Images obtained from a shooting scene can be, for example, preview images during the shooting process, photos obtained from the shooting process, or historical photos selected by the user.

[0008] A first aspect provides an image content processing method, the execution subject of which is an electronic device. In this method, an instruction to execute a preset task is received, the preset task including an image content recognition step, the preset task including image translation or image search; a first image for the image content recognition is acquired, the first image being captured by a camera; and a first prompt message is displayed, the first prompt message indicating that the first image is of low quality and indicating an improvement in the way the camera captures the image to enhance the effect of the image content recognition. For example, the first image can be captured by the camera of the current electronic device, such as a preview image during the capture process, a captured photograph, or a photograph captured in the past. Another example is that the first image can also be captured by the camera of another electronic device and then transmitted to the current electronic device.

[0009] This method analyzes the quality of the images used for image content recognition. When low image quality is detected, it sends a reminder to the user and provides guidance on improving shooting techniques. By guiding the user's shooting process, the probability of obtaining high-quality images is increased, enabling image content recognition based on high-quality images. This improves the effectiveness of image content recognition, such as enhancing image translation or image search performance.

[0010] In one possible implementation, the method of improving the camera's capture is indicated from at least one of the following aspects:

[0011] (1) Improve the stability of the camera during shooting;

[0012] (2) Point the camera at the content to be identified;

[0013] (3) Adjust the shooting angle;

[0014] (4) Avoid reflective areas;

[0015] (5) Adjust the size of the target content in the image; and

[0016] (6) Adjust the exposure level during shooting.

[0017] In this embodiment, by analyzing the reasons for the low quality of the image, corresponding improvement methods can be generated, thereby improving the efficiency and accuracy of image capture improvement, as well as the efficiency of image content recognition.

[0018] In one possible implementation, after displaying the first prompt message, the method further includes, but is not limited to, the following processing:

[0019] (i) In response to acquiring a second image of higher quality than the first image, perform the preset task based on the second image; or

[0020] (ii) In response to the failure to acquire other images of higher quality than the first image within a preset time, the preset task is executed based on the first image; or

[0021] (iii) In response to the user's confirmation instruction to perform the preset task using the first image, perform the preset task according to the first image.

[0022] In this embodiment, after displaying prompts for guiding the shooting, a higher-quality image can be acquired to perform a preset task; alternatively, if a higher-quality image cannot be acquired, a preset task can be performed based on a first image; or, the preset task can be performed directly based on the first image. Therefore, this application provides multiple possible implementation scenarios to meet different needs in different scenarios and enhance the diversity of image content processing scenarios.

[0023] In one possible implementation, the method further includes: acquiring a third image for image content recognition, the third image being captured by a camera; displaying a second prompt message indicating that the quality of the third image meets the preset task; and performing the preset task based on the third image.

[0024] In this implementation, by displaying a prompt indicating high quality when a high-quality image is detected, the user can be prompted to promptly select a high-quality image for image content processing. This increases the probability of selecting a high-quality image for processing when performing a preset task, thereby improving the processing effect of image content processing.

[0025] In one possible implementation, the first image and / or the second image is a preview image before the electronic device takes the picture, or an image taken by the electronic device, or an image stored on the electronic device.

[0026] In this implementation, by providing multiple entry points for generating images for image content processing in scenarios where preset tasks are performed, the user needs in various scenarios can be met, thereby improving the user experience.

[0027] In one possible implementation, the first image and / or the second image is a single-frame image or a multi-frame image. The multi-frame image can be a moving image or a video.

[0028] In this implementation, image content processing can be performed not only based on a single frame image, but also based on multiple frames images, which can meet the needs of users in various scenarios and thus improve the user experience.

[0029] In one possible implementation, the first image, the second image, or the third image is a moving image. Performing the preset task based on the first image, the second image, or the third image includes: selecting the frame with the best quality from the moving image for image content recognition, or performing image content recognition based on multiple frames from the moving image; and performing the preset task based on the result of the image content recognition. The multiple frames can be some or all frames from the moving image.

[0030] In this embodiment, in the scenario of image content recognition based on dynamic images, by taking advantage of the characteristic that dynamic images include multiple frames, the frame with the best quality can be selected or generated for image content recognition, thereby improving the processing effect of image content processing.

[0031] In one possible implementation, the method further includes: sending request information to a server, the request information being generated based on the first image; receiving image recognition enhancement information from the server, the image recognition enhancement information being generated based on the request information; and performing the preset task based on the first image and the image recognition enhancement information.

[0032] In this embodiment, by leveraging the large data volume of the server, correction or completion can be performed based on the first image, thereby improving the accuracy of image content recognition and enhancing the effect of image content processing.

[0033] The above implementation methods can be achieved in various ways.

[0034] One possible implementation of the method further includes: performing image content processing on the first image to obtain a first candidate result; sending the first image to a server; receiving an associated image from the server; performing image content processing on the associated image to obtain a second candidate result; and fusing the first candidate result and the second candidate result to obtain the processing result.

[0035] In this implementation, by retrieving information from the cloud and obtaining associated images, the electronic device can correct or complete the first candidate result obtained from the first image using the second candidate result obtained from the associated images, thereby obtaining an updated first candidate result, which is the final processing result, and improving the processing effect of image content processing.

[0036] Another possible implementation of the method further includes: performing image content processing on the first image to obtain a first candidate result; sending the first image to a server; receiving a second candidate result from the server, the second candidate result being generated based on the first image; and fusing the first candidate result and the second candidate result to obtain the processing result.

[0037] In this implementation, information is retrieved from the cloud to obtain a second candidate result based on the associated image. This allows the electronic device to directly use the second candidate result to correct or complete the first candidate result obtained from the first image, thus obtaining an updated first candidate result, which is the final processing result, improving the image content processing effect. Furthermore, this implementation can reduce the processing load on the electronic device.

[0038] Another possible implementation of the method further includes: sending the first image to a server; receiving a second image from the server, wherein the second image is retrieved based on the first image, or the second image is generated based on the first image; and fusing the first image and the second image to obtain the processing result.

[0039] In this implementation, by retrieving information from the cloud and obtaining related images, the electronic device can correct or complete the first image based on the related images, thereby obtaining an updated first image. Finally, image content processing is performed based on the updated first image to obtain the processing result, thus improving the processing effect of image content processing.

[0040] The second aspect provides another image content processing method, in which the executing entity is an electronic device. In this method, an instruction to execute a preset task is received, the preset task including the step of image content recognition; acquiring a dynamic image for the image content recognition, the dynamic image being captured by a camera; selecting the frame with the best quality from the dynamic image for image content recognition, or performing image content recognition based on multiple frames from the dynamic image; and executing the preset task based on the result of the image content recognition. The multiple frames can be some or all frames from the dynamic image. For example, the dynamic image can be captured by the camera of the current electronic device, such as a preview image during the photo-taking process, a photograph taken, or a previously captured photograph. Another example is that the dynamic image can also be captured by the camera of another electronic device and then transmitted to the current electronic device.

[0041] In this method, in the scenario of image content recognition based on dynamic images, the characteristic that dynamic images include multiple frames can be used to select or generate the frame with the best quality for image content recognition, thereby improving the processing effect of image content processing.

[0042] In one possible implementation, after acquiring the dynamic image for the image content recognition, the method further includes: displaying a first prompt message, the first prompt message indicating that the quality of the dynamic image is low, and indicating that the camera shooting method should be improved to enhance the effect of the image content recognition.

[0043] In the above embodiments, the method of improving the camera's shooting is indicated from at least one of the following aspects:

[0044] (1) Improve the stability of the camera during shooting;

[0045] (2) Point the camera at the content to be identified;

[0046] (3) Adjust the shooting angle;

[0047] (4) Avoid reflective areas;

[0048] (5) Adjust the size of the target content in the image; and

[0049] (6) Adjust the exposure level during shooting.

[0050] In the above embodiments, after displaying the first prompt information, the method further includes, but is not limited to, the following processing:

[0051] (i) In response to acquiring a second image of higher quality than the first image, perform the preset task based on the second image; or

[0052] (ii) In response to the failure to acquire other images of higher quality than the first image within a preset time, the preset task is executed based on the first image; or

[0053] (iii) In response to the user's confirmation instruction to perform the preset task using the first image, perform the preset task according to the first image.

[0054] In the above embodiments, the method further includes: acquiring a third image for the image content recognition, the third image being captured by a camera; displaying a second prompt message, the second prompt message indicating that the quality of the third image meets the preset task; and performing the preset task based on the third image.

[0055] In the above embodiments, the first image and / or the second image are preview images before the electronic device takes the picture, or images taken by the electronic device, or images stored on the electronic device.

[0056] In one possible implementation, the method further includes: sending request information to a server, the request information being generated based on the dynamic image; receiving image recognition enhancement information from the server, the image recognition enhancement information being generated based on the request information; and performing the preset task based on the dynamic image and the image recognition enhancement information.

[0057] The above implementation methods can be achieved in various ways.

[0058] One possible implementation of the method further includes: performing image content processing on the first image to obtain a first candidate result; sending the first image to a server; receiving an associated image from the server; performing image content processing on the associated image to obtain a second candidate result; and fusing the first candidate result and the second candidate result to obtain the processing result.

[0059] Another possible implementation of the method further includes: performing image content processing on the first image to obtain a first candidate result; sending the first image to a server; receiving a second candidate result from the server, the second candidate result being generated based on the first image; and fusing the first candidate result and the second candidate result to obtain the processing result.

[0060] Another possible implementation of the method further includes: sending the first image to a server; receiving a second image from the server, wherein the second image is retrieved based on the first image, or the second image is generated based on the first image; and fusing the first image and the second image to obtain the processing result.

[0061] The third aspect provides another image content processing method, in which the executing entity is an electronic device. In this method, an instruction to execute a preset task is received, the preset task including an image content recognition step, which may include image translation or image search; a first image for the image content recognition is acquired, the first image being captured by a camera; a request message is sent to a server, the request message being generated based on the first image; image recognition enhancement information is received from the server, the image recognition enhancement information being generated based on the request message; and the preset task is executed based on the first image and the image recognition enhancement information. For example, the first image may be captured by the camera of the current electronic device, such as a preview image during the photo-taking process, a photograph taken, or a photograph taken in the past. Another example is that the first image may also be captured by the camera of another electronic device and then transmitted to the current electronic device.

[0062] In this method, by leveraging the large data volume of the server, correction or completion can be performed based on the first image, thereby improving the accuracy of image content recognition and enhancing the effect of image content processing.

[0063] In one possible implementation, the method further includes: performing image content processing on the first image to obtain a first candidate result; sending the first image to a server; receiving an associated image from the server; performing image content processing on the associated image to obtain a second candidate result; and fusing the first candidate result and the second candidate result to obtain the processing result.

[0064] In another possible implementation, the method further includes: performing image content processing on the first image to obtain a first candidate result; sending the first image to a server; receiving a second candidate result from the server, the second candidate result being generated based on the first image; and fusing the first candidate result and the second candidate result to obtain the processing result.

[0065] In another possible implementation, the method further includes: sending the first image to a server; receiving a second image from the server, the second image being retrieved based on the first image, or the second image being generated based on the first image; and fusing the first image and the second image to obtain the processing result.

[0066] In one possible implementation, after acquiring the first image for image content recognition, the method further includes: displaying a first prompt message, the first prompt message indicating that the quality of the first image is low, and indicating a way to improve the camera shooting.

[0067] In the above embodiments, the method of improving the camera's shooting is indicated from at least one of the following aspects:

[0068] (1) Improve the stability of the camera during shooting;

[0069] (2) Point the camera at the content to be identified;

[0070] (3) Adjust the shooting angle;

[0071] (4) Avoid reflective areas;

[0072] (5) Adjust the size of the target content in the image; and

[0073] (6) Adjust the exposure level during shooting.

[0074] In the above embodiments, after displaying the first prompt information, the method further includes, but is not limited to, the following processing:

[0075] (i) In response to acquiring a second image of higher quality than the first image, perform the preset task based on the second image; or

[0076] (ii) In response to the failure to acquire other images of higher quality than the first image within a preset time, the preset task is executed based on the first image; or

[0077] (iii) In response to the user's confirmation instruction to perform the preset task using the first image, perform the preset task according to the first image.

[0078] In the above embodiments, the method further includes: acquiring a third image for the image content recognition, the third image being captured by a camera; displaying a second prompt message, the second prompt message indicating that the quality of the third image meets the preset task; and performing the preset task based on the third image.

[0079] In the above embodiments, the first image and / or the second image are preview images before the electronic device takes the picture, or images taken by the electronic device, or images stored on the electronic device.

[0080] In the above embodiments, the first image and / or the second image is a single-frame image or a multi-frame image. The multi-frame image can be a moving image or a video.

[0081] In one possible implementation, performing the preset task based on the first image and the image recognition enhancement information includes: selecting the frame with the best quality from the first image, or performing image content recognition based on multiple frames from the first image and the image recognition enhancement information; and performing the preset task based on the result of the image content recognition. The multiple frames can be some or all frames from a dynamic image.

[0082] Furthermore, the first to third aspects and their respective implementation methods described above can be combined with each other, and specific implementation methods can be referred to each other, which will not be repeated here.

[0083] A fourth aspect provides an electronic device comprising a plurality of functional modules; the plurality of functional modules interact to implement the methods executed by the electronic device in the first aspect and its embodiments, or to implement the methods executed by the electronic device in the second aspect and its embodiments, or to implement the methods executed by the electronic device in the third aspect and its embodiments. The plurality of functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the plurality of functional modules can be arbitrarily combined or divided based on specific implementations.

[0084] A fifth aspect provides an electronic device including at least one processor and at least one memory, wherein the at least one memory stores computer program instructions, and when the electronic device is in operation, the at least one processor executes the method executed by the electronic device in the first aspect and its embodiments, or executes the method executed by the electronic device in the second aspect and its embodiments, or executes the method executed by the electronic device in the third aspect and its embodiments.

[0085] The sixth aspect also provides a program product that, when run on an electronic device, causes the electronic device to perform the method performed by the electronic device in any of the above aspects and embodiments.

[0086] The seventh aspect also provides a readable storage medium storing a program that, when executed by an electronic device, causes the electronic device to perform the method of the electronic device in any of the above aspects and embodiments.

[0087] The eighth aspect also provides a chip for reading a program stored in a memory and executing the method performed by the electronic device in any of the above aspects and embodiments.

[0088] A ninth aspect also provides a chip system including a processor for supporting an electronic device in performing the methods of any of the above aspects and embodiments. In one possible embodiment, the chip system further includes a memory for storing the necessary programs and data. The chip system may be composed of chips or may include chips and other discrete devices.

[0089] It should be noted that the beneficial effects of various embodiments of the electronic device provided in the second to ninth aspects of this application can be referred to the beneficial effects of any possible embodiment of the first aspect, and will not be repeated here. Attached Figure Description

[0090] Figure 1A shows a schematic diagram of a photo translation scenario;

[0091] Figure 1B illustrates a flowchart of image translation based on dynamic photographs;

[0092] Figure 2 shows a schematic diagram of the hardware structure of a possible electronic device;

[0093] Figure 3 shows a software architecture block diagram of an electronic device;

[0094] Figure 4 is a schematic diagram of an interface of an image content processing method provided in an embodiment of this application;

[0095] Figure 5 is a flowchart illustrating one of the image content processing methods provided in this application embodiment;

[0096] Figure 6A is a schematic diagram of a process for obtaining candidate images according to an embodiment of this application;

[0097] Figure 6B is a schematic diagram of another process for obtaining candidate images provided in an embodiment of this application;

[0098] Figure 7A is a schematic diagram of a process for obtaining candidate images by user selection in an embodiment of this application;

[0099] Figure 7B is a schematic diagram of a scene for selecting an image according to an embodiment of this application;

[0100] Figure 8 is a schematic flowchart of obtaining a first processing result provided in an embodiment of this application;

[0101] Figure 9 is a schematic diagram of a dynamic image provided in an embodiment of this application;

[0102] Figure 10 is a second schematic flowchart of an image content processing method provided in an embodiment of this application;

[0103] Figure 11 is a schematic flowchart of obtaining the target processing result provided in an embodiment of this application;

[0104] Figure 12 is a third schematic flowchart of an image content processing method provided in an embodiment of this application;

[0105] Figure 13 is a fourth schematic flowchart of an image content processing method provided in an embodiment of this application;

[0106] Figure 14 is a fifth schematic flowchart of an image content processing method provided in an embodiment of this application;

[0107] Figure 15 is a schematic flowchart of an image content processing method provided in an embodiment of this application. Detailed Implementation

[0108] The embodiments of this application will now be described in detail with reference to the accompanying drawings and examples.

[0109] With the rapid development of society, electronic devices such as mobile phones are becoming increasingly common. These devices not only have communication functions but also powerful processing capabilities, storage capacity, camera functions, translation functions, and search functions. Through an operating system, electronic devices execute corresponding applications, allowing users to make calls, send text messages, browse the web, take photos, and store photos. Optionally, the camera function can be used for translation or search functions. Alternatively, historical photos stored on the device can also be used for translation or search functions.

[0110] For example, Figure 1A illustrates a photo translation scenario. As shown in interface 101 of Figure 1A, the camera application (APP) includes a "translation" function. By taking a photo of the content to be translated, translation between different languages ​​can be achieved, such as translating the content from English to Chinese. The "translation" function can include, but is not limited to, photo translation and augmented reality (AR) translation. As can be seen from Figure 1A, scenarios based on photo-based image translation are easily affected by image blur, reflections, etc., leading to difficulties in recognizing and translating text in the image.

[0111] For example, Figure 1B illustrates a flowchart of image translation based on moving photographs. This process includes the following steps:

[0112] Step 110: Obtain the animated photo input by the user.

[0113] For example, a photogrammetry record can typically capture a few seconds of moving footage, providing image information from more angles. For instance, a photogrammetry record can capture 2 seconds of moving footage.

[0114] Step 120: Extract the first frame of the animated photo.

[0115] Step 130: Perform image translation based on the first frame of the dynamic photo.

[0116] Step 140: Output the image translation results of the dynamic photo.

[0117] As can be seen from the process shown in Figure 1B, when performing image translation on moving photos, the static image of the first frame of the moving photo is usually extracted for image translation.

[0118] However, due to the nature of moving images, the first frame of a moving image may be captured before the user presses the shutter button, as the user may still be adjusting the camera angle. In this case, the first frame of a moving image often has many problems, such as light interference, hand shake, out-of-focus camera, or the camera not being pointed at the target text. Therefore, extracting the first frame of a moving image for image translation often results in a poor user experience.

[0119] Figures 1A and 1B use image translation scenarios as examples. Similar image content recognition scenarios also suffer from the above problems. For instance, image search based on taking a photo is easily affected by image blurring and reflections, making it difficult to recognize and translate text within the image. Furthermore, image search based on moving images typically involves extracting the first static frame from the moving image for the search.

[0120] In view of this, embodiments of this application provide an image content processing method. In this method, the electronic device can perform image quality recognition on a preview image during the photo-taking process, or on a photo obtained in real-time, or on a historical photo selected by the user. For example, when a preview image, a photo obtained in real-time, or a historical photo is detected as a low-quality image, a prompt message indicating low quality can be displayed. In this way, by guiding the user when obtaining candidate images for image content processing, higher-quality candidate images can be obtained, thereby improving the effect of image content recognition.

[0121] Alternatively, in scenarios involving image content processing of dynamic images, this method selects a high-quality frame from multiple frames included in the dynamic image as the target image, or generates the target image by image fusion based on multiple frames in the dynamic image; then, image content processing is performed based on the target image. In this way, by selecting or generating a higher-quality target image, the recognition and processing effects of image content processing can be improved.

[0122] Alternatively, this method can retrieve related images based on the target image; the processing result obtained from image content processing based on the related images is then fused with the processing result obtained from image content processing based on the target image to obtain the target processing result. In this way, information retrieval can be used to correct and complete the information in the target image, thereby improving the accuracy and completeness of image content recognition.

[0123] The technical solutions in this application can be applied to electronic devices, which can be any device capable of displaying an interface. For example, electronic devices can be mobile phones, foldable phones, tablets, wearable devices (e.g., watches, bracelets), in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), smart home devices (e.g., smart TVs), and other electronic devices. It is understood that this application does not impose any limitations on the specific type of electronic device.

[0124] The electronic devices to which this application's embodiments can be applied include, but are not limited to, those equipped with... Alternatively, it can be an electronic device running another operating system. For example, the electronic device described in the foregoing embodiments can be used.

[0125] Figure 2 illustrates a possible hardware structure diagram of an electronic device. The electronic device 200 includes components such as a radio frequency (RF) circuit 210, a power supply 220, a processor 230, a memory 240, an input unit 250, a display unit 260, an audio circuit 270, a communication interface 280, and a wireless fidelity (Wi-Fi) module 290. Those skilled in the art will understand that the hardware structure of the electronic device 200 shown in Figure 2 does not constitute a limitation on the electronic device 200. The electronic device 200 provided in this application embodiment may include more or fewer components than shown, may combine two or more components, or may have different component configurations. The various components shown in Figure 2 can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0126] The following is a detailed description of each component of the electronic device 200 with reference to Figure 2:

[0127] The RF circuit 210 can be used for receiving and transmitting data during communication or a call. Specifically, after receiving downlink data from the base station, the RF circuit 210 sends it to the processor 230 for processing; additionally, it sends uplink data to be transmitted to the base station. Typically, the RF circuit 210 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc.

[0128] Furthermore, the RF circuit 210 can also communicate with other devices via a wireless communication network. The wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Message Service (SMS).

[0129] Wi-Fi technology is a short-range wireless transmission technology. The electronic device 200 can connect to an access point (AP) via the Wi-Fi module 290, thereby enabling access to the data network. The Wi-Fi module 290 can be used for receiving and sending data during communication.

[0130] The electronic device 200 can physically connect to other devices through the communication interface 280. Optionally, the communication interface 280 can be connected to the communication interfaces of other devices via a cable to enable data transmission between the electronic device 200 and other devices.

[0131] The electronic device 200 can also perform communication services and interact with other electronic devices. Therefore, the electronic device 200 needs to have data transmission capabilities, meaning it needs to include a communication module. Although Figure 2 shows the RF circuit 210, the Wi-Fi module 290, and the communication interface 280, it is understood that the electronic device 200 contains at least one of the aforementioned components or other communication modules (such as a Bluetooth module) for data transmission.

[0132] For example, when the electronic device 200 is a mobile phone, the electronic device 200 may include the RF circuit 210, the Wi-Fi module 290, or a Bluetooth module (not shown in Figure 2); when the electronic device 200 is a tablet computer, the electronic device 200 may include the Wi-Fi module 290 or a Bluetooth module (not shown in Figure 2); when the electronic device 200 is a smart home device, the electronic device 200 may include the Wi-Fi module 290 or a Bluetooth module (not shown in Figure 2).

[0133] The memory 240 can be used to store software programs and modules. The processor 230 executes various functional applications and data processing of the electronic device 200 by running the software programs and modules stored in the memory 240. Optionally, the memory 240 may mainly include a program storage area and a data storage area. The program storage area may store the operating system (mainly including the software programs or modules corresponding to the kernel layer, system layer, application framework layer, and application layer).

[0134] In addition, the memory 240 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0135] The input unit 250 can be used to receive editing operations on various types of data objects, such as numbers or characters, input by the user, and to generate key signal inputs related to user settings and function control of the electronic device 200. Optionally, the input unit 250 may include a touch panel 251 and other input devices 252.

[0136] The touch panel 251, also known as a touchscreen, collects user touch operations (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on the touch panel 251) and drives corresponding connection devices according to a pre-set program. In this embodiment, the touch panel 251 can collect user operations. For example, the user operation could be selecting a photo or taking a photo, etc.

[0137] Optionally, the other input device 252 may include, but is not limited to, one or more of the following: a physical keyboard, an infrared sensor, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, a joystick, etc. For example, an infrared sensor can be used to acquire the user's air gesture operations.

[0138] The display unit 260 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device 200. The display unit 260 is the display system of the electronic device 200, used to present the interface and realize human-computer interaction. The display unit 260 may include a display panel 261. Optionally, the display panel 261 can be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar form. In this embodiment, the display unit 260 can be used to display a user interface. For example, the user interface can display a relevant interface guiding the user to take pictures in an image content processing scenario. Another example is that the user interface can also display an interface showing the processing results.

[0139] The processor 230 is the control center of the electronic device 200. It connects various components via various interfaces and lines, and executes software programs and / or modules stored in the memory 240, as well as calling data stored in the memory 240, to perform various functions and process data of the electronic device 200, thereby enabling various services based on the electronic device 200. In this embodiment, the processor 230 can be used to implement an image content processing method provided in this embodiment.

[0140] The electronic device 200 also includes a power supply 220 (such as a battery) for supplying power to various components. Optionally, the power supply 220 can be logically connected to the processor 230 through a power management system, thereby enabling the power management system to manage functions such as charging, discharging, and power consumption.

[0141] As shown in Figure 2, the electronic device 200 also includes an audio circuit 270, a microphone 271, and a speaker 272, providing an audio interface between the user and the electronic device 200. The audio circuit 270 converts audio data into signals recognizable by the speaker 272 and transmits the signals to the speaker 272, where the speaker 272 converts them into sound signals for output. The microphone 271 collects external sound signals (such as human speech or other sounds) and converts the collected external sound signals into signals recognizable by the audio circuit 270, sending them to the audio circuit 270. The audio circuit 270 can also convert the signals transmitted by the microphone 271 into audio data, and then output the audio data to the RF circuit 210 for transmission to, for example, another electronic device, or output the audio data to the memory 240 for further processing.

[0142] As shown in Figure 2, the electronic device 200 also includes a camera 2100. The camera 2100 can be used to receive calls from the processor 230 and acquire image data. For example, when the electronic device 200 detects that the camera 2100 is turned on, it can display a preview interface in real time based on the image data acquired by the camera 2100.

[0143] Although not shown in Figure 2, the electronic device 200 may also include at least one sensor, which will not be described in detail here. The at least one sensor may include, but is not limited to, a pressure sensor, a barometric pressure sensor, an accelerometer, a distance sensor, a fingerprint sensor, a touch sensor, a temperature sensor, etc.

[0144] The operating system (OS) involved in this application embodiment is the most basic system software running on the electronic device 200. The software system of the electronic device 200 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment takes an operating system adopting a layered architecture as an example to illustrate the software architecture of the electronic device 200.

[0145] Figure 3 is a software architecture block diagram of an electronic device provided in an embodiment of this application. As shown in Figure 3, the software architecture of the electronic device can be a layered architecture. For example, the software can be divided into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into five layers, from top to bottom: the application layer, the application framework layer (framework, FWK), the runtime and system library, the kernel layer, and the hardware layer.

[0146] The application layer can include a series of application packages. As shown in Figure 3, the application layer can include a user interface (UI), camera, photo album, settings, skin modules, third-party applications, etc. Third-party applications can include, for example, wireless local area network (WLAN), music, call, Bluetooth, video, etc.

[0147] In one possible implementation, the application can be developed using Java, by calling the application programming interface (API) provided by the application framework layer. Developers can then interact with the underlying operating system layers (such as the hardware layer and kernel layer) to develop their own applications. This application framework layer primarily consists of a series of services and management systems within the operating system.

[0148] The application framework layer provides application programming interfaces and a programming framework for applications within the application layer. The application framework layer includes some predefined functions. As shown in Figure 3, the application framework layer may include a view system, activity manager, window manager, content provider, phone manager, resource manager, notification manager, etc.

[0149] The Activity Manager manages the lifecycle of each application and provides commonly used navigation and back functions, offering an interactive interface for all program windows.

[0150] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0151] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0152] A view system includes both visual and non-visual controls, such as controls that display text and controls that display images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text message notification icon could include views that display text and views that display images.

[0153] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0154] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0155] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0156] The runtime includes the core libraries and the virtual machine. The runtime is responsible for the scheduling and management of the operating system.

[0157] The core library consists of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core libraries of the operating system. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0158] A system library can include multiple functional modules. For example: a surface manager, a media framework, a 3D graphics processing library (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), etc.

[0159] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0160] The media framework supports playback and recording of various commonly used audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, and AMR.

[0161] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0162] A 2D graphics engine is a drawing engine for 2D graphics. A 2D graphics engine can perform drawing operations, creating image translation interfaces, image search interfaces, and more on the screen.

[0163] In some embodiments, a 3D graphics processing library can be used to draw 3D motion trajectory images, and a 2D graphics engine can be used to draw 2D motion trajectory images.

[0164] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0165] The hardware layer can include various types of sensors, such as accelerometers, gravity sensors, and touch sensors.

[0166] Typically, an electronic device 200 can run multiple applications simultaneously. In a simpler scenario, one application corresponds to one process; in a more complex scenario, one application can correspond to multiple processes. Each process has a unique process ID.

[0167] It should be understood that in the embodiments of this application, "at least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple. "Multiple" refers to two or more. "And / or" is used to describe the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0168] Furthermore, it should be understood that in the description of this application, terms such as "first" and "second" are used only to distinguish the purpose of description and should not be construed as indicating or implying relative importance, nor should they be construed as indicating or implying order. For example, "first prompt information" and "fourth prompt information" in the following embodiments are only used to distinguish different prompt scenarios and are not used to limit specific content, etc.

[0169] It should be understood that the hardware structure of the electronic device can be as shown in Figure 2, and the software system architecture can be as shown in Figure 3. The software programs and / or modules corresponding to the software system architecture in the electronic device can be stored in the memory 240. The processor 230 can run the software programs and applications stored in the memory 240 to execute the flow of an image content processing method provided in the embodiments of this application.

[0170] The method provided in this application can be applied to image content processing scenarios. The image can be acquired through methods including, but not limited to, the following: a preview image in a photography scenario, a photo obtained through real-time photography, or a historical photo selected by the user. Furthermore, the image can be a single-frame static image or a dynamic image comprising multiple frames. For example, image content processing scenarios can include, but are not limited to, one of the following: image translation, image search, image text recognition, and image recognition. It is understood that the method provided in this application can be used for various purposes, such as translating documents at work, searching for answers to questions during study, retrieving tags in shopping scenarios, and searching for road signs in travel environments.

[0171] For ease of understanding, the image content processing method provided in the embodiments of this application will be described in several parts below.

[0172] (I) The process of taking photos

[0173] Referring to Figure 1A, the electronic device can detect when a user opens the camera app. In response to this operation, it can access the camera to capture image data and display a preview interface. For example, the electronic device can analyze the image quality of the preview image captured in real-time during the photo-taking process. This image quality analysis can include, but is not limited to, at least one of the following methods:

[0174] Method A: Is the camera stable?

[0175] For example, camera stability can be determined by judging whether the preview image is blurry (or clear). Optionally, if the preview image is not blurry (or clear), the camera is determined to be stable. Alternatively, if the preview image is blurry (or unclear), the camera is determined to be unstable.

[0176] The determination of whether the preview image is blurry (or clear) can be achieved, for example, through model training. This application does not limit the specific determination method.

[0177] Method B: Whether the camera is pointed at the text content and / or image content.

[0178] For example, feature analysis can be performed on the text and / or images in the preview image. When incomplete text display and / or incomplete image display are detected, it can be determined that the camera is not pointing at the text content and / or image content.

[0179] The determination of whether the camera is pointed at the text content and / or image content can be achieved, for example, through model training. This application does not limit the specific determination method.

[0180] Method C: Whether the text area is reflective.

[0181] For example, the text area in the preview image can first be identified or its features extracted, and then it can be determined whether the text area is reflective.

[0182] The determination of whether the text area is reflective can be achieved, for example, through model training. This application does not limit the specific determination method.

[0183] Option D: Is the text size too small?

[0184] For example, the text size in the preview image can first be identified or its features extracted, and then it can be determined whether the text size is less than a preset threshold. Optionally, if the text size is determined to be less than the preset threshold, the text size can be determined to be too small. Alternatively, if the text size is determined to be greater than or equal to the preset threshold, the text size can be determined to be normal.

[0185] One method to obtain text size is, but not limited to, identifying the top and bottom edges of the text and determining the number of pixels between them. For example, with a preset threshold of 10 pixels, if the area between the top and bottom edges of the text is 6 pixels, which is less than 10 pixels, the text size is determined to be too small.

[0186] In addition, based on the analysis results of image quality, when a preview image is detected as low-quality, shooting guidance information can be displayed. For example, a low-quality image can be determined by at least one of the following reasons, but not limited to: unstable camera, camera not pointed at text content and / or image content, glare in text areas, text size being too small, overexposure, etc.

[0187] For example, the shooting guidance information may include at least one of the following: the preview image is a low-quality image, and the reason for the low quality. For example, Figure 4 is a schematic diagram of an interface of an image content processing method provided in an embodiment of this application. As shown in interface 40A in Figure 4, taking the detection of low quality as a reason for the text size being too small as an example, a pop-up window 401 can be displayed in interface 40A. As shown in pop-up window 401, the content of the shooting guidance information can be "Please note!!! The current reason for the low-quality image is: the text is too small, please enlarge the image." It should be noted that in the implementation of this application, the display method of the shooting guidance information is not limited. For example, it can be displayed in the form of a pop-up window as shown in Figure 4, or in a semi-modal form, or through voice, vibration, etc.

[0188] During the photo-taking process, the electronic device analyzes the image quality of the preview image captured in real time by the camera. This allows for real-time shooting guidance for the user, reminding them to adjust the device's posture or shooting distance, thereby improving the quality of the captured image and ultimately enhancing the image processing results. Furthermore, this method can cultivate users' photography habits in image processing scenarios, improving the user experience.

[0189] (II) Photo taken

[0190] Optionally, when the electronic device detects the user's photo-taking action, it can determine that the photo-taking is complete. The photo obtained at this time is the photo obtained in response to the photo-taking action. For example, the photo-taking action can be a click on the photo-taking control or a press on a preset physical control, such as pressing a volume button. This application does not limit the photo-taking action.

[0191] Alternatively, the electronic device can detect that the photo-taking conditions are met and determine that the photo-taking is complete. In this case, the photo obtained by the electronic device can be generated based on the preview images, for example, it can be selected from multiple preview images, or it can be obtained by fusing multiple preview images. For example, the photo-taking conditions could be capturing a high-quality image, scanning target information, reaching a specified time, etc.

[0192] In some possible scenarios, after taking a photo, the electronic device can analyze the image quality of the captured image. If it detects that the captured image is of low quality, it can display image improvement information. For example, a low-quality image can be determined by at least one of the following reasons, but not limited to: unstable camera, camera not pointed at text and / or image content, glare in text areas, text size being too small, overexposure, etc. For example, image improvement information may include at least one of the following: the photo is a low-quality image, and the reason for the low quality. The methods for determining a low-quality image are described above and will not be repeated here.

[0193] Optionally, if a user-instructed update to the captured image is detected, the shooting process can be resumed for a retake. The specific implementation process can be found in the preceding description. Alternatively, if no user-instructed update to the captured image is detected, image content processing (described later) can be performed based on the captured photo. It can be understood that post-capture improvement information, compared to shooting guidance information during the capture process, helps improve the image quality of the user's next shot or guides the user to retake the photo; while shooting guidance information can prompt the user to adjust the angle, method, etc., during the current shooting process to obtain a high-quality photo. Thus, by analyzing the image quality of the captured photos, corresponding improvement suggestions can be provided to the user's shooting habits, thereby improving the image quality and ultimately enhancing the image content processing effect. Furthermore, this method can cultivate good shooting habits in image content processing scenarios, improving the user experience.

[0194] Based on the information provided above, photos can be obtained by taking a picture. Photos can be in two forms: still images consisting of a single frame, or moving images consisting of multiple frames.

[0195] In one possible example, the photograph obtained is a still image. In this case, the image content is processed directly on the still image, which will not be elaborated in this application.

[0196] In another possible example, when the electronic device determines that the captured photo is a moving image, it can obtain the target image based on the M frames included in the moving image; then, image content processing is performed on the target image. Here, M is a positive integer greater than 1. Optionally, the electronic device extracts high-quality frames from the M frames and uses these high-quality frames as the target image. Alternatively, the electronic device performs image fusion based on N frames from the M frames to generate the target image. Here, N frames can be images from the M frames that meet preset quality conditions. N can be less than M; or N can be equal to M, in which case it can also be understood as performing image fusion based on each frame of the moving image to generate the target image.

[0197] In some possible scenarios, electronic devices can use a control over dynamic images to determine whether a captured photo is a dynamic image. Optionally, when the control is on, it indicates that the function of capturing dynamic images is enabled, meaning that the captured photo will be a multi-frame dynamic image. Alternatively, when the control is off, it indicates that the function of capturing dynamic images is not enabled, meaning that the captured photo will be a single-frame static image.

[0198] (III) Image Content Processing

[0199] In one possible scenario, the target image for image content processing can be the target image obtained based on the shooting scenario described above. The specific implementation process of the shooting scenario can be found in the descriptions of the shooting process and completion section above, and will not be repeated here.

[0200] In another possible scenario, the target image for image content processing by the electronic device can also be a photo selected by the user. It can be understood that a photo selected by the user can also be a historical photo. It should be noted that historical photos can be photos taken by the current electronic device or photos received by the current electronic device from other electronic devices; this application does not limit the source of the electronic device for historical photos.

[0201] Some possible implementations include, during image content processing, when an electronic device detects that the captured content in the target image is blurred (or incomplete), it can trigger a retrieval on a server to obtain image recognition enhancement information, thereby completing and / or correcting the image. The server can be deployed on a cloud server or cloud platform; this application does not limit its deployment.

[0202] The electronic device determines whether the captured content is blurry (or incomplete), for example, through model training. This application does not limit the specific determination method.

[0203] Correspondingly, the server receives the target image from the electronic device; based on the target image, the server performs information retrieval to obtain associated images. Optionally, the server performs fuzzy matching based on the text in the target image. Alternatively, the server performs fuzzy matching based on the images within the target image.

[0204] In some possible scenarios, the server may include a database for image content processing, such as collecting, but not limited to, images containing product information. It should be noted that the collected images in the database can be filtered, for example, prioritizing high-quality images or images from official sources. In some possible implementations, different image content processing scenarios may correspond to the same database or different databases. For example, image translation might correspond to database 1, and image search to database 2. Or, for example, image translation of product information might correspond to database 1-1, and image translation of travel information to database 1-2, etc.

[0205] In one possible implementation, the electronic device receives an associated image from a server; and processes the image content of the target image based on the associated image. For example, the electronic device performs image content processing on the target image to obtain a first processing result; then performs image content processing on the associated image to obtain a second processing result; finally, the first and second processing results are fused to obtain the target processing result. In another example, the electronic device performs image fusion on the target image based on the associated image to obtain a fused target image; then performs image content processing on the fused target image to obtain the target processing result.

[0206] In another possible implementation, the electronic device receives a second processing result from the server; based on the second processing result, it fuses the first processing result obtained from the target image to obtain the target processing result. It can be understood that after the server obtains the associated image based on the target image, it can perform image content processing on the server to obtain the second processing result. This implementation can reduce the computational burden on the electronic device.

[0207] In the process of image content processing, by combining information search with the server, the processing results of the image content can be corrected and / or supplemented, thereby improving the quality and effect of image content processing and thus enhancing the user experience.

[0208] It should be noted that the server in the above embodiments can be a cloud server or a cloud server cluster.

[0209] (iv) Image content processing completed

[0210] In some possible scenarios, after obtaining the image content processing results, the electronic device can also analyze the image quality of the target image. If the target image is determined to be a low-quality image, it can display image improvement information. For example, a low-quality image can be determined by at least one of the following reasons, but not limited to: unstable camera, camera not pointed at text content and / or image content, glare in text areas, text size being too small, overexposure, etc. For example, image improvement information may include at least one of the following: the target image is a low-quality image, and the reason for the low quality. The method for determining a low-quality image can be found in the previous description and will not be repeated here.

[0211] By analyzing the image quality of the target image, corresponding improvement suggestions can be provided based on the user's shooting habits, thereby improving the image quality and ultimately enhancing the processing effect of image content. Furthermore, this method can cultivate good shooting habits in image content processing scenarios, improving the user experience.

[0212] Based on the foregoing descriptions, the following describes the implementation process of the image content processing method provided in this application, in order to illustrate how the method provided in this application can achieve the foregoing descriptions and thus improve the processing effect of image content processing.

[0213] Referring to Figure 5, a flowchart illustrating an image content processing method provided in an embodiment of this application is shown. This process can be applied to electronic devices; the following flowchart uses a mobile phone as an example. The process may include steps 51 and 52.

[0214] Step 51: Obtain candidate images.

[0215] In this embodiment, candidate images can be acquired in various ways. These methods may include, but are not limited to, real-time image capture and user-selected image capture. The real-time image capture method is described below with reference to Figures 6A and 6B, and the user-selected image capture method is described below with reference to Figures 7A and 7B.

[0216] In one possible example, Figure 6A is a schematic diagram of a process for acquiring candidate images using a real-time photography method according to an embodiment of this application. This process can be applied to a mobile phone and includes the following steps:

[0217] S510: Start shooting.

[0218] For example, in response to a user opening the camera app, the phone can access the camera to capture an image. It can be understood that before the phone detects the photo-taking action, the image displayed on the phone is based on data captured by the camera in real time; this can also be understood as a real-time image preview.

[0219] Another example is that any app, such as a search app, may include image translation controls or image search controls. In response to a click on these controls, the phone can also use its camera to capture an image. It can be understood that before the phone detects a photo-taking action, the image displayed on the phone is based on data captured in real-time by the camera; this can also be understood as a real-time image preview.

[0220] S511: Detect whether the preview image is a low-quality image. If yes, continue with S512A; otherwise, continue with S512B, or continue with S513B1 or S513B2.

[0221] For example, the mobile phone can analyze the image quality of the preview image. This image quality analysis may include, but is not limited to, the following detection methods: whether the phone's camera is stable, whether the camera is pointed at the text content and / or image content, whether the text area is reflective, and whether the text size is too small. Optionally, if the phone detects at least one of the following: camera instability, camera not pointed at the text content and / or image content, text area reflection, text size too small, or overexposure, it can determine that the preview image is a low-quality image. The specific implementation process can be found in the preceding description of the photo-taking process, and will not be repeated here.

[0222] S512A: Displays the first prompt message indicating the cause of the low quality.

[0223] For example, a low-quality image may be due to at least one of the following reasons: camera instability, glare in text areas, text size being too small, or overexposure. The first prompt message may be the shooting guidance information mentioned above.

[0224] Based on the initial prompt message, the phone can perform different processing procedures depending on the scenario. The following describes the processing procedures for several possible scenarios:

[0225] Scene A1: A photo-taking action is detected.

[0226] For example, the photo-taking operation can be a click operation on the photo-taking control, a press operation on a preset physical control, such as a press operation on the volume button, or other methods. This application does not limit the photo-taking operation.

[0227] S513A1: Use an image captured by the user as a candidate image. For example, the image captured by the user can be a still image or a moving image.

[0228] Scene A2: No photo-taking operation detected.

[0229] S513A2: Generate candidate images based on the preview image.

[0230] For example, in a scenario where the phone initiates shooting, the preview image can be refreshed at a fixed frame rate, thus obtaining multiple preview images. Optionally, the phone can select a high-quality image as a candidate image from the multiple preview images. Alternatively, the phone can perform image fusion based on the multiple preview images to generate a candidate image. The generated candidate image can be a static image or a dynamic image.

[0231] Scene A3: Preview image refreshed.

[0232] For example, when the preview image reaches the refresh time, a new preview image can be obtained, and at this time, execution of S511 can be returned.

[0233] The mobile phone determines the implementation process of scenario A1, scenario A2, or scenario A3 based on the actual scenario.

[0234] S512B: Displays prompts indicating high quality.

[0235] This is understandable; the prompts indicating high quality guide users to promptly acquire candidate images. The acquired candidate images are of high quality, thus ensuring the effectiveness of image content processing.

[0236] In addition to displaying prompts indicating high quality, the phone can also perform different processing procedures depending on the scenario. The following describes the processing procedures for several possible scenarios:

[0237] Scenario B1: A photo-taking operation is detected.

[0238] Similar to scenario A1, the photo-taking operation can be a click operation on the photo-taking control, a press operation on a preset physical control, such as a press operation on the volume button, or other methods. This application does not limit the photo-taking operation.

[0239] S513B1: Use an image captured by the user as a candidate image. For example, the image captured by the user can be a still image or a moving image.

[0240] Scene B2: No photo-taking operation detected.

[0241] S513B2: Generate candidate images based on the preview image.

[0242] Similar to scene A2, in the scenario where the phone initiates shooting, the preview image can be refreshed at a fixed frame rate, thus acquiring multiple preview images. Optionally, the phone can select a high-quality image as a candidate image from the multiple preview images. Alternatively, the phone can perform image fusion based on the multiple preview images to generate a candidate image. The generated candidate image can be a static image or a dynamic image.

[0243] Scene B3: Preview image refresh.

[0244] For example, when the preview image reaches the refresh time, a new preview image can be obtained, and at this time, execution of S511 can be returned.

[0245] The mobile phone determines the implementation process of scenario B1, scenario B2, or scenario B3 based on the actual scenario.

[0246] As illustrated in Figure 6A above, during the process of acquiring candidate images by taking photos, the user's photography method can be guided, thereby improving the image quality of the photos taken by the user and ensuring the processing effect of image content.

[0247] In another possible example, Figure 6B is a schematic diagram of another process for acquiring candidate images using a real-time photography method according to an embodiment of this application. This process can be applied to a mobile phone and includes the following steps:

[0248] S510: Start shooting.

[0249] For example, in response to a user opening the camera app, the phone can access the camera to capture an image. It can be understood that before the phone detects the photo-taking action, the image displayed on the phone is based on data captured by the camera in real time; this can also be understood as a real-time image preview.

[0250] Another example is that any app, such as a search app, may include image translation controls or image search controls. In response to a click on these controls, the phone can also use its camera to capture an image. It can be understood that before the phone detects a photo-taking action, the image displayed on the phone is based on data captured in real-time by the camera; this can also be understood as a real-time image preview.

[0251] S514: Acquire the captured image. Optionally, the captured image can be acquired through a photo-taking operation. Alternatively, the captured image can also be automatically acquired by the phone; in this case, the phone can select a high-quality frame as the captured image based on multiple preview images, or it can perform image fusion based on multiple preview images to generate the captured image.

[0252] S515: Detect whether the captured image is a low-quality image. If yes, continue to S516A; otherwise, continue to S516B, or continue to S518.

[0253] For example, the specific implementation process of S515 can be found in the specific implementation process of S511, and will not be repeated here.

[0254] S516A: Displays a second prompt message to indicate the reason for the low quality.

[0255] For example, a low-quality image may be due to at least one of the following reasons: unstable camera, camera not being pointed at the text content and / or image content, glare in the text area, text size being too small, or overexposure; the second prompt message may be the photo improvement information provided after the photo is taken. For details on the implementation process, please refer to the introduction to the photo taking process provided earlier.

[0256] S516B: Displays prompts indicating high quality.

[0257] This is understandable; the prompts indicating high quality guide users to promptly acquire candidate images. The acquired candidate images are of high quality, thus ensuring the effectiveness of image content processing.

[0258] S517: Should the captured image be updated? If yes, return to S514; otherwise, proceed to S518.

[0259] For example, a feedback entry point can be provided to the user to confirm whether the captured image should be updated; based on the user's feedback, it is determined whether to update the captured image. The feedback entry point can be, for example, a pop-up window or a voice command; this application does not limit the form of the feedback entry point.

[0260] S518: Generate candidate images based on the captured images.

[0261] As illustrated in Figure 6B above, by analyzing the image quality of the captured photos and displaying improvement information when low-quality images are detected, users can be guided to retake the photos to obtain higher-quality images. Furthermore, this method can cultivate good photography habits in users for future photos.

[0262] In another possible example, Figures 6A and 6B can be combined. For instance, S510 and S514 in Figure 6B can be implemented using S510 to S513B2 in Figure 6A. For example, the captured image can be obtained using S513A1, S513A2, S513B1, or S513B2 in Figure 6A. This allows for not only guidance during the shooting process but also improvement after shooting, thereby increasing the probability of obtaining higher-quality candidate images and ensuring efficient image content processing.

[0263] In one possible example, Figure 7A is a schematic diagram of a process for obtaining candidate images using a user-selected image method according to an embodiment of this application. This process may include the following steps:

[0264] S519: The image selected by the user has been detected.

[0265] For example, a photo album app may provide image translation controls or image search controls. For instance, Figure 7B is a schematic diagram of a scene for selecting images provided in an embodiment of this application. As shown in interface 70A of Figure 7B, this can be the interface of the photo album app.

[0266] In interface 70A, the phone responds to the selection of a photo and displays interface 70B. Interface 70B is a large-scale view of the photo selected in interface 70A, and may include some functional controls, such as share controls, favorite controls, edit controls, delete controls, and more controls.

[0267] In interface 70B, the phone can display interface 70C in response to a click on more controls.

[0268] In interface 70C, a pop-up window 701 may be displayed. Pop-up window 701 may include an image translation control and an image search control. For example, in response to a click on the image translation control included in pop-up window 701, the mobile phone may determine that the user has selected a photo displayed in interface 70C.

[0269] Another example is that any app, such as a search app, can also provide image translation controls or image search controls. For instance, in response to a click on the image translation control or image search control, the phone can access the phone's photo album to allow the user to select an image; the specific implementation process will not be elaborated further.

[0270] It should be noted that the specific implementation method of user image selection in this application embodiment is not limited.

[0271] S5110: Detect whether the image selected by the user is a low-quality image. If so, continue with S5111A; otherwise, continue with S5111B or S5113.

[0272] For example, the specific implementation process of S5110 can also be found in the specific implementation process of S511, and will not be repeated here.

[0273] S5111A: Displays a third prompt message indicating the cause of the low quality.

[0274] For example, a low-quality image may include, but is not limited to, at least one of the following reasons for low quality: unstable camera, camera not being pointed at text content and / or image content, glare in text area, text size being too small, or overexposure; the third prompt information may be information to improve the photo.

[0275] S5111B: Displays prompts indicating high quality.

[0276] This is understandable; the prompts indicating high quality guide users to promptly acquire candidate images. The acquired candidate images are of high quality, thus ensuring the effectiveness of image content processing.

[0277] S5112: Should the image selected by the user be replaced? If yes, return to execute S519; otherwise, execute S5113.

[0278] For example, a feedback entry point can be provided to the user to confirm whether the image selected by the user should be replaced; based on the user's feedback, it is determined whether to replace the image selected by the user. The feedback entry point can be, for example, a pop-up window, a voice command, etc., and this application does not limit the form of the feedback entry point.

[0279] For example, if it is determined that the image selected by the user will be replaced, the system can return to the interface that provides the user with a photo selection option, or it can provide the user with a real-time photo capture option, thereby enabling the user to switch to the real-time photo capture method to obtain candidate images.

[0280] S5113: Generate candidate images based on the image selected by the user. For example, the candidate image may be a static image. Alternatively, the candidate image may be a dynamic image comprising multiple frames.

[0281] Based on the description in Figure 7A above, when a user selects historical photos for image content processing, the image quality of the selected historical photos can be analyzed. If a low-quality image is detected, it is easy to guide the user to select other photos or retake the content to be processed, thereby ensuring the processing effect of image content processing.

[0282] Step 52: Obtain the first processing result based on the candidate image.

[0283] Figure 8 is a schematic flowchart of obtaining a first processing result according to an embodiment of this application. The process may include the following steps:

[0284] S521: Is the candidate image a moving image? If yes, proceed to step 522. Otherwise, proceed to step 523.

[0285] S522: Generate a target image based on multiple frames in a dynamic image.

[0286] For example, the target image can be a high-quality frame from multiple frames in a dynamic image, or it can be a fused image obtained by fusing multiple frames in a dynamic image.

[0287] For example, Figure 9 is a schematic diagram of a dynamic image provided in an embodiment of this application. Taking a dynamic image 90 comprising three frames as an example, it may include frame 1 90a captured at time point 1, frame 2 90a captured at time point 1, and frame 3 90a captured at time point 1. For example, frame 1 90a is the first frame in the dynamic image 90, but it may be blurry due to issues such as jitter, blurring, or lack of reflection. Therefore, it can be understood that the first frame is not suitable as a target image for image content processing. As another example, frame 2 90b is the second frame in the dynamic image 90. The image quality is clear, but it may be incomplete due to the camera not being properly aligned. Frame 3 90c has the same problem as frame 2 90b. In this scenario, image fusion can be performed on frames 2 90b and frame 3 90c to obtain a target image with more information and better quality.

[0288] It is understandable that the above method can improve the effect of image content processing by selecting high-quality frames from dynamic images or fusing multiple frames to obtain the target image, and reduce abnormal problems in image content processing caused by blurring or reflection of the first frame.

[0289] Step 522 can be understood as the specific implementation process after the photo is taken, which can be referred to in the previous text and will not be repeated here.

[0290] S523: Select the candidate image as the target image.

[0291] It is understandable that when the candidate image is a static image, it usually includes a frame of image, so the frame of image can be directly used as the target image.

[0292] S524: Based on the target image, perform image content processing to obtain the first processing result.

[0293] The method provided in this application, in scenarios involving image content processing based on dynamic images, allows for the extraction of high-quality frames or the fusion of multiple frames within the dynamic image to obtain a target image of higher quality. This approach, processing image content based on the target image, can improve the recognition and processing effectiveness of the image content, such as enhancing translation completeness and search accuracy.

[0294] Referring to Figure 10, another schematic flowchart of an image content processing method provided in an embodiment of this application is shown. This process can be applied to electronic devices; the following flowchart uses a mobile phone as an example. This process may include not only steps 51 and 52 shown in Figure 5, but also the following step 1001.

[0295] Step 51: Obtain candidate images.

[0296] Step 52: Obtain the first processing result based on the candidate image.

[0297] For a description of steps 51 and 52, please refer to the previous text; they will not be repeated here.

[0298] Step 1001: Optimize the first processing result to obtain the target processing result.

[0299] For example, Figure 11 is a schematic flowchart of obtaining the target processing result provided by an embodiment of this application. This process may include the following steps:

[0300] S1001A: Target image is detected as blurry or incomplete. For example, after determining the target image, the image quality of the target image can be analyzed. If the target image is detected as blurry or incomplete, the process shown in Figure 11 can be continued to obtain the target processing result. Alternatively, if the target image is not detected as blurry or incomplete, there is no need to continue with the process shown in Figure 11 to optimize the first processing result.

[0301] S1001B: Obtain the associated image based on the target image.

[0302] For example, a mobile phone can send a target image to a server; the server performs information retrieval based on the target image to obtain a related image; and then the server sends the related image to the mobile phone.

[0303] S1001C: Based on the associated image, perform image content processing to obtain the second processing result. For details on the implementation process, please refer to the previous description of step 52; it will not be repeated here.

[0304] In another example, the electronic device can also directly obtain the second processing result from the server based on the target image. It can be understood that the server obtains a related image based on the target image; then, the server performs image content processing on the related image to obtain the second processing result. This reduces the computational burden on the electronic device.

[0305] S1001D: The first processing result and the second processing result are fused to obtain the target processing result.

[0306] By referring to the content in Figure 11 and combining it with the information retrieval from the server, the processing results can be corrected and completed, thereby improving the processing effect of image content and enhancing the user experience of using image translation or image search functions.

[0307] In another possible implementation, the electronic device may first acquire a related image after determining the target image; then, the electronic device may perform image fusion based on the target image and the related image to obtain the fused target image; finally, image content processing may be performed on the fused target image to obtain the target processing result. The specific implementation process of image processing on the fused target image can be found in step 52 above, and will not be repeated here.

[0308] Additionally, Figure 12 is another schematic flowchart of an image content processing method provided in an embodiment of this application. This process can be executed after obtaining the first processing result based on the process shown in Figure 5, or after obtaining the target processing result based on the process shown in Figure 10. The process may include the following steps:

[0309] Step 1201: The target image is detected as a low-quality image.

[0310] Step 1202: Display a fourth prompt message indicating the reason for the low quality.

[0311] For example, low-quality images may be due to at least one of the following reasons: unstable camera, camera not pointed at text and / or image content, glare in text areas, text size being too small, or overexposure. The fourth prompt can be the photo improvement information described in the image content processing completion section above.

[0312] Steps 1201 to 1202 can be understood as the specific implementation process of image content processing, which can be referred to in the previous introduction and will not be repeated here.

[0313] Based on the information presented in Figure 12, after image content processing is completed, analyzing the quality of the target image can help users learn how to obtain a better target image in the next image content processing, thereby improving the image content processing effect.

[0314] Based on the same inventive concept, Figure 13 is another schematic flowchart of an image content processing method provided in an embodiment of this application. This process can be applied to electronic devices. The process may include the following steps:

[0315] Step 1301: Receive an instruction to execute a preset task, wherein the preset task includes a step of image content recognition, and the preset task includes image translation or image search.

[0316] Step 1302: Obtain a first image for the image content recognition, wherein the first image is captured by a camera.

[0317] Step 1303: Display a first prompt message, which indicates that the quality of the first image is low and indicates how to improve the camera's shooting method.

[0318] The specific implementation process of steps 1301 to 1303 can be referred to the descriptions in Figures 5, 6A, 6B, 7A, and 7B above, and will not be repeated here. In addition, the specific implementation process of steps 1301 to 1303 can also be combined with the descriptions in Figures 8 to 12, which will not be repeated here.

[0319] Based on the same inventive concept, Figure 14 is another schematic flowchart of an image content processing method provided in an embodiment of this application. This process can be applied to electronic devices. The process may include the following steps:

[0320] Step 1401: Receive an instruction to execute a preset task, wherein the preset task includes the step of image content recognition;

[0321] Step 1402: Obtain a dynamic image for the image content recognition, wherein the dynamic image is captured by a camera;

[0322] Step 1403: Select the frame with the best quality in the dynamic image for image content recognition, or perform image content recognition based on multiple frames in the dynamic image;

[0323] Step 1404: Execute the preset task based on the result of the image content recognition.

[0324] The specific implementation process of steps 1401 to 1404 can be referred to the descriptions in Figures 8 and 9 above, and will not be repeated here. In addition, the specific implementation process of steps 1401 to 1404 can also be combined with the descriptions in Figures 5 to 7B and Figures 10 to 12, and will not be repeated here.

[0325] Based on the same inventive concept, Figure 15 is another schematic flowchart of an image content processing method provided in an embodiment of this application. This process can be applied to electronic devices. The process may include the following steps:

[0326] Step 1501: Receive an instruction to execute a preset task, wherein the preset task includes a step of image content recognition, and the preset task includes image translation or image search;

[0327] Step 1502: Obtain a first image for the image content recognition, wherein the first image is captured by a camera;

[0328] Step 1503: Send a request message to the server, the request message being generated based on the first image;

[0329] Step 1504: Receive image recognition enhancement information from the server, the image recognition enhancement information being generated based on the request information;

[0330] Step 1505: Execute the preset task based on the first image and the image recognition enhancement information.

[0331] The specific implementation process of steps 1501 to 1505 can be referred to the descriptions in Figures 10 and 11 above, and will not be repeated here. In addition, the specific implementation process of steps 1501 to 1505 can also be combined with the descriptions in Figures 5 to 9 and Figure 12, and will not be repeated here.

[0332] The specific implementation processes of Figures 13 to 15 can be combined. For details of the implementation process, please refer to the previous introduction. They will not be repeated here.

[0333] Based on the above embodiments, this application also provides an electronic device, which includes multiple functional modules; the multiple functional modules interact with each other to realize the functions performed by the electronic device in the various methods described in the embodiments of this application. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on specific implementations.

[0334] Based on the above embodiments, this application also provides an electronic device, which includes at least one processor and at least one memory, wherein the at least one memory stores computer program instructions, and when the electronic device is running, the at least one processor performs the functions performed by the electronic device in the various methods described in the embodiments of this application.

[0335] Based on the above embodiments, this application also provides an image content processing system, which may include the electronic device and server as described in the foregoing embodiments.

[0336] Based on the above embodiments, this application also provides a computer program product, which includes a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the methods described in the embodiments of this application.

[0337] Based on the above embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the methods described in the embodiments of this application.

[0338] Based on the above embodiments, this application also provides a chip for reading computer programs stored in a memory to implement the methods described in the embodiments of this application.

[0339] Based on the above embodiments, this application provides a chip system including a processor for supporting a computer device in implementing the methods described in the embodiments of this application. In one possible design, the chip system further includes a memory for storing necessary programs and data of the computer device. This chip system may be composed of chips or may include chips and other discrete devices. Those skilled in the art will understand that the embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0340] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0341] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0342] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0343] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of protection of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An image content processing method, characterized in that, Applied to electronic devices, the method includes: Receive an instruction to execute a preset task, the preset task including a step of image content recognition, the preset task including image translation or image search; Acquire a first image for the image content recognition, the first image being captured by a camera; The system displays a first prompt message indicating that the quality of the first image is low and suggesting ways to improve the camera's shooting capabilities.

2. The method according to claim 1, characterized in that, The method of improving the camera's capture is indicated from at least one of the following aspects: Improve the stability of the camera during shooting; Point the camera at the content to be identified; Adjust the shooting angle; Avoid reflective areas; Adjust the size of the target content in the image; and Adjust the exposure level during shooting.

3. The method according to claim 1 or 2, characterized in that, After displaying the first prompt message, the method further includes: In response to acquiring a second image of higher quality than the first image, the preset task is performed based on the second image; or In response to the failure to acquire other images of higher quality than the first image within a preset time, the preset task is performed based on the first image; or In response to the user's confirmation instruction to perform the preset task using the first image, the preset task is performed based on the first image.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: A third image, captured by a camera, is acquired for the image content recognition. A second prompt message is displayed, which indicates that the quality of the third image meets the preset task. The preset task is performed based on the third image.

5. The method according to any one of claims 1-4, characterized in that, The first image and / or the second image are preview images taken by the electronic device before the electronic device takes the picture, or images taken by the electronic device, or images saved on the electronic device.

6. The method according to any one of claims 1-5, characterized in that, The first image is and / or the second image is a single-frame image or a multi-frame image.

7. The method according to any one of claims 3-6, characterized in that, The first image, the second image, or the third image is a dynamic image; Performing the preset task based on the first image, the second image, or the third image includes: The image content recognition is performed by selecting the frame with the best quality from the dynamic image, or by performing the image content recognition based on multiple frames from the dynamic image. The preset task is executed based on the result of the image content recognition.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: Send a request message to the server, the request message being generated based on the first image; Receive image recognition enhancement information from the server, the image recognition enhancement information being generated based on the request information; The preset task is performed based on the first image and the image recognition enhancement information.

9. An image content processing method, characterized in that, Applied to electronic devices, the method includes: Receive an instruction to execute a preset task, the preset task including the step of image content recognition; Acquire a dynamic image for the image content recognition, the dynamic image being captured by a camera; The image content recognition is performed by selecting the frame with the best quality from the dynamic image, or by performing the image content recognition based on multiple frames from the dynamic image. The preset task is executed based on the result of the image content recognition.

10. The method according to claim 9, characterized in that, After acquiring the dynamic image for the image content recognition, the method further includes: The system displays a first prompt message indicating that the quality of the moving image is low and suggesting ways to improve the camera's shooting capabilities.

11. The method according to claim 9 or 10, characterized in that, The method further includes: Send a request message to the server, the request message being generated based on the dynamic image; Receive image recognition enhancement information from the server, the image recognition enhancement information being generated based on the request information; The preset task is performed based on the dynamic image and the image recognition enhancement information.

12. An image content processing method, characterized in that, Applied to electronic devices, the method includes: Receive an instruction to execute a preset task, the preset task including a step of image content recognition, the preset task including image translation or image search; Acquire a first image for the image content recognition, the first image being captured by a camera; Send a request message to the server, the request message being generated based on the first image; Receive image recognition enhancement information from the server, the image recognition enhancement information being generated based on the request information; The preset task is performed based on the first image and the image recognition enhancement information.

13. The method according to claim 12, characterized in that, The method further includes: The first image is processed to obtain a first candidate result; Send the first image to the server; Receive associated images from the server; Image content processing is performed on the associated image to obtain a second candidate result; The first candidate result and the second candidate result are fused to obtain the processing result.

14. The method according to claim 12, characterized in that, The method further includes: The first image is processed to obtain a first candidate result; Send the first image to the server; Receive a second candidate result from the server, the second candidate result being generated based on the first image; The first candidate result and the second candidate result are fused to obtain the processing result.

15. The method according to claim 12, characterized in that, The method further includes: Send the first image to the server; Receive a second image from the server, the second image being retrieved based on the first image, or the second image being generated based on the first image; The first image and the second image are fused to obtain the processing result.

16. The method according to any one of claims 12-15, characterized in that, After acquiring the first image for image content recognition, the method further includes: The system displays a first prompt message indicating that the quality of the first image is low and suggesting ways to improve the camera's shooting capabilities.

17. The method according to any one of claims 12-16, characterized in that, The step of performing the preset task based on the first image and the image recognition enhancement information includes: The image content recognition is performed by selecting the frame with the best quality from the first image or by using multiple frames from the first image and the image recognition enhancement information. The preset task is executed based on the result of the image content recognition.

18. An electronic device, characterized in that, The method includes at least one processor coupled to at least one memory, the at least one processor being configured to read a program stored in the at least one memory to perform the method as claimed in any one of claims 1-8, or to perform the method as claimed in any one of claims 9-11, or to perform the method as claimed in any one of claims 12-17.

19. A readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-8, or the method as described in any one of claims 9-11, or the method as described in any one of claims 12-17.

20. A computer program product comprising instructions, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-8, or the method as described in any one of claims 9-11, or the method as described in any one of claims 12-17.

Images