Image processing method, apparatus, device, and storage medium

Abstract

Embodiments of the present application provide an image processing method, device and equipment and storage medium, the method comprises: when detecting that the target application program acquires the first image, the first image is preprocessed through the plug-in program to reduce part of the image information in the first image; the first image after preprocessing is input to the target application program, so that the target application program carries out subsequent processing to the first image after preprocessing. Through the plug-in program, part of the image information in the first image has been reduced, and then the calculation amount of the target application program in the image processing process can be reduced, and the image processing complexity is reduced, so that the target application program can better adapt to the hardware running environment of the smart television, and the occurrence of phenomena such as lag, time-consuming processing, recognition error and the like is reduced.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] In practical applications, both smartphones and smart TVs can install the same operating system, such as Android. Therefore, applications that run on a smartphone's operating system can also run on a smart TV's operating system. With the widespread use of smartphones, many manufacturers invest significant R&D resources in developing smartphone applications. Since applications that run on smartphones can also run on smart TVs, manufacturers simply migrate applications installed on smartphones directly to smart TVs.

[0003] Because smartphones have a significantly better hardware operating environment than smart TVs—for example, their central processing unit (CPU) and graphics processing unit (GPU) are more powerful, they have more RAM, and their refresh rates are higher—image processing applications running on smartphones are less prone to lag, processing delays, and recognition errors. However, if the same application runs on a smart TV, the limitations of the smart TV's hardware environment cause frequent lag, processing delays, and recognition errors, significantly impacting the user experience. Summary of the Invention

[0004] This invention provides an image processing method, apparatus, device, and storage medium to reduce issues such as lag, time-consuming processing, and recognition errors in image processing by applications running on smart TVs.

[0005] In a first aspect, embodiments of the present invention provide an image processing method, the method being applied to a plug-in program, the plug-in program being integrated into a target application program, the target application program being an application installed on a smart TV, the target application program being used to process images, the method comprising:

[0006] When the target application is detected to have acquired the first image, the first image is preprocessed by the plug-in program to reduce some image information in the first image;

[0007] The preprocessed first image is input into the target application so that the target application can perform subsequent processing on the preprocessed first image.

[0008] Optionally, the preprocessing of the first image through the plug-in program includes:

[0009] The plugin program performs color weakening and / or contour detection processing on the first image.

[0010] Optionally, the color weakening process on the first image includes:

[0011] Determine the size of the first image;

[0012] Create a blank canvas whose size is the same as that of the first image, and the pixels in the blank canvas correspond one-to-one with the pixels in the first image;

[0013] For a target pixel in the first image, the pixel value of the target pixel is converted from the initial data format to a target preset data format, wherein the number of bytes occupied by the pixel value with the target preset data format is less than the number of bytes occupied by the corresponding pixel value with the initial data format, and the target pixel is any pixel in the first image;

[0014] The converted pixel values ​​are assigned to the pixels in the blank canvas that correspond to the target pixel to obtain the second image.

[0015] Optionally, the contour detection processing of the first image includes:

[0016] The second image is subjected to contour detection processing to obtain contour detection processing results, wherein the contour detection processing results are used to reflect whether each pixel in the second image is located at the boundary position;

[0017] The method further includes:

[0018] For the first pixel located at the boundary position in the second image, retain the pixel value of each first pixel;

[0019] For the second pixel points in the second image that are not located at the boundary position, the pixel value of each second pixel point is set to a preset value.

[0020] Optionally, inputting the preprocessed first image into the target application so that the target application can perform subsequent processing on the preprocessed first image includes:

[0021] The preprocessed first image is input into the target application so that the target application can perform recognition processing on the preprocessed first image.

[0022] Secondly, embodiments of the present invention provide an image processing apparatus, the apparatus being disposed in a plug-in program, the plug-in program being connected to a target application program, the target application program being a program installed on a smart TV, the target application program being used to process images, the apparatus comprising:

[0023] The preprocessing module is used to preprocess the first image through the plug-in program when the target application is detected to acquire the first image, so as to reduce some image information in the first image;

[0024] An input module is used to input the preprocessed first image into the target application so that the target application can perform subsequent processing on the preprocessed first image.

[0025] Optionally, the preprocessing module is used for:

[0026] The plugin program performs color weakening and / or contour detection processing on the first image.

[0027] Optionally, the preprocessing module is used for:

[0028] Determine the size of the first image;

[0029] Create a blank canvas whose size is the same as that of the first image, and the pixels in the blank canvas correspond one-to-one with the pixels in the first image;

[0030] For a target pixel in the first image, the pixel value of the target pixel is converted from the initial data format to a target preset data format, wherein the number of bytes occupied by the pixel value with the target preset data format is less than the number of bytes occupied by the corresponding pixel value with the initial data format, and the target pixel is any pixel in the first image;

[0031] The converted pixel values ​​are assigned to the pixels in the blank canvas that correspond to the target pixel to obtain the second image.

[0032] Optionally, the preprocessing module is used for:

[0033] The second image is subjected to contour detection processing to obtain contour detection processing results, wherein the contour detection processing results are used to reflect whether each pixel in the second image is located at the boundary position;

[0034] For the first pixel located at the boundary position in the second image, retain the pixel value of each first pixel;

[0035] For the second pixel points in the second image that are not located at the boundary position, the pixel value of each second pixel point is set to a preset value.

[0036] Optionally, the input module is used for:

[0037] The preprocessed first image is input into the target application so that the target application can perform recognition processing on the preprocessed first image.

[0038] Thirdly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the image processing method of the first aspect.

[0039] Fourthly, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the image processing method of the first aspect.

[0040] By employing this invention, before the target application actually processes the received first image, a plug-in program integrated into the target application can preprocess the first image to reduce some image information. This ensures that the first image, even after reducing some image information, still retains the necessary image information for subsequent image processing by the target application. Therefore, even with the reduction of some image information, the target application's subsequent normal image processing is not affected. Furthermore, because the plug-in program has reduced some image information in the first image, the computational load during image processing by the target application is reduced, lowering the complexity of image processing. This allows the target application to better adapt to the hardware operating environment of smart TVs, reducing lag, processing time, and recognition errors. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0047] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0048] Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0049] This invention provides an image processing method applicable to electronic devices, such as smart TVs and other smart terminals. The method provided in this invention can be written as a plug-in program and integrated into a target application. Specifically, the method provided in this invention can be written as a corresponding plug-in program and integrated into the target application so that the corresponding algorithm library file of the plug-in program can be called and executed at appropriate times when the target application is executed.

[0050] The target application mentioned above can be an image processing application installed on a smart TV, or it can be a game application. For example, the target application can be an application used for processing input images, such as recognition. For instance, the target application could be a shopping application installed on a smart TV that receives the input image, identifies objects in the image, and then searches for and provides corresponding products to the user based on the recognition results. Or, for another example, the target application could be a photo editing application installed on a smart TV that receives the input image and makes corresponding modifications to the received image according to the user's actions.

[0051] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0052] 101. When it is detected that the target application has acquired the first image, the first image is preprocessed by the plug-in program to reduce some of the image information in the first image.

[0053] 102. Input the preprocessed first image into the target application so that the target application can perform subsequent processing on the preprocessed first image.

[0054] In practical applications, a first image can be input into the target application. This first image is the image that needs to be processed by the functions provided by the target application. The first image can be an image taken by a camera connected to the smart TV, an image uploaded by the user to the smart TV, or an image downloaded from a server by a smart TV with internet access, etc.

[0055] Understandably, the plugin can be integrated into the target application. When the target application receives the first image, it stops processing the first image and instead preprocesses it. This preprocessing reduces some image information, such as color and texture details. This reduction does not affect subsequent processing by the target application; the reduced information is unused in later processing.

[0056] In some alternative embodiments, the process of preprocessing the first image by means of a plug-in program can be implemented as follows: performing color weakening processing on the first image by means of a plug-in program; or performing contour detection processing on the first image; or performing both color weakening processing and contour detection processing on the first image.

[0057] For color weakening processing, the color information in the first image can be reduced, for example, by reducing the saturation of the first image. Assuming the target application is an image recognition application, even with some color information in the first image reduced, the object type in the first image can still be identified. Therefore, appropriately reducing the color information in the first image can reduce the computational load of the image recognition application during image recognition, lower the complexity of image processing, and thus allow the image recognition application to better adapt to the hardware operating environment of smart TVs, reducing phenomena such as lag, processing time, and recognition errors.

[0058] Contour detection processing can reduce the texture details of objects in the first image. Even when some texture details of objects in the first image are missing, it does not affect subsequent processing by the target application. For example, if the target application is an image recognition application, suppose contour detection processing reduces the pattern and texture of a backpack in the first image, but the backpack's outline remains in the first image, thus not affecting the image recognition application's accurate identification of the object type in the first image.

[0059] The following section will detail the specific implementation methods of the two image preprocessing techniques.

[0060] (1) Color weakening treatment.

[0061] Optionally, the process of color weakening processing on the first image can be implemented as follows: determine the size of the first image; create a blank canvas, the size of which is consistent with the size of the first image, and the pixels in the blank canvas correspond one-to-one with the pixels in the first image; for a target pixel in the first image, convert the pixel value of the target pixel from the initial data format to a target preset data format, wherein the number of bytes occupied by the pixel value with the target preset data format is less than the number of bytes occupied by the corresponding pixel value with the initial data format, and the target pixel is any pixel in the first image; assign the converted pixel value to the pixel in the blank canvas corresponding to the target pixel to obtain the second image.

[0062] In practical applications, firstly, the size of the first image can be determined, and then a blank canvas of the same size can be created based on the size of the first image. For example, if the first image is 16×16 pixels, a blank canvas of the same size can be created. Since the size of the blank canvas matches the size of the first image, the number of pixels in the blank canvas is equal to the number of pixels in the first image. Therefore, every pixel in the blank canvas can be found in a corresponding pixel in the first image. For instance, the pixels in the first row and first column of the blank canvas correspond to the pixels in the first row and first column of the first image, and the pixels in the fifth row and fifth column of the blank canvas correspond to the pixels in the fifth row and fifth column of the first image.

[0063] Next, for the target pixels in the first image, assuming the pixel value data format is the initial data format, the pixel value can be converted from the initial data format to the target preset data format using corresponding commands. The initial data format could be, for example, an `int` format, and the target preset data format could be, for example, a `short` format. Each `int` format pixel value requires 4 bytes of storage space, while each `short` format pixel value only requires 2 bytes of storage space. Therefore, by converting the data format of the target pixel values, the storage space occupied by the target pixel values ​​can be reduced. By converting the data format of each pixel value in the first image, the overall storage space occupied by the first image can be reduced. Finally, the converted pixel values ​​can be assigned to the corresponding pixels on the blank canvas, thereby generating the second image.

[0064] In this way, the storage space occupied by the second image is smaller than that occupied by the first image. The second image implements color weakening processing based on the first image. Compared with the first image, the color representation space of the second image is smaller, but it does not affect the normal processing of the second image by the target application.

[0065] (2) Contour detection processing.

[0066] Optionally, the process of performing contour detection processing on the first image can be implemented as follows: performing contour detection processing on the second image to obtain contour detection processing results, wherein the contour detection processing results are used to reflect whether each pixel in the second image is located at the boundary position.

[0067] Accordingly, the method provided in this embodiment of the invention may further include: for first pixels located at the boundary position in the second image, retaining the pixel value of each first pixel; for second pixels not located at the boundary position in the second image, setting the pixel value of each second pixel to a preset value.

[0068] In practical applications, assuming that the first image has been color-weakening processed to obtain the second image, contour detection processing can then be performed on the second image to determine whether each pixel in the second image is located at the boundary position of an object.

[0069] For the first pixel located at the boundary in the second image, the pixel value of the first pixel that has already undergone color weakening processing in the second image can be retained. For the second pixel not located at the boundary in the second image, the pixel values ​​of all second pixels can be uniformly set to the same preset value, such as all set to a fixed value such as 0 or 255. In this way, the outline of the object in the second image can be preserved, while the texture details of the object in the second image can be removed.

[0070] The above describes how to reduce some image information in the first image through preprocessing. After preprocessing the first image, the preprocessed first image can be input into the target application so that the target application can perform subsequent processing on the preprocessed first image. In some optional embodiments, the subsequent processing may optionally include: inputting the preprocessed first image into the target application so that the target application can perform recognition processing on the preprocessed first image.

[0071] For example, in a shopping application, a user takes and uploads a first image. The plugin can preprocess the first image to reduce unnecessary color information used in the image recognition process. It can also extract the outline of the target object from the first image after color reduction. The plugin then inputs the image corresponding to the target object's outline into the shopping application. This significantly reduces unnecessary information elements in the image that the shopping application needs to process, thereby improving image recognition efficiency and reducing the computational load. Finally, the shopping application can perform image recognition processing based on the image corresponding to the target object's outline to identify the object type.

[0072] Besides the application scenarios mentioned above, the target application in this embodiment of the invention can also be a game application. It is understood that for a 3D game application, the posture of each character in world space needs to be determined first based on the player's control, and then the scene in world space is mapped onto screen space. The image in screen space is a two-dimensional image, which can be displayed on a smart TV screen. Using the method provided in this embodiment of the invention, before displaying the image, the two-dimensional image in screen space (i.e., the first image) can be preprocessed. For example, the color information in the first image can be reduced and the outlines of objects or characters other than the main character can be extracted, ultimately generating a second image. In the second image, objects or characters other than the main character only display outlines and not internal texture details. This reduces the second image's footprint on the smart TV's display memory and reduces lag during gameplay.

[0073] By employing this invention, before the target application actually processes the received first image, a plug-in program integrated into the target application can preprocess the first image to reduce some image information. This ensures that the first image, even after reducing some image information, still retains the necessary image information for subsequent image processing by the target application. Therefore, even with the reduction of some image information, the target application's subsequent normal image processing is not affected. Furthermore, because the plug-in program has reduced some image information in the first image, the computational load during image processing by the target application is reduced, lowering the complexity of image processing. This allows the target application to better adapt to the hardware operating environment of smart TVs, reducing lag, processing time, and recognition errors.

[0074] The image processing apparatus of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will understand that these image processing apparatuses can all be configured using commercially available hardware components through the steps taught in this invention.

[0075] Figure 2 This is a schematic diagram of an image processing device provided in an embodiment of the present invention. The device is installed in a plug-in program, which is connected to a target application program. The target application program is a program installed on a smart TV, and the target application program is used to process images, such as... Figure 2 As shown, the device includes:

[0076] The preprocessing module 21 is used to preprocess the first image through the plug-in program when the target application is detected to acquire the first image, so as to reduce some image information in the first image;

[0077] The input module 22 is used to input the preprocessed first image into the target application so that the target application can perform subsequent processing on the preprocessed first image.

[0078] Optionally, the preprocessing module 21 is used for:

[0079] The plugin program performs color weakening and / or contour detection processing on the first image.

[0080] Optionally, the preprocessing module 21 is used for:

[0081] Determine the size of the first image;

[0082] Create a blank canvas whose size is the same as that of the first image, and the pixels in the blank canvas correspond one-to-one with the pixels in the first image;

[0083] For a target pixel in the first image, the pixel value of the target pixel is converted from the initial data format to a target preset data format, wherein the number of bytes occupied by the pixel value with the target preset data format is less than the number of bytes occupied by the corresponding pixel value with the initial data format, and the target pixel is any pixel in the first image;

[0084] The converted pixel values ​​are assigned to the pixels in the blank canvas that correspond to the target pixel to obtain the second image.

[0085] Optionally, the preprocessing module 21 is used for:

[0086] The second image is subjected to contour detection processing to obtain contour detection processing results, wherein the contour detection processing results are used to reflect whether each pixel in the second image is located at the boundary position;

[0087] For the first pixel located at the boundary position in the second image, retain the pixel value of each first pixel;

[0088] For the second pixel points in the second image that are not located at the boundary position, the pixel value of each second pixel point is set to a preset value.

[0089] Optionally, the input module 22 is used for:

[0090] The preprocessed first image is input into the target application so that the target application can perform recognition processing on the preprocessed first image.

[0091] Figure 2 The device shown can perform the aforementioned Figure 1The image processing method provided in the illustrated embodiment, for detailed execution process and technical effects, can be found in the description of the foregoing embodiments, and will not be repeated here.

[0092] In one possible design, the above Figure 2 The structure of the image processing device shown can be implemented as an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 91 and a memory 92. The memory 92 stores executable code, which, when executed by the processor 91, enables the processor 91 to at least perform the functions described above. Figure 1 The image processing method provided in the illustrated embodiment.

[0093] Optionally, the electronic device may also include a communication interface 93 for communicating with other devices.

[0094] Furthermore, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least perform the functions described above. Figure 1 The image processing method provided in the illustrated embodiment.

[0095] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] The image processing method provided in this embodiment of the invention can be executed by a program / software, which can be provided by a network side. The electronic device mentioned in the foregoing embodiments can download the program / software to a local non-volatile storage medium, and when it needs to execute the aforementioned image processing method, the CPU reads the program / software into memory, and then the CPU executes the program / software to implement the image processing method provided in the foregoing embodiments. The execution process can be referred to the foregoing... Figure 1 The illustration is shown in the image.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized in that, The method is applied to a plug-in program, which is integrated into a target application, which is a program installed on a smart TV and is used to process images. The method includes: When the target application is detected to have acquired the first image, the first image is preprocessed by the plug-in program; The preprocessed first image is input into the target application so that the target application can identify the object type in the preprocessed first image. The preprocessing of the first image using the plugin program includes: determining the size of the first image using the plugin program; creating a blank canvas whose size matches that of the first image, with each pixel in the blank canvas corresponding to a pixel in the first image; converting the pixel value of a target pixel in the first image from an initial data format to a target preset data format, wherein the number of bytes occupied by the pixel value with the target preset data format is less than the number of bytes occupied by the corresponding pixel value with the initial data format, and the target pixel being any pixel in the first image; and assigning the converted pixel value to the pixel in the blank canvas corresponding to the target pixel to obtain the second image. The second image is subjected to contour detection processing to obtain contour detection processing results, wherein the contour detection processing results are used to reflect whether each pixel in the second image is located at the boundary position; for the first pixel in the second image located at the boundary position, the pixel value of each first pixel is retained; for the second pixel in the second image not located at the boundary position, the pixel value of each second pixel is set to a preset value.

2. An image processing apparatus, characterized in that, The device is installed in a plug-in program, which is connected to a target application. The target application is a program installed on a smart TV and is used to process images. The device includes: The preprocessing module is used to preprocess the first image through the plug-in program when the target application is detected to have acquired the first image; An input module is used to input the preprocessed first image into the target application, so that the target application can identify the object type in the preprocessed first image; The preprocessing module is used for: The size of the first image is determined by the plug-in program; A blank canvas is created, the size of which is the same as the size of the first image, and the pixels in the blank canvas correspond one-to-one with the pixels in the first image. For a target pixel in the first image, the pixel value of the target pixel is converted from an initial data format to a target preset data format, wherein the number of bytes occupied by the pixel value with the target preset data format is less than the number of bytes occupied by the corresponding pixel value with the initial data format. The target pixel is any pixel in the first image. The converted pixel value is assigned to the pixel in the blank canvas corresponding to the target pixel to obtain the second image. The second image is subjected to contour detection processing to obtain contour detection processing results, wherein the contour detection processing results are used to reflect whether each pixel in the second image is located at the boundary position; for the first pixel in the second image located at the boundary position, the pixel value of each first pixel is retained; for the second pixel in the second image not located at the boundary position, the pixel value of each second pixel is set to a preset value.

3. An electronic device, characterized in that, include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor performs the image processing method as described in claim 1.

4. A non-transitory machine-readable storage medium, characterized in that, The non-transitory machine-readable storage medium stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the image processing method as described in claim 1.

Images