Image processing method and device

Abstract

The invention discloses an image processing method and device, the method is applied to at least one image processing module, and the method comprises the steps: obtaining a first image; the first image comes from the image acquisition module; processing the first image based on a target mechanism to obtain a processing result; the target mechanism is a mechanism configured for calling an application of the image acquisition module or a mechanism configured for a hardware module related to the image acquisition module; different mechanisms correspond to different image processing logics; and transmitting the processing result to the next image processing module or application.

Description

An image processing method and apparatus Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method and apparatus. Background Technology

[0002] At least one image processing module can be installed on an electronic device. After the image acquisition module acquires a frame of image, the electronic device can call these image processing modules to process the image, output the obtained processing result, or execute relevant control logic based on the processing result.

[0003] In related technologies, each image processing module can only implement fixed image processing logic. For example, one image processing module is fixed to enhance image brightness, while another is fixed to recognize faces in the image. However, electronic devices may have multiple usage scenarios, and the fixed image processing logic of the image processing modules is obviously not conducive to meeting the different image processing logic required for different usage scenarios. Summary of the Invention

[0004] Therefore, this application discloses the following technical solution:

[0005] The first aspect of this application provides an image processing method applied to at least one image processing module, comprising:

[0006] The first image is obtained; the first image comes from the image acquisition module;

[0007] The first image is processed based on a target mechanism to obtain a processing result; the target mechanism is a mechanism configured for the application that calls the image acquisition module, or a mechanism configured for the hardware module related to the image acquisition module; different mechanisms correspond to different image processing logic;

[0008] The processing result is transmitted to the next image processing module or the application.

[0009] Optional, also includes:

[0010] Based on at least one of the configuration for the application and the configuration for the hardware module, at least one mechanism is determined as the target mechanism from a plurality of pre-registered mechanisms; the plurality of pre-registered mechanisms are mechanisms of the at least one image processing module.

[0011] Optionally, the number of the target mechanisms is at least two;

[0012] The process of processing the first image based on the target mechanism to obtain the processing result includes:

[0013] The first image is processed based on the first objective mechanism among at least two of the objective mechanisms to obtain an intermediate result;

[0014] The intermediate result is processed based on the second objective mechanism of at least two objective mechanisms to obtain the processing result.

[0015] Optionally, target mechanisms can be configured for applications or hardware modules, including:

[0016] Display the configuration interface corresponding to the application or the hardware module, the configuration interface including different functional information for describing the functions implemented by different mechanisms;

[0017] In response to an operation instruction for at least one functional information, the mechanism corresponding to the targeted functional information is configured as the target mechanism corresponding to the application.

[0018] Optionally, the registration process for the target mechanism includes:

[0019] Obtain the target registration token of the target mechanism;

[0020] Determine whether the target registration token matches at least one token in the registry, the registry including tokens written to the application when an application that invokes the image acquisition module is installed;

[0021] If the target registration token matches at least one token in the registry, the target mechanism is confirmed to be a registered mechanism.

[0022] Optional, also includes:

[0023] If the processing time of the first image based on the target mechanism is greater than the processing response time of the target mechanism, the first image is discarded and the second image is processed based on the target mechanism. The second image is the next frame image from the first image obtained by the image acquisition module.

[0024] The processing response time of the target mechanism is determined when registering the target mechanism based on the processing capability of the target mechanism and the maximum response time preset by the electronic device.

[0025] Optionally, obtaining the first image includes at least one of the following:

[0026] The image output by the image acquisition module is obtained as the first image;

[0027] The image processed by another image processing module, which is different from the at least one image processing module, is obtained as the first image.

[0028] Optionally, the processing of the first image based on the target mechanism to obtain the processing result includes at least one of the following:

[0029] The target text is added to the first image to obtain an image containing the target text as the processing result. The target text is obtained based on speech recognition collected by the audio acquisition module.

[0030] The sharpness of the first image is reduced to obtain a blurred image with a sharpness lower than that of the first image as the processing result;

[0031] Obtain control instructions related to the image recognition result of the first image. The control instructions serve as the processing result and are used to control the hardware modules related to the image acquisition module to be enabled or disabled.

[0032] A second aspect of this application provides an electronic device, including an image acquisition module, at least one image processing module, a memory, and at least one processor;

[0033] The memory is used to store at least one image processing module;

[0034] The processor is used to load at least one image processing module to perform:

[0035] The first image is obtained; the first image comes from the image acquisition module;

[0036] The first image is processed based on a target mechanism to obtain a processing result; the target mechanism is a mechanism configured for the application that calls the image acquisition module, or a mechanism configured for the hardware module related to the image acquisition module; different mechanisms correspond to different image processing logic;

[0037] The processing result is transmitted to the next image processing module or the application.

[0038] Optionally, the at least one image processing module includes a registration interface and an authorization interface;

[0039] The registration interface is used to confirm that the target mechanism is a registered mechanism based on the target registration token of the target mechanism;

[0040] The authorization interface is used to confirm that the target mechanism has been registered, so as to process the first image based on the registered target mechanism and obtain the processing result. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0042] Figure 1 is a flowchart of an image processing method provided in an embodiment of this application;

[0043] Figure 2 is a schematic diagram showing the connection relationship between an image processing module and an effects library provided in an embodiment of this application;

[0044] Figure 3 is a schematic diagram of a system architecture provided in an embodiment of this application;

[0045] Figure 4 is a schematic diagram of a registration and authorization mechanism provided in an embodiment of this application;

[0046] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0047] Figure 6 is a schematic diagram of a configuration interface provided in an embodiment of this application;

[0048] Figure 7 is a schematic diagram of another configuration interface provided in an embodiment of this application. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] To facilitate understanding of the technical solutions of the embodiments of this application, some terms that may be involved will be explained first.

[0051] The video capture stack is a program component provided by the Windows operating system for capturing and processing images. This component can capture images acquired by the image acquisition module of an electronic device, process the captured images through at least one image processing module within the component, and output the processed images or other data to other programs on the electronic device.

[0052] The Device Media Foundation Transform (DMFT) module is a filter-level image processing module supported in the video capture stack. A video capture stack can include one or more DMFT modules. Each DMFT module runs on the Media Foundation framework and processes the input data based on its own processing logic. As an example, the video capture stack in the Windows operating system can include four DMFT modules. These four DMFT modules can be chained together to form a chain structure, called a DMFT chain. The video capture stack can sequentially call the four DMFT modules in the DMFT chain to process the captured images.

[0053] In related technologies, different DMFT modules have different processing logic (or different functions). For example, the four DMFT modules in a video capture stack may include a device proxy DMFT module, a platform DMFT module, an independent hardware vendor DMFT module, and an original equipment manufacturer DMFT module.

[0054] The device agent DMFT module is an asynchronous DMFT module used to marshal commands and video frames between the camera driver that drives the image acquisition module and the DMFT. The processing logic of this module can be implemented by the operating system vendor.

[0055] The Platform Device Media Foundation Transform (PDMFT) module is a device-level DMFT module built into the video capture stack of the Windows operating system. The core function of this module is to allow IHV (Independent Hardware Vendor) and OEM (Original Equipment Manufacturer) to directly call the post-processing algorithms built into the Windows operating system, such as basic image optimization and format conversion. The processing logic of this DMFT module can be implemented by the operating system vendor.

[0056] The Independent Hardware Vendor (IHV) DMFT module is a device-specific DMFT module provided by an independent hardware vendor. Its main function is to receive pre-processed video frames from the device and perform further customized post-processing on these frames within the module according to its inherent processing logic, such as hardware-level noise reduction and color enhancement. This processing logic can be implemented by the independent hardware vendor.

[0057] Original Equipment Manufacturer (OEM) DMFT modules are image processing modules provided by OEMs. Their function is similar to that of independent hardware vendor DMFT modules, used to process input images according to their own inherent processing logic. This processing logic can be implemented by the OEM.

[0058] In related technologies, after electronic devices are manufactured and released to users, the processing logic of their built-in image processing modules, such as the aforementioned DMFT module, is pre-configured and fixed by the operating system or relevant manufacturers. Therefore, the functions these image processing modules can perform are fixed. If there are other functional requirements different from these fixed functions during actual use, they cannot be achieved through the image processing module. For example, the DMFT module contains processing logic for format conversion, hardware-level noise reduction, and color enhancement. When it is necessary to blur the acquired image, the DMFT module cannot be invoked to perform this function.

[0059] On the other hand, even if new processing logic is introduced by adding new image processing modules, there is still the problem that too many image processing modules will increase the complexity of the electronic device's software system. Therefore, it is necessary to limit the number of image processing modules, which in turn restricts the functionality of the image processing modules.

[0060] To address the shortcomings of the aforementioned technologies, this embodiment provides an image processing method applied to at least one image processing module. As shown in Figure 1, the method may include the following steps.

[0061] S101, Obtain the first image; the first image comes from the image acquisition module.

[0062] S102, process the first image based on the target mechanism to obtain the processing result; the target mechanism is a mechanism configured for the application that calls the image acquisition module, or a mechanism configured for the hardware module related to the image acquisition module; different mechanisms correspond to different image processing logic.

[0063] S103, transmits the processing results to the next image processing module or application.

[0064] The beneficial effects of this embodiment are as follows:

[0065] After applying the method of this embodiment, for different applications or hardware modules, the image processing module can select different mechanisms with different processing logic to process the first image as needed according to the configuration of the application or hardware module. For example, if application 1 is configured with mechanism 1 as the target mechanism, the first image can be processed based on mechanism 1. If application 2 is configured with mechanism 2 as the target mechanism, the first image can be processed based on mechanism 2.

[0066] In this way, the image processing module can no longer process the first image solely based on its own inherent processing logic. Instead, it can flexibly apply different image processing logics to process the first image according to the configuration of the application or hardware module. This allows the electronic device to adapt to the functional requirements of different usage scenarios by configuring different mechanisms without adding an image processing module. This does not increase the software complexity of the electronic device, facilitates software maintenance and updates, and expands the image processing capabilities of the electronic device.

[0067] Furthermore, there are sometimes situations where multiple applications need to use the same image processing module to implement their respective processing logic. For example, a conferencing application might need to use image processing module 2 for image blurring to achieve anti-eavesdropping effects, while a video playback application might need to use image processing module 2 for image enhancement. Image processing modules with fixed processing logic in related technologies obviously cannot meet the conflicting functional requirements of multiple applications, resulting in these applications being unable to be installed and function simultaneously on electronic devices.

[0068] By applying the method of this embodiment, the same image processing module can process images according to current needs based on mechanisms that implement different image processing logics, enabling electronic devices to be compatible with multiple applications with conflicting functional requirements, and meeting the needs of users to install applications with different functions.

[0069] The image acquisition module can be at least one camera on an electronic device.

[0070] Optionally, the first image is obtained, including at least one of the following:

[0071] Method 1: Obtain the image output by the image acquisition module as the first image;

[0072] Method 2: Obtain the image processed by another image processing module that is different from at least one image processing module as the first image.

[0073] At least one image processing module executing the image processing method of this embodiment can be any one or more image processing modules in an image processing chain contained in an electronic device. An image processing chain refers to a chain structure formed by sequentially connecting multiple image processing modules of an electronic device, used to process acquired images in sequence.

[0074] For example, at least one image processing module can be any one of the DMFT modules in the DMFT chain formed by connecting DMFT modules 0 to DMFT modules 3 as shown in Figure 2.

[0075] Depending on the position of the image processing module in the image processing chain, the way the image processing module obtains the first image and the way it transmits the processing results can be different.

[0076] Taking a Windows operating system as an example, the method in this embodiment can be executed by the first DMFT module in the DMFT chain, so the way to obtain the first image is the first method. In this embodiment, the image captured by the image acquisition module can be directly output by its corresponding driver, such as the camera driver shown in Figure 2, to the program installed on the electronic device. The image output by the camera driver will first be passed to the DMFT chain in the video capture stack. Therefore, the first DMFT module in the DMFT chain (such as DMFT module 0 shown in Figure 2) can directly obtain this image, use this image as the first image, and process it according to the method of this embodiment.

[0077] The method of this embodiment can also be executed by any DMFT module other than the first one in the DMFT chain, such as DMFT module 2 shown in Figure 2. In this case, the first image is obtained in the second way. The image output by the camera driver is first passed to the first DMFT module in the DMFT chain. After processing by the first DMFT module, the processed image is passed to the second DMFT module, and so on, until it is passed to the DMFT module that executes the image processing method of this embodiment. Then, this DMFT module takes the image processed by the previous DMFT module as the first image and processes it according to the method of this embodiment. For example, in Figure 2, the image output by the camera driver is first processed by DMFT module 0 and DMFT module 1 in sequence. DMFT module 2 obtains the image processed by DMFT module 1 as the first image.

[0078] Furthermore, the way the processing results are transmitted varies depending on the location of the image processing module.

[0079] If the image processing module in this embodiment is the last DMFT module in the DMFT chain, then the processing result in S103 can be directly transmitted by the image processing module to the application on the electronic device. Specifically, it can be transmitted to the application currently running on the electronic device, such as a video playback application currently running.

[0080] If the image processing module in this embodiment is not the last DMFT module in the DMFT chain, but is the first DMFT module, or any intermediate DMFT module between the first and the last, then the processing result in S103 can be transmitted by the image processing module to the next DMFT module in the DMFT chain, and the next DMFT module can continue to process the processing result. When all DMFT modules in the DMFT chain have finished processing, the last DMFT module transmits its output to the currently running application.

[0081] Optional, also includes:

[0082] Based on at least one of the configuration for the application and the configuration for the hardware module, at least one mechanism is selected as the target mechanism from a plurality of pre-registered mechanisms; the plurality of pre-registered mechanisms are mechanisms of at least one image processing module.

[0083] In this embodiment, the image processing module can pre-register multiple mechanisms. Each mechanism corresponds to an effect library pre-installed on the electronic device. An effect library can be understood as a specific image processing program. The effect library receives images provided by the image processing module, processes the images based on its own implemented processing logic, and returns the results to the image processing module. The image processing logic corresponding to each mechanism can be specifically implemented using the effect library corresponding to that mechanism.

[0084] Therefore, the image processing module processes the first image based on the target mechanism. This is equivalent to the image processing module calling the effect library corresponding to the target mechanism to process the first image and obtain the processing result. If there are multiple target mechanisms, multiple corresponding effect libraries can be called to process the first image. Referring to the example in Figure 2, if the target mechanisms are mechanism 1 and mechanism 2, with mechanism 1 corresponding to effect library 1 and mechanism 2 corresponding to effect library 2, then the image processing module calls effect library 1 and effect library 2 to process the first image and obtain the processing result.

[0085] The target mechanism for processing the first image in S102 may include a target mechanism determined for the configuration of the currently running application, a target mechanism determined for the configuration of the hardware module related to the image acquisition module, or both.

[0086] The image processing module can determine the target mechanism based on the configuration for the currently running application. The currently running application can include applications on the electronic device that are currently running in the foreground, occupying the screen of the electronic device, as well as the keyboard and mouse (or other hardware modules used for input). The currently running application can also include applications and / or application plugins that are currently running in the background of the electronic device.

[0087] For example, if a user is currently participating in an online meeting using an online meeting application on an electronic device, then the online meeting application is the currently running application. If the target mechanism for the online meeting application is configured as mechanism 1 in advance, then the image processing module determines that mechanism 1 is the target mechanism based on the above configuration, and processes the first image based on mechanism 1 in S102.

[0088] The image processing module can determine the target mechanism based on the configuration for the currently running application. The currently running application refers to the application in the electronic device that is currently running in the foreground and occupies the screen, keyboard, and mouse (or other hardware modules used for inputting information).

[0089] The image processing module can determine the target mechanism based on the configuration of the hardware modules, which can include every existing hardware module of the electronic device that is related to the image acquisition module. The hardware module being related to the image acquisition module means that when the operating system controls the operation of this hardware module, it can at least partially utilize the image acquired by the image acquisition module.

[0090] For example, the hardware modules related to the image acquisition module may include at least one of a microphone module, a speaker module, and a camera. The operating system can control the microphone module to turn on or off based on the user's gestures in the image acquired by the image acquisition module, and can also control the volume of the speaker module to increase or decrease based on whether there is a face in the image acquired by the image acquisition module.

[0091] Optionally, methods for configuring the target mechanism for an application or hardware module may include:

[0092] Display the configuration interface corresponding to the application or hardware module. The configuration interface includes different functional information used to describe the functions implemented by different mechanisms.

[0093] In response to an operation command for at least one functional information, the mechanism corresponding to the targeted functional information is configured to apply the corresponding target mechanism.

[0094] Referring to Figure 3, the image processing module in this embodiment can be DMFT module 2 in the DMFT chain. The DMFT chain can run in the basic conversion driver of the electronic device based on the media infrastructure framework. The basic conversion driver can include a core service module (which can be referred to as Core Service) for configuring the corresponding target mechanism for different applications and hardware modules.

[0095] The operating system of an electronic device may contain an application that can interact with the core service module and provide the above configuration interface, such as application 1 shown in Figure 2.

[0096] Through the application that provides the configuration interface, the electronic device can respond to the user's operation by displaying a configuration interface for any application that can call the image acquisition module, or displaying a configuration interface for any hardware module related to the image acquisition module.

[0097] The functional information included in the configuration interface depends on two factors. First, it depends on the functions implemented by the mechanisms registered on the image processing module. Specifically, the configuration interface displays only the corresponding functional information based on which mechanisms have been registered and what functions they can perform. Second, the functional information included in the configuration interface can also depend on the configured application or hardware module. In other words, the configuration interface may only include functional information related to the currently configured application or hardware module.

[0098] For example, when displaying the configuration interface for an online meeting application as shown in Figure 6, the online meeting application may need to call the voice caption function and the anti-peeping function, but will not call the gesture control function to control the microphone. Therefore, the configuration interface can only include the function information 601 representing the voice caption function and the function information 602 representing the anti-peeping function, as shown in Figure 6. When displaying the configuration interface for the microphone module as shown in Figure 7, the operating system will not control the microphone module based on the voice caption function and the anti-peeping function, but can turn the microphone module on and off based on the gesture control function. Therefore, the configuration interface can only include the function information 701 representing the gesture control function, as shown in Figure 7.

[0099] The configuration interface can contain configuration controls corresponding to the function information, such as control 6011 corresponding to function information 601 in Figure 6, control 6021 corresponding to function information 602, and control 7011 corresponding to function information 701 in Figure 7. Each control can be used to enable or disable the function of the corresponding function information.

[0100] An operation instruction for at least one functional information, such as an operation instruction for the control corresponding to the functional information, can determine whether the user needs to enable or disable the corresponding function in response to an operation instruction for any control. When it is determined that the user needs to enable a certain function, the core service module can configure the mechanism for implementing this function as the target mechanism of the application or hardware module currently being used.

[0101] Referring to the example in Figure 6, the current display shows the configuration interface for configuring the online conferencing application. The user clicks controls 6011 and 6021, making both controls in the "on" state shown in Figure 6. This determines that the voice captioning function and the anti-peeping function need to be enabled when running the online conferencing application. The core service module configures the mechanism for implementing the voice captioning function to the target mechanism corresponding to the online conferencing application, and configures the mechanism for implementing the anti-peeping function to the target mechanism corresponding to the online conferencing application.

[0102] The functions that each registered mechanism can perform can be obtained by the image processing module during the registration phase of this mechanism and recorded as registration information, for example, within the image processing module or the aforementioned core service module. Thus, when displaying the configuration interface, the electronic device can display function information based on the functions implemented by the registered mechanisms.

[0103] In short, by registering each mechanism, the system can record the correspondence between the mechanism and the implemented function. Through the configuration interface, it can determine which functions need to be enabled for each application and hardware module. Then, by combining the correspondence between the mechanism and the function, it can configure the target mechanism for each application and hardware module to implement the enabled function, so as to meet the functional requirements of each application and hardware module.

[0104] Optionally, the number of target mechanisms may be at least two;

[0105] The image processing module processes the first image based on the target mechanism to obtain the processing results, including:

[0106] The first image is processed based on the first objective mechanism out of at least two objective mechanisms to obtain intermediate results;

[0107] The intermediate results are processed based on the second objective mechanism out of at least two objective mechanisms to obtain the final result.

[0108] The above processing method can also be applied to two or more target mechanisms. For example, if there are three target mechanisms, the image processing module first processes the first image based on the first target mechanism, then processes the result returned by the first target mechanism based on the second target mechanism, then processes the result returned by the second target mechanism based on the third target mechanism, and finally obtains the processing result returned by the third target mechanism.

[0109] In other words, when there are two or more target mechanisms, the image processing module can process the first image one by one using each target mechanism in a serial calling manner until the processing result returned by the last target mechanism is obtained.

[0110] Referring to the example in Figure 3, assuming the target mechanism includes mechanism 1 and mechanism 2, mechanism 1 corresponds to effect library 1, and its corresponding image processing logic is implemented by effect library 1. Mechanism 2 corresponds to effect library 2, and its corresponding image processing logic is implemented by effect library 2. When the image processing module obtains the first image, it inputs the first image into effect library 1 through interface 1 corresponding to effect library 1. After processing, effect library 1 returns the result to the image processing module. Then, the image processing module inputs the result returned by effect library 1 and / or the first image into effect library 2 through interface 2 corresponding to effect library 2. After processing, effect library 2 returns the result to the image processing module. The image processing module outputs the result returned by effect library 2 as the processing result of S103.

[0111] The image processing module can process multiple consecutive frames of images according to the method of this embodiment, such as processing the first image and processing the next frame after the first image, i.e., the second image. In the case of processing multiple consecutive frames of images, parallel processing can be achieved between different images. For example, if the target mechanism includes mechanism 1 and mechanism 2, the first image can be processed based on mechanism 1. After mechanism 1 returns the result of the first image, the first image can be processed based on mechanism 2. While mechanism 2 is processing the first image, the second image can be processed based on mechanism 1. This can reduce the latency caused by processing multiple consecutive frames of images based on the target mechanism.

[0112] The interface corresponding to each effect library can be obtained by the image processing module during the registration phase of the corresponding mechanism and recorded in the image processing module as part of the registration information. The mechanism of any embodiment of this application can be understood as a collection of corresponding effect libraries and interfaces.

[0113] Referring to Figure 4, the registration process for the target mechanism may include:

[0114] Obtain the target registration token for the target mechanism;

[0115] Determine whether the target registration token matches at least one token in the registry, which includes tokens written to the application when an application that invokes the image acquisition module is installed;

[0116] If the target registration token matches at least one token in the registry, the target mechanism is confirmed to be a registered mechanism.

[0117] Any mechanism other than the target mechanism can be registered using the registration process described above, without further explanation.

[0118] The registration process described above can be performed by the image processing module or by the aforementioned core service module. The registration process for the target mechanism can be performed only once when the target mechanism is installed on the electronic device; there is no need to perform the registration process again as long as the target mechanism is not uninstalled from the electronic device.

[0119] After the target mechanism is installed on the electronic device, the target mechanism sends its own registration information to the image processing module. This registration information may include at least the target mechanism's own target registration token, which may be stored in the effect library corresponding to the target mechanism.

[0120] In addition, the registration information of the target mechanism may also include the functional information of the target mechanism. The functional information describes the functions that the target mechanism can perform. Thus, when the image processing module displays the configuration interface, it can display the functional information of the registered mechanism on the configuration interface so that the user can configure it as needed according to the aforementioned configuration method.

[0121] The registration information of the target mechanism may also include information about the target mechanism's interface, including but not limited to the interface specification and interface name, so that after the registration process is completed, the image processing module can access the corresponding interface based on this information and interact with the effect library corresponding to the target mechanism through the interface.

[0122] After receiving the registration information through its own registration interface, the image processing module extracts the target registration token and then checks whether the token recorded in the registry matches the target registration token.

[0123] If at least one token in the registry matches this target registration token, for example, if there is a token identical to this target registration token, then it can be confirmed that the target mechanism is a secure image processing mechanism that conforms to the relevant specifications, and the target mechanism can be allowed to register. Thus, the image processing module can confirm that the target mechanism is a registered mechanism, load the registration information provided by the target mechanism into the image processing module and record it in the relevant table or data structure so that the target mechanism can be called to process the image when needed based on this registration information.

[0124] If every token in the registry does not match the target registration token, for example, if every token is different from the target registration token, it can be confirmed that the target mechanism does not comply with the relevant specifications. In this case, the registration of the target mechanism can be rejected, the registration information of the target mechanism can be discarded to prevent the target mechanism from being called in subsequent runtime, and an installation failure message can be output to indicate that the target mechanism installation failed.

[0125] The registry may contain one or more pre-recorded tokens. These tokens can be written by any one or more programs in the operating system of the electronic device during the installation phase of the corresponding program. These programs can be applications that users can use directly, or drivers located in the driver layer that drive hardware modules.

[0126] Any program that needs to access the image processing module can apply to the relevant manufacturer of the electronic device in advance. Once the application is approved, the manufacturer can issue a unique token to the program that indicates permission to access the image processing module. This token can be written into the program by the developer. When the program that needs to access the image processing module is installed on the electronic device, the program provides this token to the operating system of the electronic device, and the operating system records this token in the aforementioned registry.

[0127] In this way, the image processing module can confirm whether the target mechanism complies with the relevant specifications. Only after confirming compliance with the specifications will the target mechanism be allowed to register and subsequently called. If it is confirmed that the target mechanism does not comply with the specifications, registration and subsequent calls will not be allowed. This improves the security of electronic devices.

[0128] Based on the above registration mechanism, the image processing module can further implement an authorization mechanism when processing the first image based on the target mechanism, so as to prevent the image from being processed based on an unregistered mechanism and avoid potential data leakage or system failure risks.

[0129] As shown in Figure 4, the image processing module may include an authorization interface. Before processing the first image based on the target mechanism, the image processing module can use this authorization interface to query whether there is registration information for the target mechanism in the previously recorded registration information. If there is registration information for the target mechanism, it is confirmed that the target mechanism is a registered mechanism, and processing of the first image based on the target mechanism is allowed. If there is no registration information for the target mechanism, it is confirmed that the target mechanism is an unregistered mechanism, and processing of the first image based on the target mechanism is not allowed.

[0130] In some optional embodiments, the image processing module or the aforementioned core service module may also pre-configure the working parameters of the target mechanism. Based on this, when the image processing module processes the first image based on the target mechanism, it may combine the working parameters of the target mechanism to process the first image.

[0131] As an example, the working parameter of the target mechanism can be the processing response time of the target mechanism.

[0132] When the processing response time is configured, when the image processing module processes the first image based on the target mechanism, it can wait for the target mechanism to return the processing result. If the processing time of the target mechanism is less than or equal to the pre-configured processing response time, that is, if the processing result is fed back to the image processing module within the processing response time, the image processing module executes S103 to transmit the processing result to other image processing modules or applications.

[0133] Optionally, if the processing time of the first image based on the target mechanism is greater than the processing response time of the target mechanism, the first image is discarded and the second image is processed based on the target mechanism. The second image is the next frame image from the first image in the image acquisition module.

[0134] The processing response time of the target mechanism is determined when registering the target mechanism, based on the processing capability of the target mechanism and the maximum response time preset by the electronic device.

[0135] In this embodiment, after the image processing module calls the target mechanism to process the first image, it starts waiting for the target mechanism to return the processing result and keeps track of the waiting time. If the waiting time exceeds the processing response time of the target mechanism, it can be confirmed that the processing time of the first image based on the target mechanism is longer than the processing response time of the target mechanism. At this time, it can be confirmed that the first image has been lost. Instead of waiting for the target mechanism to return the processing result corresponding to the first image, it continues to process the next frame image from the image acquisition module, that is, continues to process the second image based on the target mechanism.

[0136] Meanwhile, the image processing module can report the message that the first image has been lost to other subsequent image processing modules or to the application, so that other image processing modules or the application no longer have to wait for the processing result corresponding to the first image, and can continue the subsequent processing flow.

[0137] By setting the above processing response time, delays or stutters can be avoided when the image processing module processes the first image based on the target mechanism due to excessive processing time, thus improving the smoothness of the image processing module and subsequent programs.

[0138] The following example illustrates how to configure the processing response time of any mechanism.

[0139] The core service module has a default maximum response time pre-set. The maximum response time can be a factory-preset default parameter and / or can be configured by the user during the use of the electronic device.

[0140] During the registration process of the target mechanism, the registration information sent by the target mechanism can carry the registration capability of the target mechanism. The registration capability of the target mechanism can be, for example, the response time range of the target mechanism. The upper limit of the response time range indicates how long the target mechanism needs to process one frame of image at most, such as 50 milliseconds, and the lower limit of the response time range indicates how long the target mechanism needs to process one frame of image at least, such as 30 milliseconds.

[0141] The response time range of the target mechanism is determined by the target mechanism based on its own image processing logic and the performance of the hardware platform on which the target mechanism is currently running.

[0142] The core service module can determine a minimum value (e.g., 30 milliseconds) within the response time range that is less than the maximum response time, and feed this minimum value back to the target mechanism for confirmation. If the target mechanism confirms that it can feed back the processing result according to this minimum value, the core service module can configure this minimum value as the processing response time of the target mechanism.

[0143] If the target mechanism confirms that it cannot return the processing result according to this minimum value, the target mechanism can also return a value less than the maximum response time (e.g., 40 milliseconds) to the core service module. The core service module can configure this value returned by the target mechanism as the processing response time of the target mechanism.

[0144] In summary, the core service module can configure the processing response time for the target mechanism during the registration process by negotiating with the target mechanism as described above.

[0145] Optionally, the processing response time of the target mechanism can be configured during registration and dynamically adjusted during the image processing module's invocation of the target mechanism. The core service module can statistically analyze the proportion of images lost during recent image processing based on the target mechanism. If this proportion is too high, the processing response time can be appropriately increased to reduce the number of lost images.

[0146] The above content regarding the target mechanism is only an example. The method of configuring the working parameters of the target mechanism and calling the target mechanism based on the working parameters can be applied to any registered mechanism on an electronic device, and will not be elaborated further.

[0147] Optionally, depending on the implemented function, the method of processing the first image based on the target mechanism can be different, and the processing results obtained can also be different. As some examples, the processing results obtained by processing the first image based on the target mechanism can include at least one of the following:

[0148] When implementing the voice caption function, the processing method can be to add the target text to the first image, obtain the image containing the target text as the processing result, and obtain the target text based on the speech recognition collected by the audio acquisition module;

[0149] In order to achieve the anti-peeping function, the processing method can be to reduce the clarity of the first image and obtain a blurry image with a lower clarity than the first image as the processing result;

[0150] When implementing gesture control functionality, the processing method can be to obtain control instructions related to the image recognition result of the first image, use the control instructions as the processing result, and use the control instructions to enable or disable the hardware modules related to the image acquisition module.

[0151] In the voice caption function, the target mechanism can obtain the first image and the target text output by the speech recognition module of the electronic device. The target text is obtained by the speech collected by the audio acquisition module (such as a microphone) by the speech recognition module. Thus, the target mechanism can synthesize the first image and the target text into an image containing the target text, and feed the image containing the target text back to the image processing module. The image processing module outputs the image containing the target text as the processing result.

[0152] In the anti-spying function, if the target mechanism detects someone in a specific area based on the first image—for example, detecting someone a certain distance behind the current user using the electronic device—the first image can be blurred to reduce its clarity, resulting in a blurred image. This blurred image is then fed back to the image processing module, which transmits it as the processing result to other image processing modules or applications. Optionally, the target mechanism can also provide the detection result of whether someone is in the specific area, and the image processing module can also output this detection result as a processing result.

[0153] In the gesture control function, the hardware module can be, for example, a microphone. If the target mechanism recognizes that the first image contains a gesture for turning on the microphone, the target mechanism sends a control command to the image processing module to turn on the microphone. The image processing module then sends this control command as a processing result back to the application that calls the microphone, thereby turning on the microphone.

[0154] The following examples illustrate the application scenarios of the method in this embodiment.

[0155] Example 1: An electronic device is currently running an online meeting application. The target mechanism pre-configured for the online meeting application is Mechanism 1. The image processing logic corresponding to Mechanism 1 is used to implement the real-time translation function. When the first image is obtained, the image processing module processes the first image based on Mechanism 1. Mechanism 1 combines the translated text provided by the electronic device's speech recognition and translation module with the first image into an image containing the translated text. The image processing module outputs the image containing the translated text fed back by Mechanism 1 as the processing result to the online meeting application. The online meeting application displays the image containing the translated text, thereby realizing the real-time translation function.

[0156] Example 2: The electronic device is currently running a view application. The target mechanisms pre-configured for the view application include mechanism 2 and mechanism 3. The image processing logic corresponding to mechanism 2 is used to implement the image quality enhancement function, and the image processing logic corresponding to mechanism 3 is used to implement the posture reminder function.

[0157] Upon obtaining the first image, the image processing module first processes the first image based on mechanism 2. Mechanism 2 adjusts the brightness, intensity, color, and other parameters of the first image according to the ambient brightness and performs noise reduction processing on the first image to obtain an enhanced first image. The enhanced first image is then fed back to the image processing module, which calls mechanism 3 based on the enhanced first image. Mechanism 3 processes the enhanced first image, identifies whether the current user's current posture in the enhanced first image is a correct sitting posture, and identifies whether the current user has been staring at the screen for a long time. If it is not a correct sitting posture, a posture prompt is added to the enhanced first image; if the user has been staring at the screen for a long time, an eye health prompt is added to the enhanced first image. Then, the enhanced first image with the added prompts is fed back to the image processing module, which feeds back the enhanced first image with the added prompts to the view application. The view application outputs the enhanced first image with the added prompts.

[0158] Example 3: The electronic device is currently running an AI assistant application. The target mechanism pre-configured for the AI ​​assistant application includes mechanism 4. The image processing logic corresponding to mechanism 4 is used to implement the anti-peeping function.

[0159] When the first image is obtained, the image processing module processes the first image based on mechanism 4. If mechanism 4 identifies a person in a specific area based on the first image, mechanism 4 blurs the first image to obtain a blurred image. The blurred image and the identification result of a person in a specific area are fed back to the image processing module. The image processing module outputs the blurred image and the identification result as the processing result to the artificial intelligence assistant application. The artificial intelligence assistant application can display this blurred image and / or add a blur effect to the currently displayed content on the screen of the electronic device based on this identification result, thereby preventing the currently displayed content from being spied on.

[0160] And / or, mechanism 4 can also add an anti-peeping warning message to the first image when there are people in a specific area, and feed the first image with the anti-peeping warning message back to the image processing module. The image processing module outputs the first image with the anti-peeping warning message as the processing result to the AI ​​assistant application, and the AI ​​assistant application displays the first image with the anti-peeping warning message.

[0161] This embodiment provides an electronic device, as shown in FIG5. The electronic device includes an image acquisition module 501, at least one image processing module 502, a memory 503, and at least one processor 504.

[0162] The image acquisition module 501 may be, for example, a camera of an electronic device, and at least one image processing module 502 is a software module stored in memory 503 and loaded by processor 504 for use on processor 504.

[0163] Memory 503 is used to store at least one image processing module 502;

[0164] Processor 504 is used to load at least one image processing module 502 to perform:

[0165] The first image is obtained; the first image comes from the image acquisition module.

[0166] The first image is processed based on the target mechanism to obtain the processing result; the target mechanism is either a mechanism configured for the application that calls the image acquisition module, or a mechanism configured for the hardware module related to the image acquisition module; different mechanisms correspond to different image processing logic;

[0167] The processing results are then transmitted to the next image processing module or application.

[0168] Optionally, at least one image processing module includes a registration interface and an authorization interface;

[0169] The registration interface is used to confirm that the target mechanism is a registered mechanism based on the target registration token of the target mechanism;

[0170] The authorization interface is used to confirm that the target mechanism has been registered, so as to process the first image based on the registered target mechanism and obtain the processing result.

[0171] The working principle of the above electronic devices can be found in the relevant steps of the image processing method in the foregoing embodiments, and will not be repeated here.

[0172] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0173] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.

[0174] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0175] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0176] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image processing method, applied to at least one image processing module, comprising: Obtain the first image; The first image comes from the image acquisition module; The first image is processed based on the target mechanism to obtain the processing result; The target mechanism is either a mechanism configured for the application that calls the image acquisition module, or a mechanism configured for the hardware module related to the image acquisition module; different mechanisms correspond to different image processing logic; the processing result is transmitted to the next image processing module or the application.

2. The method according to claim 1, further comprising: Based on at least one of the configuration for the application and the configuration for the hardware module, at least one mechanism is determined as the target mechanism from a plurality of pre-registered mechanisms; The pre-registered mechanisms are the mechanisms of the at least one image processing module.

3. The method according to claim 1, wherein the number of target mechanisms is at least two; the step of processing the first image based on the target mechanisms to obtain the processing result includes: The first image is processed based on the first objective mechanism among at least two of the objective mechanisms to obtain an intermediate result; The intermediate result is processed based on the second objective mechanism of at least two objective mechanisms to obtain the processing result.

4. The method according to claim 1, wherein configuring the target mechanism for the application or hardware module includes: Display the configuration interface corresponding to the application or the hardware module, the configuration interface including different functional information for describing the functions implemented by different mechanisms; In response to an operation instruction for at least one functional information, the mechanism corresponding to the targeted functional information is configured as the target mechanism corresponding to the application.

5. The method according to claim 2, wherein the registration process of the target mechanism includes: Obtain the target registration token of the target mechanism; Determine whether the target registration token matches at least one token in the registry, the registry including tokens written to the application when an application that invokes the image acquisition module is installed; If the target registration token matches at least one token in the registry, the target mechanism is confirmed to be a registered mechanism.

6. The method according to claim 2, further comprising: If the processing time of the first image based on the target mechanism is greater than the processing response time of the target mechanism, the first image is discarded and the second image is processed based on the target mechanism. The second image is the next frame image from the first image obtained by the image acquisition module. The processing response time of the target mechanism is determined based on the processing capability of the target mechanism and the maximum response time preset by the electronic device when the target mechanism is registered.

7. The method according to claim 1, wherein obtaining the first image comprises at least one of the following: obtaining an image output by the image acquisition module as the first image; obtaining an image processed by another image processing module different from the at least one image processing module as the first image.

8. The method according to claim 1, wherein processing the first image based on the target mechanism to obtain a processing result includes at least one of the following: adding target text to the first image to obtain an image containing the target text as a processing result, wherein the target text is obtained based on speech recognition acquired by the audio acquisition module; reducing the clarity of the first image to obtain a blurred image with a clarity less than that of the first image as a processing result; obtaining a control instruction related to the image recognition result of the first image, wherein the control instruction is used as a processing result, and the control instruction is used to control the hardware module related to the image acquisition module to be enabled or disabled.

9. An electronic device comprising an image acquisition module, at least one image processing module, a memory, and at least one processor; the memory is used to store at least one image processing module; the processor is used to load at least one image processing module to perform: acquiring a first image; the first image being from the image acquisition module; processing the first image based on a target mechanism to obtain a processing result; the target mechanism being a mechanism configured for an application that calls the image acquisition module, or a mechanism configured for a hardware module related to the image acquisition module; different mechanisms correspond to different image processing logic; and transmitting the processing result to the next image processing module or the application.

10. The device according to claim 9, wherein the at least one image processing module includes a registration interface and an authorization interface; the registration interface is used to confirm that the target mechanism is a registered mechanism based on the target mechanism's target registration token; the authorization interface is used to confirm that the target mechanism is registered, and to process the first image based on the registered target mechanism to obtain a processing result.

Images