Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an image processing method and device, electronic equipment and storage medium, the method comprising: in the case of receiving a processing task of a to-be-processed image, determining a target function unit for executing the processing task according to the type of the processing task and the type of the function unit; performing feature extraction on the to-be-processed image through the target function unit to obtain first features; caching the first features to a first message queue; reading the first features from the first message queue through a post-processing unit, and obtaining an image processing result according to the first features. According to the image processing method of the embodiment of the present disclosure, by selecting the target function unit for executing the processing task and the first message queue that can cache features, the resource requirement of task processing is reduced, so that the architecture of distributed processing can be reused for edge computing platforms, and the maintenance cost is reduced.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer vision, and particularly relates to an image processing method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the popularity of 5G and the Internet of Things, systems closer to data sources are needed to process data in real time, improve user experience, and reduce network construction costs. Edge computing can be applied to such scenarios. However, in the related art, the cloud and the edge are two independent systems maintained separately, resulting in high development and maintenance costs. Moreover, the hardware configuration of the edge side system is generally lower than that of the cloud, and therefore, the architecture of the system is difficult to reuse. SUMMARY

[0003] The present disclosure provides an image processing method and device, an electronic device and a storage medium.

[0004] According to an aspect of the present disclosure, an image processing method is provided, including: in a case where a processing task for a to-be-processed image is received, determining a target functional unit for executing the processing task from a plurality of functional units according to a type of the processing task and a type of the plurality of functional units of an edge computing platform; performing feature extraction processing on the to-be-processed image by the target functional unit to obtain first features corresponding to the processing task; caching the first features to a first message queue; and reading the first features from the first message queue by a post-processing unit and obtaining an image processing result according to the first features.

[0005] The image processing method according to the embodiment of the present disclosure reduces the resource requirement of task processing by selecting a target functional unit for executing a processing task and a first message queue in which features can be cached, so that the architecture of distributed processing can be reused for the edge computing platform, thereby reducing the maintenance cost.

[0006] In a possible implementation, the target functional unit for executing the processing task is determined from the plurality of functional units according to the type of the processing task and the type of the plurality of functional units, including: determining the target functional unit according to the type of the processing task, the type of the plurality of functional units, a resource occupation status of the plurality of functional units, and a progress of the processing task.

[0007] In this way, a functional unit that matches the type of the processing task can be selected, and the processing efficiency can be improved.

[0008] In a possible implementation, the method further includes: obtaining reference features by the post-processing unit; and storing the reference features to a database of the edge computing platform.

[0009] In a possible implementation, the first feature is read from the first message queue by the post-processing unit, and an image processing result is obtained according to the first feature, including: performing matching processing on a plurality of reference features in the database according to the first feature, to obtain a matching result; and obtaining the image processing result according to the matching result.

[0010] In a possible implementation, the method further includes: obtaining, by the post-processing unit, an operation instruction for a reference feature in the database, and performing corresponding processing on the reference feature according to the operation instruction, where the operation instruction includes a creation instruction, a reading instruction, a replacement instruction, and a deletion instruction.

[0011] In a possible implementation, the method further includes: caching, by the post-processing unit, the image processing result to a second message queue; and outputting, by the second message queue, the image processing result.

[0012] In a possible implementation, the target functional unit determined from the plurality of functional units to execute the processing task is implemented by a service agent component and a key-value storage component, and the first message queue is implemented by an open source message agent component.

[0013] According to an aspect of the present disclosure, an image processing apparatus is provided, including: a functional unit selection module configured to, in a case where a processing task for a to-be-processed image is received, determine a target functional unit for executing the processing task from a plurality of functional units of an edge computing platform according to a type of the processing task and types of the plurality of functional units; a feature extraction module configured to perform feature extraction processing on the to-be-processed image by the target functional unit, to obtain a first feature corresponding to the processing task; a caching module configured to cache the first feature to a first message queue; and an image processing module configured to read the first feature from the first message queue by a post-processing unit, and obtain an image processing result according to the first feature.

[0014] In a possible implementation, the functional unit selection module is further configured to determine the target functional unit according to the type of the processing task, the types of the plurality of functional units, resource occupation statuses of the plurality of functional units, and a progress of the processing task.

[0015] In a possible implementation, the apparatus further includes: a reference feature acquisition module configured to acquire a reference feature by the post-processing unit; and store the reference feature to a database of the edge computing platform.

[0016] In a possible implementation, the image processing module is further configured to: perform matching processing on the plurality of reference features in the database according to the first feature, to obtain a matching result; and obtain the image processing result according to the matching result.

[0017] In a possible implementation, the apparatus further includes an operation module configured to acquire, by the post-processing unit, an operation instruction for a reference feature in the database, and perform corresponding processing on the reference feature according to the operation instruction, wherein the operation instruction includes a creation instruction, a reading instruction, a replacement instruction, and a deletion instruction.

[0018] In a possible implementation, the apparatus further includes a result output module configured to cache, by the post-processing unit, the image processing result to a second message queue, and output the image processing result through the second message queue.

[0019] In a possible implementation, the determination of the target functional unit for performing the processing task from the plurality of functional units is implemented by a service agent component and a key-value storage component, and the first message queue is implemented by an open source message agent component.

[0020] According to an aspect of the present disclosure, an electronic device is provided, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the above method.

[0021] According to an aspect of the present disclosure, a computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the above method.

[0022] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments consistent with the present disclosure and serve to explain the technical solutions of the present disclosure together with the specification.

[0024] Figure 1 A flow chart of an image processing method according to an embodiment of the present disclosure is shown;

[0025] Figure 2 An application schematic diagram of an image processing method according to an embodiment of the present disclosure is shown;

[0026] Figure 3A block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown.

[0027] Figure 4 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0028] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Various exemplary embodiments, features and aspects of the present disclosure will be explained in detail below with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar elements. Although various aspects of the embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0030] The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0031] The term "and / or" used herein only means an association relationship of the associated objects, and means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0032] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. Those skilled in the art should understand that the present disclosure can also be implemented without some specific details. In some examples, methods, means, elements and circuits that are well known to those skilled in the art are not described in detail, in order to highlight the main idea of the present disclosure.

[0033] Figure 1 A flowchart of an image processing method according to an embodiment of the present disclosure is shown, as shown in Figure 1 The method comprises:

[0034] Step S11, in the case of receiving a processing task of a to-be-processed image, according to the type of the processing task and the type of a plurality of functional units, determining a target functional unit for executing the processing task in the plurality of functional units;

[0035] Step S12, performing feature extraction processing on the to-be-processed image by the target functional unit to obtain a first feature corresponding to the processing task;

[0036] Step S13, cache the first feature to a first message queue;

[0037] Step S14, read the first feature from the first message queue by a post-processing unit, and obtain an image processing result according to the first feature.

[0038] In a possible implementation, the image processing method can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like. The method can be implemented by a processor invoking computer-readable instructions stored in a memory. Alternatively, the method can be executed by a server.

[0039] According to the image processing method of the embodiments of the present disclosure, by selecting a target function unit for executing a processing task and a first message queue of cacheable features, the resource requirement of task processing is reduced, so that the architecture of distributed processing can be reused for an edge computing platform, and the maintenance cost is reduced.

[0040] In a possible implementation, the distributed processing system processing architecture in the related art is mostly implemented based on JVM (Java Virtual Machine). Since the data volume processed is large, the requirement for hardware configuration is high, for example, the memory requirement is high, and therefore, a hardware device with strong processing capability is needed to support the operation. In an example, the Java-based program language is widely applied to the development of a server (for example, a cloud), the open source library is rich, and it is easy to build an application system, but the performance is poor. The Java source code needs to be compiled into bytecode, and then compiled into machine code by a virtual machine. Therefore, when the Java program is started, JVM, JIT (Just In Time), and the like need to be loaded, which all need to consume a large amount of memory additionally, and therefore, the requirement for hardware configuration is high.

[0041] However, the hardware configuration of the edge side device is low since the data volume to be processed is small, for example, the memory of some edge computing platforms, that is, edge side devices is only about 8 GB. The hardware configuration of these edge side devices is difficult to reuse the architecture of the distributed processing system.

[0042] In a possible implementation, to solve the above problems, the disclosure reduces the demand for hardware configuration by lightening the architecture of the distributed processing system in the edge computing platform, so that the edge computing platform can reuse the architecture of the distributed processing system to reduce the maintenance cost of the system. The edge computing platform can include various devices with processing functions, such as personal computers, mobile devices, smart homes, etc., and the disclosure does not limit the type of edge computing platform.

[0043] In a possible implementation, in step S11, a task management unit can be arranged in the edge computing platform to receive tasks and schedule other units in the edge computing platform to execute the tasks. The image to be processed can be an image captured by a camera in a city, an image captured by an access control device, etc., and the disclosure does not limit the type of image to be processed. The processing task of the image to be processed can include various tasks, such as face recognition, liveness detection, key point detection, etc., and the disclosure does not limit the processing task.

[0044] In a possible implementation, the function unit is a unit capable of performing a certain specific processing. The function unit can be an entity unit in the edge computing platform, such as an entity unit including one or more electronic components, or a virtual function unit, such as a specific algorithm unit capable of performing a certain processing, and the disclosure does not limit the type of function unit.

[0045] In a possible implementation, when the processing task of the image to be processed is received, a function unit matching the type of the processing task can be selected. For example, the processing task is a face recognition task, and a unit having the function of extracting face features can be selected. For another example, the processing task is a key point detection task, and a unit having the function of extracting key point features can be selected. For another example, the processing task is a liveness detection task, and a unit having the function of extracting depth features can be selected, and the disclosure does not limit the function of the function unit.

[0046] In a possible implementation, in addition to the matching of the function and the type, the resource occupation status of the function unit and the progress of the processing task can also be referred to when the function unit is selected. Step S11 can include: determining the target function unit according to the type of the processing task, the types of the plurality of function units, the resource occupation status of the plurality of function units, and the progress of the processing task.

[0047] In an example, if there are multiple function units matching the type of the processing task, the resource occupation status of the function units and the progress of the processing task can also be referred to. For example, a certain task requires 100 MB of memory, the progress of the task is currently 40%, and it can be estimated that 60 MB of memory is still required to process the remaining task. Therefore, the function unit can be selected based on the resource occupation status of the function unit, for example, the function unit with a memory quantity that can be invoked exceeding the memory quantity required by the task and a higher resource occupation ratio can be selected, so that the function unit is concentrated in execution, other function units can be used to execute other tasks, and the existing task does not occupy too many function units, so as to improve the overall processing efficiency of the edge computing platform, i.e., the edge device.

[0048] In an example, the function unit with a lower resource occupation ratio can also be selected, for example, if there are multiple function units matching the type of the task, the function unit with the lowest resource occupation ratio can be used to process the task, so as to improve the execution efficiency of the current task.

[0049] In an example, the processing resource that can be invoked by the function unit can also be referred to when selecting the function unit, for example, a certain task requires the joint execution of CPU and GPU, and the function unit that can invoke both kinds of processing resources can be selected. The specific selection manner of the function unit is not limited in the present disclosure.

[0050] In a possible implementation, determining the target function unit for executing the processing task in the plurality of function units is implemented by a service proxy component (zetcd) and a key-value storage component (etcd). In an example, the above two components can replace the Zookeeper component in the distributed architecture of the cloud to provide distributed configuration and synchronization functions, so as to manage the processing task and respectively mobilize one or more target function units to execute the processing task. The Zookeeper component is usually implemented by a Java program, and has a higher demand for hardware configuration. Using the service proxy component and the key-value storage component to replace the Zookeeper component can realize the lightweight of the distributed configuration and synchronization functions, for example, the service proxy component and the key-value storage component can be implemented by Go, c / c++, and other program languages. These program languages belong to static compilation languages, and can be directly compiled into machine code without the step of bytecode translation. Therefore, JVM, JIT, and the like do not need to be loaded, the memory demand is reduced, the demand for hardware configuration is also reduced, and the edge computing platform is more suitable. The specific type of the component providing the above functions is not limited in the present disclosure.

[0051] In this way, the function unit matching the type of the processing task can be selected, and the processing efficiency can be improved.

[0052] In a possible implementation, in step S12, the target function unit selected can perform feature extraction on the image to be processed to obtain first features corresponding to the processing task. As described above, different processing tasks can require different feature information. For example, a face recognition task can require multi-dimensional feature information of a face, a key point detection task can require feature information (e.g., position information, distance information between each other, etc.) of key points on a face, and a live detection task can require depth features. The present disclosure does not limit the type of first features corresponding to the task.

[0053] In a possible implementation, in step S13, the first features obtained by the target function unit can be cached to the first message queue, so that the post-processing unit can read the first features from the first message queue for further processing.

[0054] In a possible implementation, caching the first features to the first message queue can further reduce the requirement for hardware configuration. For example, the post-processing unit does not need to process features obtained by multiple target function units at the same time, but can read the features in the first message queue one by one or in batches according to the processing capability of the post-processing unit, so that a higher hardware configuration can not be required. For another example, the post-processing unit does not need to process the features immediately after the target function unit obtains the features. If the post-processing unit has not finished processing the features currently being processed, the post-processing unit can not continue to read the features, and the feature extraction processing of the target function unit for other tasks will not be affected. That is, the post-processing unit can read the features in the first message queue one by one or in batches according to the processing capability of the post-processing unit, and the progress of the feature extraction processing will not be affected.

[0055] In an example, the first message queue is implemented by using an open source message broker component (mosquitto), which can be used to replace the kafka message queue in the distributed architecture of the cloud. Similar to the service broker component and the key-value storage component, the open source message broker component can reduce the requirement for memory, that is, reduce the requirement for hardware configuration, and can meet the caching requirement in the edge computing platform. The present disclosure does not limit the component used by the first message queue.

[0056] In a possible implementation, in step S14, the post-processing unit can read the cached first feature from the first message queue based on its own processing progress, and perform further processing on the first feature to obtain the image processing result. For example, the processing task is a face detection task, and the post-processing unit can perform further detection and recognition processing on the first feature to obtain the position of the face in the to-be-processed image as the image processing result. For another example, the processing task is a face recognition task, and the post-processing unit can compare the first feature with a reference feature used for comparison to determine a matched reference feature, and further recognize the identity information of the face in the to-be-processed image as the image processing result. The type of the image processing result is not limited in the present disclosure.

[0057] In a possible implementation, if the processing task requires comparison between the first feature and a reference feature, the edge computing platform can further obtain the reference feature. The method further includes: obtaining, by the post-processing unit, the reference feature; and storing the reference feature to a database of the edge computing platform.

[0058] In an example, the database can replace a Cassandra database in a distributed architecture of a cloud by using a MySQL database. The MySQL database is more lightweight than the Cassandra database, and is suitable for the data processing amount of the edge computing platform.

[0059] In a possible implementation, the database can be deployed in the edge computing platform, or can be invoked by the edge computing platform, for example, the database can be invoked by a feature retrieval service (a component with retrieval function, for example, an application with retrieval function) to query the reference feature in the database. The use manner of the database is not limited in the present disclosure.

[0060] In a possible implementation, the method further includes: obtaining, by the post-processing unit, an operation instruction for the reference feature in the database, and performing corresponding processing on the reference feature according to the operation instruction, wherein the operation instruction includes a creation instruction, a reading instruction, a replacement instruction, and a deletion instruction.

[0061] In an example, the reference feature can be processed by the operation instruction, for example, a new reference feature is created (for example, a new reference feature is added), the reference feature is read, one or more reference features are replaced, or one or more reference features are deleted, and the processing of the reference feature is not limited in the present disclosure.

[0062] In a possible implementation, the database for storing the reference features can be obtained through the above processing, and is deployed in the edge computing platform or can be invoked by the edge computing platform. In step S14, the post-processing unit can compare the read first features with the reference features in the database to obtain the image processing result. Step S14 can include: performing matching processing on the plurality of reference features in the database according to the first features to obtain a matching result; and obtaining the image processing result according to the matching result.

[0063] In an example, the post-processing unit can read the first features from the first message queue, and perform feature retrieval in the database through the first features, that is, perform feature comparison one by one in the database through the first features, for example, determine the feature similarity (for example, cosine similarity) between the reference features and the first features one by one. In an example, if the database is deployed in the edge computing platform, the reference features can be directly invoked for comparison with the first features, if the database is not deployed in the edge computing platform, but needs a feature retrieval service to invoke, the post-processing unit can invoke the reference features for comparison with the first features through the feature retrieval service, or obtain the first features through the feature retrieval service and compare the first features with the reference features in the database. The present disclosure does not limit this.

[0064] In an example, through the above comparison, a matching result can be determined, for example, a reference feature matched with the first feature. And the image processing result can be obtained based on the matching result. For example, the identity information of the face in the image to be processed can be determined based on the identity information corresponding to the reference feature matched with the first feature as the image processing result. The present disclosure does not limit the type of image processing result.

[0065] In a possible implementation, the edge computing platform can output the image processing result, for example, to a display device, an audio device, etc., to display or play the image processing result, so that the user obtains the image processing result. The method further includes: caching, by the post-processing unit, the image processing result to a second message queue; and outputting, by the second message queue, the image processing result.

[0066] In an example, the second message queue can be the same as the first message queue, for example, the second message queue can also be implemented through an open source message broker component to reduce the demand for memory. The present disclosure does not limit the component used by the second message queue.

[0067] In the example, the post-processing component can cache the image processing results in a second message queue, which can transmit the image processing results in the message queue to the device for output to the user according to the output frequency of the device. For example, if the display device can display the image processing results of one image to be processed every 10 seconds, then the second message queue can transmit one image processing result to the display device every 10 seconds. This disclosure does not limit the output frequency.

[0068] The image processing method according to the embodiments of this disclosure reduces the resource requirements for task processing by selecting a target functional unit for performing processing tasks and a first message queue for cacheable features, making the distributed processing architecture more lightweight and reusable to edge computing platforms. It also eliminates the need to maintain multiple different systems during maintenance, thus reducing maintenance costs.

[0069] Figure 2 The diagram illustrates an application of the image processing method according to an embodiment of the present disclosure, such as... Figure 2 As shown, to reuse the distributed processing architecture for edge computing platforms, lightweight components can be used to perform image processing tasks on the edge computing platform.

[0070] In one possible implementation, such as Figure 2 As shown, the task management unit can receive image processing tasks and select a suitable functional unit to perform the above tasks through a lightweight service proxy component (zetcd) and a key-value storage component (etcd). For example, one or more target functional units can be selected from functional unit 1 to functional unit n (n is a positive integer greater than 1) to perform the above processing tasks based on the type of task, the process of the task, the type of functional unit, and the resource consumption of the functional unit.

[0071] In one possible implementation, the target functional unit can perform feature extraction processing on the image to obtain the features required by the processing task. For example, if the processing task is a face recognition task, the image features of the face can be obtained and cached in the open-source message broker component (mosquitto).

[0072] In one possible implementation, the post-processing unit can read cached features from the open-source message broker component based on its own processing frequency and perform post-processing. In the example, the post-processing unit can call a feature retrieval service to retrieve matching reference features from the database. For example, it can determine the feature similarity between each reference feature and the features of the aforementioned image, and determine the reference feature with the highest similarity as the matching reference feature. Then, the identity information corresponding to the matching reference feature is used as the image processing result.

[0073] In a possible implementation, the post-processing unit can further cache the image processing result to another open source message broker component, so that the display device reads the image processing result and displays it to the user.

[0074] In a possible implementation, the image processing method can be used in processing of video, image and other data in scenarios such as smart cities, and can use a distributed processing architecture in the cloud to reduce the maintenance cost of the processing architecture. The application field of the image processing method is not limited in the present disclosure.

[0075] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form combined embodiments without deviating from the principle logic. Limited by the length of the present disclosure, the present disclosure will not be described again. It can be understood by those skilled in the art that the specific execution order of each step in the above-mentioned method should be determined according to its function and possible internal logic.

[0076] In addition, the present disclosure also provides an image processing apparatus, an electronic device, a computer readable storage medium, and a program, all of which can be used to implement any one of the image processing methods provided by the present disclosure. The corresponding technical solutions and descriptions are described in the method part and are not described again.

[0077] Figure 3 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown as follows. Figure 3 As shown, the apparatus includes: a function unit selection module 11, configured to, in a case where a processing task of a to-be-processed image is received, determine a target function unit for executing the processing task from a plurality of function units of an edge computing platform according to a type of the processing task and a type of the plurality of function units; a feature extraction module 12, configured to perform feature extraction processing on the to-be-processed image by the target function unit to obtain a first feature corresponding to the processing task; a cache module 13, configured to cache the first feature to a first message queue; and an image processing module 14, configured to read the first feature from the first message queue by a post-processing unit, and obtain an image processing result according to the first feature.

[0078] In a possible implementation, the function unit selection module is further configured to: determine the target function unit according to the type of the processing task, the type of the plurality of function units, a resource occupation status of the plurality of function units, and a progress of the processing task.

[0079] In a possible implementation, the apparatus further includes: a reference feature acquisition module, configured to acquire a reference feature by the post-processing unit; and a reference feature storage module, configured to store the reference feature to a database of the edge computing platform.

[0080] In a possible implementation, the image processing module is further configured to: perform matching processing on the plurality of reference features in the database according to the first feature, to obtain a matching result; and obtain the image processing result according to the matching result.

[0081] In a possible implementation, the apparatus further includes an operation module configured to acquire, by the post-processing unit, an operation instruction for a reference feature in the database, and perform corresponding processing on the reference feature according to the operation instruction, where the operation instruction includes a creation instruction, a reading instruction, a replacement instruction, and a deletion instruction.

[0082] In a possible implementation, the apparatus further includes a result output module configured to cache, by the post-processing unit, the image processing result to a second message queue, and output the image processing result through the second message queue.

[0083] In a possible implementation, the target functional unit for performing the processing task in the plurality of functional units is implemented by a service agent component and a key-value storage component, and the first message queue is implemented by an open source message agent component.

[0084] The method has specific technical association with the internal structure of the computer system, and can solve the technical problem of how to improve the hardware operation efficiency or execution effect (including reducing data storage amount, reducing data transmission amount, improving hardware processing speed, etc.), so as to obtain the technical effect of improving the internal performance of the computer system in line with the natural law.

[0085] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, details are not repeated here.

[0086] The embodiments of the present disclosure also provide a computer-readable storage medium having computer program instructions stored therein, and the computer program instructions are executed by a processor to implement the above method. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0087] The embodiments of the present disclosure also provide an electronic device, including: a processor; a memory for storing processor-executable instructions; and wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

[0088] The embodiments of the present disclosure further provide a computer program product, comprising computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, when the computer readable code is run in the processor of the electronic device, the processor in the electronic device performs the above method.

[0089] The electronic device can be provided as a terminal, a server or other forms of devices.

[0090] Figure 4 A block diagram of an electronic device 800 is shown according to an embodiment of the present disclosure. For example, the electronic device 800 can be a terminal device such as a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, and the like.

[0091] Referring to Figure 4 The electronic device 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0092] The processing component 802 usually controls overall operations of the electronic device 800, such as operations associated with displaying, phone calling, data communication, camera operation and recording operation. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of steps of the methods described above. Further, the processing component 802 can include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0093] The memory 804 is configured to store various types of data to support operations of the electronic device 800. Examples of these data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0094] The power component 806 provides power to the various components of the electronic device 800. The power component 806 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0095] The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0096] The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting an audio signal.

[0097] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0098] The sensor component 814 includes one or more sensors for providing status assessments for various aspects of the electronic device 800. For example, the sensor component 814 can detect an open / closed position of the electronic device 800, relative positioning of components of the electronic device 800, such as a display and a keypad of the electronic device 800, a change in position of the electronic device 800 or a component of the electronic device 800, presence or absence of user contact with the electronic device 800, orientation or acceleration / deceleration / g-force and temperature changes of the electronic device 800. The sensor component 814 can include an optical sensor that is configured to detect ambient light, a proximity sensor configured to detect the presence of nearby objects without any physical touch, or a color sensor configured to detect colors in images captured by the image sensor. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0099] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a corresponding communication standard, such as wireless fidelity (Wi-Fi), second generation (2G) cellular technology, third generation (3G) cellular technology, fourth generation (4G) cellular technology, long term evolution (LTE) of universal mobile telecommunications technology, fifth generation (5G) cellular technology, or a combination thereof. In an example embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system using a broadcast channel. In an example embodiment, the communication component 816 can further include a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technology.

[0100] In an example embodiment, the electronic device 800 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements to perform the above-described methods.

[0101] In an example embodiment, a non-transitory computer-readable storage medium, such as the memory 804 including computer program instructions, is also provided, which can be executed by the processor 820 of the electronic device 800 to perform the above-described methods.

[0102] The present disclosure relates to the field of augmented reality, by acquiring image information of a target object in a real environment, and then detecting or recognizing the related features, states and attributes of the target object by means of various vision-related algorithms, so as to obtain an AR effect combining virtual and real objects that matches a specific application. Exemplarily, the target object can involve a face, a limb, a gesture, a motion, etc. related to a human body, or a marker, a sign, etc. related to an object, or a sand table, a display area or a display object, etc. related to a venue or a site. The vision-related algorithms can involve visual positioning, SLAM, three-dimensional reconstruction, image registration, background segmentation, key point extraction and tracking of an object, pose or depth detection of an object, etc. The specific application can not only involve interactive scenarios such as touring, navigation, explanation, reconstruction, virtual effect superimposed display, etc. related to a real scene or an object, but also interactive scenarios such as special effect processing related to a person, e.g. makeup beautification, limb beautification, special effect display, virtual model display, etc. The detection or recognition of the related features, states and attributes of the target object can be achieved by means of a convolutional neural network. The convolutional neural network is a network model obtained by model training based on a deep learning framework.

[0103] Figure 5 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 can be provided as a server or a terminal device. Referring to Figure 5 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932, for storing instructions executable by the processing component 1922, such as an application program. The application program stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-mentioned method.

[0104] The electronic device 1900 can further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as a Microsoft server operating system (Windows Server TM ), a graphical user interface operating system (Mac OSX TM ) developed by Apple Inc., a multi-user multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or the like.

[0105] In example embodiments, a non-transitory computer-readable storage medium, e.g., memory 1932 including computer program instructions, is also provided, which can be executed by processing component 1922 of electronic device 1900 to implement the above-described methods.

[0106] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

[0107] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a

[0108] The computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0109] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0110] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0111] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other data storage device. When the computer readable program instructions are loaded into the computer and other programmable data processing apparatus, a series of operational steps are implemented that provide processes such that the instructions which operate on the computer or other programmable data processing apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0112] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0113] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0114] The computer program product can be embodied in a tangible medium of

[0115] The above description of the various embodiments is intended to be illustrative in all aspects, rather than being restrictive. Those skilled in the art can be aware that not all aspects have been described. The descriptions of the various embodiments are intended to enable those skilled in the art to make or use the various embodiments, and it is therefore intended that the scope of the various embodiments be defined by the appended claims, and not limited to the foregoing description. Modifications of the various embodiments made during the implementation process are intended to fall within the scope of the various embodiments. In some instances, details unnecessary to

[0116] If the technical solutions of the present application involve personal information, the product applying the technical solutions of the present application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solutions of the present application involve sensitive personal information, the product applying the technical solutions of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent mark is set to inform that it has entered the personal information collection range and will collect personal information. If the individual voluntarily enters the collection range, it is considered to agree to collect personal information. Or on the device for processing personal information, through the pop-up information or by asking the individual to upload his personal information, the individual's authorization is obtained under the condition that the device uses obvious mark / information to inform the individual of the personal information processing rules. The personal information processing rules can include personal information processor, personal information processing purpose, processing method and personal information type, etc.

[0117] The above has described various embodiments of the present disclosure, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles, practical application or improvement of technology in the market of the embodiments, or to enable other ordinary skilled in the art to understand the embodiments disclosed herein.

Claims

1. An image processing method, characterized by, The method is used for an edge computing platform, and comprises: in the case of receiving a processing task for a to-be-processed image, determining a target functional unit for executing the processing task from a plurality of functional units of the edge computing platform according to a type of the processing task and a type of the plurality of functional units; performing feature extraction processing on the to-be-processed image by the target functional unit to obtain first features corresponding to the processing task; caching the first features to a first message queue; reading the first features from the first message queue by a post-processing unit and obtaining an image processing result according to the first features; determining the target functional unit for executing the processing task from the plurality of functional units according to the type of the processing task and the type of the plurality of functional units, comprising: determining the target functional unit according to the type of the processing task, the type of the plurality of functional units, resource occupation conditions of the plurality of functional units and a process of the processing task; the determination of the target functional unit from the plurality of functional units is implemented by a service agent component and a key-value storage component, and the first message queue is implemented by an open-source message agent component.

2. The method of claim 1, wherein, The method further comprises: acquiring reference features by the post-processing unit; storing the reference features to a database of the edge computing platform.

3. The method of claim 2, wherein, reading the first features from the first message queue by the post-processing unit and obtaining the image processing result according to the first features, comprising: performing matching processing on a plurality of reference features in the database according to the first features to obtain a matching result; obtaining the image processing result according to the matching result.

4. The method of claim 2, wherein, The method further comprises: acquiring an operation instruction for the reference features in the database by the post-processing unit and performing corresponding processing on the reference features according to the operation instruction, wherein the operation instruction comprises a creation instruction, a reading instruction, a replacement instruction and a deletion instruction.

5. The method of claim 1, wherein, The method further comprises: caching the image processing result to a second message queue by the post-processing unit; outputting the image processing result through the second message queue.

6. An image processing apparatus characterized by comprising: comprising: a functional unit selection module, configured to determine a target functional unit for executing a processing task for a to-be-processed image from a plurality of functional units of an edge computing platform according to a type of the processing task and a type of the plurality of functional units in the case of receiving the processing task; a feature extraction module, configured to perform feature extraction processing on the to-be-processed image by the target functional unit to obtain first features corresponding to the processing task; a caching module, configured to cache the first features to a first message queue; an image processing module, configured to read the first features from the first message queue by a post-processing unit and obtain an image processing result according to the first features; the functional unit selection module is further configured to determine the target functional unit according to the type of the processing task, the type of the plurality of functional units, resource occupation conditions of the plurality of functional units and a process of the processing task; The determining the target function unit for executing the processing task in the plurality of function units is implemented by a service proxy component and a key-value storage component, and the first message queue is implemented by an open source message proxy component.

7. An electronic device, comprising: Comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by a processor, implement the method of any one of claims 1 to 5.

Images