Image processing method, device, computer readable medium and electronic device

Abstract

The embodiments of the present application provide an image processing method, device, computer-readable medium and electronic device. The image processing method includes: performing target detection on the image to be processed; if it is detected that the image to be processed contains a target object, generating a heat map according to the feature map of the image to be processed; determining the target pixel in the heat map according to the thermal value of each pixel in the heat map, and determining the display area corresponding to the position of the target pixel in the image to be processed according to the position of the target pixel; adding a first multimedia element in the display area. The technical solution of the embodiment of the present application effectively improves the efficiency of image processing.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, computer-readable medium and electronic device. Background Art

[0002] With the advent of the 5G era, the application of live broadcast technology in entertainment, education and other industries is becoming more and more popular, and many novel and interesting new ways of playing have emerged, such as live broadcast interaction, personalized stickers, etc., which can enhance the user experience by adding lively and interesting stickers to the user's avatar in real time.

[0003] However, existing image processing methods involving sticker applications have many defects such as long algorithm flow, large amount of calculation, and too much redundant calculation. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, apparatus, computer-readable medium, and electronic device, thereby effectively improving the efficiency of image processing at least to a certain extent.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0006] According to one aspect of an embodiment of the present application, there is provided an image processing method, comprising: performing target detection on an image to be processed; if it is detected that the image to be processed contains a target object, generating a heat map according to a feature map of the image to be processed; determining a target pixel in the heat map according to the thermal value of each pixel in the heat map, and determining a display area in the image to be processed corresponding to the position of the target pixel according to the position of the target pixel; and adding a first multimedia element within the display area.

[0007] According to one aspect of an embodiment of the present application, an image processing device is provided, comprising: a detection unit, configured to perform target detection on an image to be processed; a generation unit, configured to generate a heat map according to a feature map of the image to be processed if it is detected that the image to be processed contains a target object; a first determination unit, configured to determine the target pixel in the heat map according to the thermal values ​​of each pixel in the heat map, and determine a display area corresponding to the position of the target pixel in the image to be processed according to the position of the target pixel; and a first adding unit, configured to add a first multimedia element in the display area.

[0008] In some embodiments of the present application, based on the aforementioned scheme, the first determination unit includes: a first determination subunit, configured to determine the pixel point with the largest thermal value in the thermodynamic map according to the thermal values ​​of each pixel point in the thermodynamic map; and a second determination subunit, configured to use the pixel point with the largest thermal value in the thermodynamic map as the target pixel point.

[0009] In some embodiments of the present application, based on the aforementioned scheme, the second determination subunit is configured as follows: if the pixel point with the largest thermal value is within the specified area of ​​the thermal map, the pixel point with the largest thermal value is used as the target pixel point; if the pixel point with the largest thermal value is not within the specified area, it is determined that the target pixel point does not exist in the thermal map.

[0010] In some embodiments of the present application, based on the aforementioned scheme, the device further includes: a second adding unit, configured to add a second multimedia element in a predetermined area in the image to be processed if it is determined that the target pixel does not exist in the heat map.

[0011] In some embodiments of the present application, based on the aforementioned solution, the generating unit is configured to: generate the heat map based on the feature value of a partial area specified in the feature map of the image to be processed.

[0012] In some embodiments of the present application, based on the aforementioned scheme, the device also includes: an input unit, configured to input the image to be processed into a pre-trained feature extraction model, wherein the feature extraction model includes multiple feature extraction layers; a selection unit, configured to select a feature map output by any feature extraction layer in the feature extraction model as the feature map of the image to be processed, or to generate a feature map of the image to be processed based on the feature maps output by multiple feature extraction layers in the feature extraction model.

[0013] In some embodiments of the present application, based on the aforementioned scheme, the device also includes: an acquisition unit, configured to acquire a first probability that the target object is included in the image to be processed and a second probability that the target object is not included in the image to be processed; a calculation unit, configured to calculate a first ratio of the first probability to the second probability, and a second ratio of the second probability to the first probability; a second determination unit, configured to determine whether the target object is included in the image to be processed based on the relationship between the first ratio and the second ratio.

[0014] In some embodiments of the present application, based on the aforementioned scheme, the acquisition unit includes: an extraction subunit, configured to perform feature extraction on the image to be processed to obtain a feature vector of the image to be processed; a classification subunit, configured to classify the image to be processed based on the feature vector to obtain a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object.

[0015] In some embodiments of the present application, based on the aforementioned scheme, the extraction subunit is configured to: use a pre-trained feature extraction model to perform feature extraction on the image to be processed, and obtain the features output by each feature extraction layer of the feature extraction model; based on the features output by each feature extraction layer, generate a feature vector of the image to be processed.

[0016] In some embodiments of the present application, based on the aforementioned scheme, the classification subunit is configured to: input the feature vector of the image to be processed into a predetermined classifier; obtain the classification result output by the classifier, and the classification result includes a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object.

[0017] In some embodiments of the present application, based on the aforementioned scheme, the first determination unit is configured as: according to the mapping relationship between the heat map and the image to be processed, mapping the position of the target pixel point in the heat map to the image to be processed, and obtaining the target position corresponding to the position of the target pixel point in the image to be processed; using an area of ​​a preset size containing the target position as a display area corresponding to the position of the target pixel point.

[0018] In some embodiments of the present application, based on the aforementioned scheme, the device further includes: a third adding unit, configured to add a second multimedia element in a predetermined area in the image to be processed if it is determined that the image to be processed does not contain the target object.

[0019] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method as described in the above embodiment is implemented.

[0020] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in the above embodiments.

[0021] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the image processing method provided in the above various optional embodiments.

[0022] In the technical solutions provided in some embodiments of the present application, when it is detected that the image to be processed contains the target object, a heat map is generated according to the feature map of the image to be processed, and then the target pixel in the heat map is determined according to the thermal value of each pixel in the heat map, and according to the position of the target pixel, the display area corresponding to the position of the target pixel is determined in the image to be processed, so as to achieve the purpose of adding the first multimedia element in the display area. The technical solution of the embodiment of the present application generates a heat map through the feature map of the image to be processed, and then determines the display area according to the position of the target pixel in the heat map, so as to add the first multimedia element in the display area, which overcomes the problems of large amount of calculation and long algorithm flow in the image processing process in the prior art, saves labor costs, effectively improves the efficiency of image processing, and can be applied to any image file, with wide applicability.

[0023] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0025] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied;

[0026] Figure 2 A flowchart of an image processing method according to an embodiment of the present application is shown;

[0027] Figure 3 A flowchart of an image processing method according to an embodiment of the present application is shown;

[0028] Figure 4 A flowchart of an image processing method according to an embodiment of the present application is shown;

[0029] Figure 5 A flowchart of an image processing method according to an embodiment of the present application is shown;

[0030] Figure 6 A flowchart of an image processing method according to an embodiment of the present application is shown;

[0031] Figure 7A-7B A schematic diagram showing a thermal map and an image to be processed according to an embodiment of the present application is shown;

[0032] Figure 8 A logic diagram of an image processing method according to an embodiment of the present application is shown;

[0033] Fig. 9 A block diagram of an image processing device according to an embodiment of the present application is shown;

[0034] Fig.10 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0035] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0036] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0037] It should be noted that the terms used in the specification and claims of this application and the above-mentioned drawings are only used to describe the embodiments and are not intended to limit the scope of this application. It should be understood that the terms "include", "comprises", "having", etc., when used in this article, specify the existence of the stated features, wholes, steps, operations, elements, components and / or their groups, but do not exclude the existence or addition of one or more of other features, wholes, steps, operations, elements, components and / or their groups.

[0038] It will be further understood that, although the terms "first", "second", "third", etc. may be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the present invention, the first element may be referred to as the second element. Similarly, the second element may be referred to as the first element. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0039] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0040] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0041] It should be noted that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0042] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied is shown.

[0043] like Figure 1 As shown, the system architecture 100 may include a terminal device 102 and a server 104. The terminal device 102 and the server 104 may be used to execute the image processing method provided in the embodiment of the present application independently. The terminal device 102 and the server 104 may also execute the image processing method provided in the embodiment of the present application through interaction and collaboration.

[0044] It should be noted that in the present application, the terminal device 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal device 102 and the server 104 may be directly or indirectly connected via wired or wireless communication, and the present application does not limit this.

[0045] Server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0046] The following describes the image processing method of the embodiment of the present application with the terminal device 102 as the execution subject. In one embodiment of the present application, the terminal device 102 can perform target detection on the image to be processed, wherein the image to be processed can be an image obtained by the terminal device 102 from a local or other database. If the terminal device 102 detects that the image to be processed contains a target object, a heat map can be further generated according to the feature map of the image to be processed. Then, the terminal device 102 can determine the target pixel in the heat map according to the thermal value of each pixel in the heat map. After determining the target pixel, the terminal device 102 can determine the display area corresponding to the position of the target pixel in the image to be processed according to the position of the target pixel. Finally, the terminal device 102 can add the first multimedia element to the display area. The first multimedia element can be a static image, a dynamic image (such as a dynamic GIF file), or a flash, video, or a three-dimensional animation file.

[0047] The image processing method provided in the embodiment of the present application involves artificial intelligence computer vision technology and machine learning technology. Among them:

[0048] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning.

[0049] Computer vision technology (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform image processing so that the computer processing becomes an image that is more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0050] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0051] This application illustrates the computer vision technology and machine learning technology involved through the following embodiments.

[0052] In one embodiment, Figure 2 As shown, an image processing method is provided, and the method is applied to a computer device as an example. The computer device can be Figure 1 The terminal device 102 or the server 104 in the image processing method comprises the following steps:

[0053] Step S210, performing target detection on the image to be processed;

[0054] Step S220: if it is detected that the image to be processed contains a target object, a heat map is generated according to the feature map of the image to be processed;

[0055] Step S230, determining a target pixel in the heat map according to the heat value of each pixel in the heat map, and determining a display area corresponding to the position of the target pixel in the image to be processed according to the position of the target pixel;

[0056] Step S240: adding a first multimedia element in the display area.

[0057] These steps are described in detail below.

[0058] In step S210, target detection is performed on the image to be processed.

[0059] In this embodiment, target detection can be first performed on the image to be processed. The target detection is used to detect whether there is a target object in the image to be processed. The target object may include but is not limited to human faces, animals, buildings, virtual objects (game characters, animation characters, etc.) or other physical objects, and the target object may be one or more different objects, which may be determined according to the actual application scenario. For example, when the image to be processed is an image of a human face, the target object may be the human face in the image to be processed. When the image to be processed is an image of a building, the target object may be the building in the image to be processed. When the image to be processed is an image of a classroom, the target object may include items such as blackboards, chalk, desks, and may also include people.

[0060] The image to be processed may be acquired in real time, such as an image captured in real time by a camera of a terminal device, or may be an image acquired from a database, a blockchain, or a distributed file system, which is not limited in the embodiments of the present application. The image to be processed described here may be a separate image or a frame in a video.

[0061] The target detection of the processed image can be realized by using the existing target detection algorithm. The target detection algorithm is mainly divided into two categories: one-stage method and two-stage method. The one-stage method mainly outputs the coordinates of the target object (i.e., the predicted box including the target object) and confidence through several feature maps. The main representative algorithms of the one-stage method are Yolo algorithm and Single Shot MultiBox Detector (SSD). Among them, YoLo algorithm is a one-stage fast target detection algorithm. YoLo divides the whole image into S×S grids, and each grid is responsible for the target detection centered in the grid. The problem is solved at one time by predicting the coordinate values, confidence and all category probability vectors of the target objects contained in all grids at one time. The SSD algorithm is a one-stage target detection algorithm, which uses a convolutional neural network to perform detection directly, instead of performing detection after the fully connected layer like Yolo.

[0062] The two-step method generally goes through two steps. The first step is to process the image to obtain many candidate regions (i.e., Proposals), and then finely classify and regress the candidate regions to obtain the final result. However, due to the huge number of candidate regions, in order to eliminate redundant candidate regions, find the best object detection position, and speed up the speed of target detection and recognition, methods such as non-maximum suppression (NMS) algorithms are generally used to remove redundant candidate regions. The main representative algorithms of the two-step method are R-CNN algorithms (R-CNN, Fast R-CNN, Faster R-CNN, Face R-CNN, etc.).

[0063] In step S220, if it is detected that the image to be processed contains a target object, a heat map is generated according to the feature map of the image to be processed.

[0064] If it is detected that the image to be processed contains a target object, a heat map can be further generated based on the feature map of the image to be processed. The generated heat map is a diagram showing the image feature information of the image to be processed in a preset display form, where the preset display form may include a special highlight form, a specified display color form, etc. The image feature information of the image to be processed includes the feature information of the target object in the image to be processed.

[0065] In the specific implementation, a thermal map is generated according to the feature map of the image to be processed, which can be realized in the following ways: first, the feature points of the feature map of the image to be processed are marked, and the feature points can be feature points on the target object in the image to be processed, and the coordinates of the feature points are generated; then, the thermal area corresponding to the feature point coordinates can be determined with the feature point coordinates as the center and the preset distance threshold as the side length; further, the thermal value of each pixel in the thermal area can be calculated according to the thermal weight corresponding to each pixel in the thermal area and the characteristic value corresponding to each pixel, and the color corresponding to the thermal value of each pixel is presented in the thermal area according to the thermal value of each pixel in the thermal area, so as to generate a thermal map. Among them, the thermal weight corresponding to each pixel in the thermal area can be determined according to the distance between each pixel in the thermal area and the center point of the thermal area. The farther the pixel is from the center point, the smaller the corresponding thermal value is, and the thermal value of the pixel at the boundary of the thermal area is gradually reduced to 0.

[0066] The above-mentioned thermal area can be a square area, or a triangle, circle, regular or irregular polygon. The specific shape of the thermal area can be determined according to actual needs and is not limited here.

[0067] In some embodiments, after determining the thermal region, an initial thermal value may be set for the feature point coordinates, and the initial thermal value is used as the mean value, and then the mean value and the preset variance are input into the thermal value calculation function to calculate the thermal value of each pixel in the thermal region, and finally, according to the thermal value of each pixel in the thermal region, the color corresponding to the thermal value of each pixel is presented in the thermal region to generate a thermal map. Specifically, the thermal value calculation function may be a Gaussian function.

[0068] In step S230, the target pixel in the heat map is determined according to the thermal value of each pixel in the heat map, and the display area corresponding to the position of the target pixel is determined in the image to be processed according to the position of the target pixel.

[0069] After the heat map is generated, the target pixel in the heat map can be further determined according to the thermal value of each pixel in the heat map, and the display area corresponding to the position of the target pixel can be determined in the image to be processed according to the position of the target pixel.

[0070] It can be understood that the heat map is a diagram that displays the image feature information of the image to be processed in a preset display format. The size of the thermal value can reflect the feature significance of each pixel point. The computer device can directly determine the target pixel point based on the thermal value size of each pixel point in the heat map. In other words, the computer device can locate the pixel point with significant features in the image to be processed by selecting the target pixel point. In this way, the display area corresponding to the position of the target pixel point is determined in the image to be processed, that is, the area with significant features in the image to be processed is determined.

[0071] Among them, the meaning of feature significance can be understood as the difference between the pixel value of a pixel point and the pixel values ​​of multiple surrounding pixels is greater than the preset pixel difference, and the feature significance of each pixel point of the target object in the image to be processed is stronger than the feature significance of each pixel point outside the target object.

[0072] In step S240, a first multimedia element is added in the display area.

[0073] The first multimedia element can be used to be added to the display area of ​​the image to be processed. By adding the first multimedia element to the image to be processed, the expression form of the image can be enriched, giving the user a fresher and more interesting experience.

[0074] The first multimedia element can be a static image file, a dynamic image file (such as a dynamic GIF file), or a flash, video, or three-dimensional animation file in terms of file format; and can be clothing or scenes in terms of content. For example, the first multimedia element can be a decorative element of a human face, such as a hat, a fan, a handkerchief, glasses, etc.; a component element of a human face, such as a specially made mouth, eyebrows, beard, and eyes, etc.; a background element, such as a landscape, a funny background, etc.

[0075] After the display area is determined, the first multimedia element can be added to the display area. In specific implementation, the first multimedia element can be added to the display area in a preset manner, which can be a manner of aligning the center of the first multimedia element with the center of the display area, or can be performed according to the target object in the display area. The corresponding first multimedia element is added, for example, when the target object is a person, the glasses element as the first multimedia element can be added to the eye part of the target object in the display area, and other methods are also possible, which are not specifically limited here.

[0076] Based on the technical solution of the above embodiment, a heat map is generated by a feature map of the image to be processed, and then the display area is determined according to the position of the target pixel in the heat map, so as to add the first multimedia element in the display area. This overcomes the problems of large amount of calculation and long algorithm flow in the image processing process in the prior art, saves labor costs, effectively improves the efficiency of image processing, and can be applied to any image file, with wide applicability.

[0077] In one embodiment of the present application, if the image to be processed does not contain the target object, an operation of adding a second multimedia element in a predetermined area in the image to be processed can be performed. In this embodiment, different from the case where the image to be processed contains the target object, in the case where the image to be processed contains the target object, a display area can be further determined in the image to be processed, and in the case where the image to be processed does not contain the target object, the second multimedia element can be directly added in the predetermined area in the image to be processed.

[0078] The predetermined area may be a fixed area pre-set in the image to be processed, and the second multimedia element may be any multimedia element that can be used for display, including but not limited to static images, dynamic images, flash, video, three-dimensional animation files, etc.

[0079] It is worth mentioning that the second multimedia element can be used to prompt that the target object is not included in the image to be processed. For example, when the target object is a human face, an image with a hollow part can be added to the predetermined area as the second multimedia element. The hollow part forms the shape of a human face, thereby prompting that the target object is not included in the image to be processed.

[0080] In one embodiment of the present application, Figure 3 As shown, determining the target pixel point in the heat map may specifically include steps S310 to S320:

[0081] Step S310, according to the thermal values ​​of each pixel in the thermal map, determine the pixel with the largest thermal value in the thermal map;

[0082] Step S320: Take the pixel with the largest thermal value in the thermal map as the target pixel.

[0083] In this embodiment, the pixel with the largest thermal value in the thermodynamic map can be first determined according to the thermal values ​​of each pixel in the thermodynamic map, and then the pixel with the largest thermal value in the thermodynamic map can be used as the target pixel.

[0084] In one embodiment of the present application, after determining the pixel with the largest thermal value in the heat map, it can be further determined whether the pixel with the largest thermal value is within a specified area of ​​the heat map. If the pixel with the largest thermal value is within the specified area of ​​the heat map, the pixel with the largest thermal value can be used as the target pixel.

[0085] In this embodiment, considering that the target object may be in the edge area of ​​the image to be processed, and adding the first multimedia element in the edge area is not very meaningful, therefore, only when the pixel with the largest thermal value is in the specified area of ​​the heat map, the pixel with the largest thermal value will be used as the target pixel, and then the display area is determined according to the target pixel, and the first multimedia element is added to the display area. On the contrary, if the pixel with the largest thermal value is not in the specified area of ​​the heat map, it can be determined that the target pixel does not exist in the heat map.

[0086] In one embodiment of the present application, if it is determined that there is no target pixel in the heat map, and therefore there is no need to determine a display area corresponding to the position of the target pixel in the image to be processed based on the target pixel, then in this case, a second multimedia element can be directly added to a predetermined area in the image to be processed.

[0087] The predetermined area may be a fixed area pre-set in the image to be processed, and the second multimedia element may be any multimedia element that can be used for display, including but not limited to static images, dynamic images, flash, video, three-dimensional animation files, etc.

[0088] In one embodiment of the present application, since the probability of the target object being in the edge area of ​​the image to be processed is very small in actual applications, when generating a heat map based on the feature map of the image to be processed, the heat map can be generated only for a specified portion of the area in the feature map. This can further reduce the amount of calculation in the image processing process.

[0089] Specifically, in this embodiment, the computer device may first obtain the feature value of the specified partial area in the feature map of the image to be processed, the feature value is the value corresponding to each feature point in the specified partial area, and the computer device may calculate the feature value of each feature point through a preset algorithm. The specified partial area may be determined according to actual conditions and is not specifically limited here.

[0090] It can be understood that the designated partial area is also the thermal area. Therefore, after obtaining the characteristic value of the designated partial area, the thermal value of each pixel point can be calculated based on the characteristic value corresponding to each pixel point in the designated partial area. Then, based on the thermal value of each pixel point, the color corresponding to the thermal value of each pixel point is presented in the designated partial area to generate a thermal map.

[0091] In one embodiment of the present application, the feature map of the image to be processed can be obtained by inputting the image to be processed into a pre-trained feature extraction model for feature extraction to obtain the corresponding feature map. Specifically, the feature extraction model can include multiple feature extraction layers, each feature extraction layer is used to perform feature processing on the input image to be processed and output a feature map.

[0092] The feature extraction model is obtained by training a model composed of a neural network. The neural network model can be a convolutional neural network model (CNN), a deep neural network model (DNN), and a recurrent neural network model (RNN), etc. The embodiments of the present application are not specifically limited here.

[0093] Among them, the convolutional neural network includes convolutional layers and pooling layers. The deep neural network includes input layers, hidden layers and output layers, and the layers are fully connected. The recurrent neural network is a neural network that models sequence data, that is, the current output of a sequence is also related to the previous output. The specific manifestation is that the network will remember the previous information and apply it to the calculation of the current output, that is, the nodes between the hidden layers are no longer disconnected but connected, and the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the previous moment.

[0094] In this embodiment, the image to be processed is input into a pre-trained feature extraction model for feature extraction to obtain a feature map. The feature map output by any feature extraction layer in the feature extraction model can be selected as the feature map of the image to be processed. The feature map of the image to be processed can also be generated based on the feature maps output by multiple feature extraction layers in the feature extraction model. For example, the feature maps respectively input by multiple feature extraction layers are fused to obtain the feature map of the image to be processed.

[0095] In one embodiment of the present application, whether the image to be processed contains the target object can be determined by a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object, such as Figure 4 As shown, in this embodiment, the steps S410 to S430 may be specifically included, which are described in detail as follows:

[0096] Step S410: Obtain a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object.

[0097] In order to determine whether the image to be processed contains the target object, the computer device can pre-acquire a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object. It can be understood that the image to be processed either contains the target object or does not contain the target object, so the sum of the first probability and the second probability is 1.

[0098] Among them, the method for obtaining the first probability that the image to be processed contains the target object and the second probability that the image to be processed does not contain the target object can be predicted through a probability prediction model, for example, using a machine learning method to train a probability prediction model based on a training image sample set, and then obtaining the corresponding probability based on the probability prediction model.

[0099] Step S420: Calculate a first ratio of the first probability to the second probability, and a second ratio of the second probability to the first probability.

[0100] After the first probability and the second probability are obtained, a first ratio of the first probability to the second probability and a second ratio of the second probability to the first probability may be calculated according to the first probability and the second probability.

[0101] Step S430: Determine whether the image to be processed contains a target object based on the relationship between the first ratio and the second ratio.

[0102] In this embodiment, after the first ratio and the second ratio are calculated, it can be determined whether the image to be processed contains the target object based on the relationship between the first ratio and the second ratio.

[0103] In some embodiments, since the probability problem can be converted into a chance ratio problem of predicting the occurrence of an event, after calculating the first ratio and the second ratio, the logarithm of the first ratio and the logarithm of the second ratio can be further calculated. If the logarithm of the first ratio is greater than or equal to the logarithm of the second ratio, it can be determined that the image to be processed contains the target object. On the contrary, if the logarithm of the first ratio is less than the logarithm of the second ratio, it can be determined that the image to be processed does not contain the target object.

[0104] In one embodiment of the present application, the method for obtaining the first probability that the image to be processed contains the target object and the second probability that the image to be processed does not contain the target object can also be obtained by classifying the image to be processed. In this embodiment, Figure 5 As shown, it may specifically include steps S510 to S520, which are specifically described as follows:

[0105] In step S510, feature extraction is performed on the image to be processed to obtain a feature vector of the image to be processed.

[0106] In this embodiment, first, feature extraction can be performed on the image to be processed to obtain a feature vector of the image to be processed. The feature vector is a vectorized representation of the features of the image. The feature is used to describe the characteristics of the image and can be extracted based on relevant information of the image to be processed, such as at least one of the information such as the content contained in the image, and can also be extracted based on attribute information corresponding to the image to be processed, such as the author of the image to be processed or the label corresponding to the image to be processed.

[0107] In one embodiment, feature extraction of the image to be processed can be performed using a pre-trained feature extraction model to extract features of the image to be processed, obtain features output by each feature extraction layer of the feature extraction model, and generate a feature vector of the image to be processed based on the features output by each feature extraction layer. The feature vector of the image to be processed is composed of multiple layers of features of the image to be processed extracted from the feature extraction model. In the feature extraction model, the image to be processed is input into the lowest layer of the feature extraction model, a feature is extracted at each layer, and transmitted to the next layer in sequence, that is, the features extracted from the previous layer are input to the next layer. The entire feature extraction model can extract multiple layers of features of the image to be processed, and the feature vector of the image to be processed can be composed based on the multiple layers of features.

[0108] In step S520, the image to be processed is classified based on the feature vector to obtain a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object.

[0109] After extracting the feature vector of the image to be processed, the computer device can classify the image to be processed based on the feature vector of the image to be processed, and then obtain a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object based on the classification data.

[0110] In a specific implementation, one implementation method is to input the feature vector of the image to be processed into a predetermined classifier, and the predetermined classifier outputs the classification result. The classifier is a machine learning model capable of classifying images, and the machine learning model can be a model trained by a convolutional neural network model, a recurrent neural network model, a support vector machine model, etc.

[0111] The classification result may include probability values ​​corresponding to each preset classification category, and the classification result may be a vector, each vector value in the vector represents the probability corresponding to each preset classification category. In this embodiment, the preset classification category may include two categories: the image to be processed contains the target object and the image to be processed does not contain the target object. Therefore, after the feature vector of the image to be processed is input into a predetermined classifier, the classification result output by the classifier can obtain a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object.

[0112] For example, the preset classification categories include two categories: the image to be processed contains the target object and the image to be processed does not contain the target object. The classification result output by the classifier is (0.8 0.2). According to the classification result, it can be known that the first probability that the image to be processed contains the target object is 0.8, and the second probability that the image to be processed does not contain the target object is 0.2.

[0113] Another implementation is that, for each preset classification category, the probability that the image to be processed belongs to each preset classification category can be determined based on the distance between the feature vector and the category feature vector of each preset classification category, and the weight of the category feature vector of each preset classification category. The category feature vector can be determined based on the training feature vector of the training sample under the corresponding preset classification category.

[0114] In one embodiment of the present application, Figure 6 As shown, the method for determining the display area corresponding to the position of the target pixel point may include steps S610 to S620, which are described in detail as follows:

[0115] Step S610: According to the mapping relationship between the heat map and the image to be processed, the position of the target pixel point in the heat map is mapped to the image to be processed, so as to obtain the target position corresponding to the position of the target pixel point in the image to be processed.

[0116] After determining the target pixel point in the heat map, the position of the target pixel point in the heat map can be directly mapped to the image to be processed according to the mapping relationship between the heat map and the image to be processed, so as to obtain the target position corresponding to the position of the target pixel point in the image to be processed.

[0117] Specifically, the heat map can be regarded as a first matrix composed of multiple rows and columns of pixels, and the image to be processed can also be regarded as a second matrix composed of multiple rows and columns of pixels. The proportional relationship between the heat map and the image to be processed can be determined according to the length and width of the first matrix and the length and width of the second matrix, and the mapping relationship between the heat map and the image to be processed can be obtained based on the proportional relationship.

[0118] Therefore, in an embodiment of the present application, a mapping relationship is applied. If the position of a target pixel is known, the position of the target pixel can be mapped to the image to be processed to obtain a target position in the image to be processed that corresponds to the position of the target pixel.

[0119] See also Fig. 7A and Figure 7B , Figure 7A-7B A schematic diagram of a heat map and an image to be processed according to an embodiment of the present application is shown, as shown in FIG. Fig. 7A As shown, each rectangular block in the heat map represents a pixel point. The size of the heat map is a 5×5 matrix. There is a target pixel point in the heat map. The target pixel point is represented by the letter "C". The target pixel point is located at the center of the first matrix, as shown in Figure 7B As shown in the figure, each rectangular block in the image to be processed also represents a pixel, and the size of the image to be processed is a 20×20 matrix. Therefore, according to the length and width relationship of the two matrices, it can be concluded that one pixel in the heat map can correspond to four pixels in the image to be processed. Based on this mapping relationship, the position of the target pixel C can be mapped to the image to be processed, and the target position corresponding to the position of the target pixel in the image to be processed is obtained, that is, the black area C'.

[0120] Step S620: Use a region of a preset size including the target position as a display region corresponding to the position of the target pixel.

[0121] During specific implementation, considering that the display area is used to add the first multimedia element, and the size of the target position may be inconsistent with the size of the first multimedia element, therefore, in this embodiment, an area of ​​a preset size including the target position can be used as the display area corresponding to the position of the target pixel.

[0122] The area of ​​preset size including the target position may be an area of ​​preset size centered on the target position or an area of ​​preset size not centered on the target position. The preset size may be the same as the size of the first multimedia element or larger than the first multimedia element.

[0123] Figure 8 A logic diagram of an image processing method according to an embodiment of the present application is shown, as shown in FIG. Figure 8 As shown, the image processing method may specifically include the following steps:

[0124] S1. Perform target detection on the image to be processed. If it is detected that the image to be processed does not contain the target object, execute step S2; if it is detected that the image to be processed contains the target object, execute step S3.

[0125] S2. Add a second multimedia element in a predetermined area of ​​the image to be processed.

[0126] The second multimedia element may be used to indicate that the image to be processed does not contain the target object, for example, Figure 8 As shown in the image indicated by the middle number (1), when the target object is a human face, an image with a hollowed-out portion can be added to the predetermined area as a second multimedia element. The hollowed-out portion forms the shape of a human face, thereby indicating that the target object - a person - is not included in the image to be processed.

[0127] S3. Generate a heat map according to the feature map of the image to be processed, and determine the target pixel in the heat map according to the heat value of each pixel in the heat map.

[0128] See also Figure 8 The image corresponding to the number (2) is shown in the figure, wherein the letter “C” represents the target pixel point in the determined heat map.

[0129] S4. According to the position of the target pixel, determine the display area corresponding to the position of the target pixel in the image to be processed. Figure 8 The image corresponding to the label (3) is shown, which shows that according to Figure 8 The display area determined by the position of the target pixel point C in the image corresponding to the number (2) is also represented by the letter “C”.

[0130] S5. Add a first multimedia element in the display area.

[0131] like Figure 8 The image corresponding to the number (4) shows an image obtained after adding the first multimedia element in the display area, wherein the image to be processed contains the target object, that is, a face, and the first multimedia element is an image containing the word "GOOD".

[0132] The following describes an apparatus embodiment of the present application, which can be used to execute the image processing method in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above-mentioned embodiment of the image processing method of the present application.

[0133] Fig. 9 A block diagram of an image processing device according to an embodiment of the present application is shown. Fig. 9 As shown, an image processing device 900 according to an embodiment of the present application includes: a detection unit 902 , a generation unit 904 , a first determination unit 906 and a first adding unit 908 .

[0134] Among them, the detection unit 902 is configured to perform target detection on the image to be processed; the generation unit 904 is configured to generate a heat map according to the feature map of the image to be processed if it is detected that the image to be processed contains a target object; the first determination unit 906 is configured to determine the target pixel point in the heat map according to the thermal value of each pixel point in the heat map, and determine the display area corresponding to the position of the target pixel point in the image to be processed according to the position of the target pixel point; the first adding unit 908 is configured to add a first multimedia element in the display area.

[0135] In some embodiments of the present application, the first determination unit 906 includes: a first determination subunit, configured to determine the pixel with the largest thermal value in the thermodynamic map according to the thermal values ​​of each pixel in the thermodynamic map; and a second determination subunit, configured to use the pixel with the largest thermal value in the thermodynamic map as the target pixel.

[0136] In some embodiments of the present application, the second determination subunit is configured as follows: if the pixel with the largest thermal value is within the specified area of ​​the thermal map, the pixel with the largest thermal value is used as the target pixel; if the pixel with the largest thermal value is not within the specified area, it is determined that the target pixel does not exist in the thermal map.

[0137] In some embodiments of the present application, the device further includes: a second adding unit, configured to add a second multimedia element in a predetermined area in the image to be processed if it is determined that the target pixel does not exist in the heat map.

[0138] In some embodiments of the present application, the generating unit 904 is configured to generate the heat map based on the feature value of a partial area specified in the feature map of the image to be processed.

[0139] In some embodiments of the present application, the device also includes: an input unit, configured to input the image to be processed into a pre-trained feature extraction model, wherein the feature extraction model includes multiple feature extraction layers; a selection unit, configured to select a feature map output by any feature extraction layer in the feature extraction model as the feature map of the image to be processed, or to generate a feature map of the image to be processed based on the feature maps output by multiple feature extraction layers in the feature extraction model.

[0140] In some embodiments of the present application, the device also includes: an acquisition unit, configured to acquire a first probability that the target object is included in the image to be processed and a second probability that the target object is not included in the image to be processed; a calculation unit, configured to calculate a first ratio of the first probability to the second probability, and a second ratio of the second probability to the first probability; a second determination unit, configured to determine whether the target object is included in the image to be processed based on the relationship between the first ratio and the second ratio.

[0141] In some embodiments of the present application, the acquisition unit includes: an extraction subunit, configured to perform feature extraction on the image to be processed to obtain a feature vector of the image to be processed; a classification subunit, configured to classify the image to be processed based on the feature vector to obtain a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object.

[0142] In some embodiments of the present application, the extraction subunit is configured to: use a pre-trained feature extraction model to perform feature extraction on the image to be processed, and obtain the features output by each feature extraction layer of the feature extraction model; and generate a feature vector of the image to be processed based on the features output by each feature extraction layer.

[0143] In some embodiments of the present application, the classification subunit is configured to: input the feature vector of the image to be processed into a predetermined classifier; obtain the classification result output by the classifier, wherein the classification result includes a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object.

[0144] In some embodiments of the present application, the first determination unit 906 is configured to: map the position of the target pixel point in the heat map to the image to be processed according to a mapping relationship between the heat map and the image to be processed, and obtain a target position in the image to be processed corresponding to the position of the target pixel point; and use an area of ​​a preset size containing the target position as a display area corresponding to the position of the target pixel point.

[0145] In some embodiments of the present application, the device further includes: a third adding unit configured to add a second multimedia element in a predetermined area in the image to be processed if it is determined that the image to be processed does not contain the target object.

[0146] Fig.10 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0147] It should be noted that Fig.10 The computer system 1000 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0148] like Fig.10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 to the random access memory (RAM) 1003, such as executing the method described in the above embodiment. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002 and the RAM 1003 are connected to each other through the bus 1004. The input / output (I / O) interface 1005 is also connected to the bus 1004.

[0149] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0150] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1009, and / or installed from a removable medium 1011. When the computer program is executed by a central processing unit (CPU) 1001, various functions defined in the system of the present application are executed.

[0151] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0152] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0154] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0155] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0156] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0157] Those skilled in the art will readily come up with other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0158] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Perform target detection on the image to be processed; If it is detected that the image to be processed contains a target object, a heat map is generated according to the feature map of the image to be processed; According to the thermal values ​​of each pixel in the thermal map, the pixel with the largest thermal value in the thermal map is determined; if the pixel with the largest thermal value is within a specified area of ​​the thermal map, the pixel with the largest thermal value is used as the target pixel in the thermal map, and according to the position of the target pixel, a display area corresponding to the position of the target pixel is determined in the image to be processed; A first multimedia element is added in the display area.

2. The method according to claim 1, characterized in that The method further comprises: If the pixel point with the largest thermal value is not in the specified area, it is determined that the target pixel point does not exist in the thermal map.

3. The method according to claim 2, characterized in that The method further comprises: If it is determined that the target pixel does not exist in the heat map, a second multimedia element is added in a predetermined area in the image to be processed.

4. The method according to claim 1, characterized in that: Generating a heat map according to the feature map of the image to be processed includes: The heat map is generated based on the feature values ​​of the partial area specified in the feature map of the image to be processed.

5. The method according to claim 1, characterized in that The method further comprises: Inputting the image to be processed into a pre-trained feature extraction model, wherein the feature extraction model includes multiple feature extraction layers; A feature map output by any one feature extraction layer in the feature extraction model is selected as the feature map of the image to be processed, or a feature map of the image to be processed is generated according to feature maps respectively output by multiple feature extraction layers in the feature extraction model.

6. The method according to claim 1, characterized in that The method further comprises: Acquire a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object; Calculating a first ratio of the first probability to the second probability, and a second ratio of the second probability to the first probability; Based on the relationship between the first ratio and the second ratio, it is determined whether the image to be processed contains the target object.

7. The method according to claim 6, characterized in that Acquiring a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object, including: Extracting features of the image to be processed to obtain a feature vector of the image to be processed; The image to be processed is classified based on the feature vector to obtain a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object.

8. The method according to claim 7, characterized in that Extracting features from the image to be processed to obtain a feature vector of the image to be processed includes: Using a pre-trained feature extraction model to extract features from the image to be processed, and obtaining features output by each feature extraction layer of the feature extraction model; Based on the features output by each feature extraction layer, a feature vector of the image to be processed is generated.

9. The method according to claim 7, characterized in that: Classifying the image to be processed based on the feature vector to obtain a first probability that the image to be processed contains the target object and a second probability that the image to be processed does not contain the target object, including: Inputting the feature vector of the image to be processed into a predetermined classifier; A classification result output by the classifier is obtained, wherein the classification result includes a first probability that the image to be processed includes the target object and a second probability that the image to be processed does not include the target object.

10. The method according to claim 1, characterized in that Determining, according to the position of the target pixel point, a display area corresponding to the position of the target pixel point in the image to be processed, comprises: According to the mapping relationship between the heat map and the image to be processed, the position of the target pixel point in the heat map is mapped to the image to be processed, so as to obtain the target position in the image to be processed corresponding to the position of the target pixel point; An area of ​​a preset size including the target position is used as a display area corresponding to the position of the target pixel point.

11. The method according to claim 1, characterized in that: The method further comprises: If it is determined that the image to be processed does not contain the target object, a second multimedia element is added in a predetermined area of ​​the image to be processed.

12. An image processing device, characterized in that: The device comprises: A detection unit configured to perform target detection on the image to be processed; A generating unit, configured to generate a heat map according to a feature map of the image to be processed if it is detected that the image to be processed contains a target object; A first determining unit, the first determining unit includes: a first determining subunit, configured to determine the pixel with the largest thermal value in the thermal map according to the thermal values ​​of each pixel in the thermal map; a second determining subunit, configured to take the pixel with the largest thermal value as the target pixel in the thermal map if the pixel is within a specified area of ​​the thermal map, and determine a display area corresponding to the position of the target pixel in the image to be processed according to the position of the target pixel; A first adding unit is configured to add a first multimedia element in the display area.

13. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 11 is implemented.

14. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the image processing method as described in any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the image processing method as described in any one of claims 1 to 11.

Images