Image processing method, device, electronic equipment and medium

Abstract

The embodiments of the present application disclose an image processing method, device, electronic device and medium, which can be applied to the field of data processing technology. The method includes: obtaining a heat map to be processed, determining the pixel point with the largest heat value in the heat map to be processed as the target key point, and determining M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed; determining the regional representation value of each coding grouping area based on the heat value corresponding to the pixel points contained in each coding grouping area in the M coding grouping areas; determining the heat map encoding corresponding to the heat map to be processed according to the coordinate information of the target key point and the regional representation value of each coding grouping area. Using the embodiments of the present application, the heat map can be processed into a heat map encoding, which helps to suppress the areas with incorrect thermal value predictions in the heat map, improve the accuracy of the heat map used for image processing tasks, and help reduce the amount of data transmitted by the heat map.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to image processing methods, devices, electronic equipment and media. Background Art

[0002] In some scenarios, such as human posture estimation, a heat map can be generated based on the original image, and the thermal value of each pixel in the heat map can be used to characterize the probability that the pixel is located at the corresponding part. Generally speaking, when generating a heat map, there may be areas in the heat map where the thermal value is predicted incorrectly, perhaps because the accuracy of the neural network used to generate the heat map is not enough, or the content of the original image is relatively complex. If the heat map is used directly for downstream image processing tasks, it is easy to affect the accuracy of the downstream image processing tasks. Therefore, how to improve the accuracy of the heat map used for downstream image processing tasks is an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present application provide an image processing method, device, electronic device and medium, which can process a heat map into a heat map encoding, help suppress areas in the heat map with incorrect thermal value predictions, improve the accuracy of the heat map used for image processing tasks, and help reduce the amount of data transmitted for the heat map.

[0004] In one aspect, an embodiment of the present application discloses an image processing method, the method comprising:

[0005] Obtain a heat map to be processed, wherein the heat map to be processed includes a plurality of pixels, and each pixel corresponds to a heat value;

[0006] Determine the pixel point with the largest thermal value in the heat map to be processed as the target key point, and determine M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed, where M is a positive integer;

[0007] Determine the region representation value of each coding group area based on the thermal value corresponding to the pixel point contained in each coding group area of ​​the M coding group areas;

[0008] The heat map code corresponding to the heat map to be processed is determined according to the coordinate information of the target key point and the area representation value of each coding group area. The heat map code is used to decode to obtain a decoded heat map and perform the target image processing task.

[0009] In one aspect, an embodiment of the present application discloses an image processing device, the device comprising:

[0010] An acquisition unit, used for acquiring a heat map to be processed, wherein the heat map to be processed includes a plurality of pixels, each pixel corresponding to a heat value;

[0011] A processing unit, used to determine the pixel point with the largest thermal value in the heat map to be processed as the target key point, and determine M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed, where M is a positive integer;

[0012] The processing unit is further used to determine the region representation value of each coding group area based on the thermal value corresponding to the pixel point contained in each coding group area of ​​the M coding group areas;

[0013] The processing unit is also used to determine the heat map code corresponding to the heat map to be processed according to the coordinate information of the target key point and the area representation value of each coding group area, and the heat map code is used to decode to obtain a decoded heat map and perform the target image processing task.

[0014] On the one hand, an embodiment of the present application provides an electronic device, the electronic device includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to perform the following steps:

[0015] Obtain a heat map to be processed, wherein the heat map to be processed includes a plurality of pixels, and each pixel corresponds to a heat value;

[0016] Determine the pixel point with the largest thermal value in the heat map to be processed as the target key point, and determine M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed, where M is a positive integer;

[0017] Determine the region representation value of each coding group area based on the thermal value corresponding to the pixel point contained in each coding group area of ​​the M coding group areas;

[0018] The heat map code corresponding to the heat map to be processed is determined according to the coordinate information of the target key point and the area representation value of each coding group area. The heat map code is used to decode to obtain a decoded heat map and perform the target image processing task.

[0019] On the one hand, an embodiment of the present application provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed by a processor, the computer program instructions are used to perform the following steps:

[0020] Obtain a heat map to be processed, wherein the heat map to be processed includes a plurality of pixels, and each pixel corresponds to a heat value;

[0021] Determine the pixel point with the largest thermal value in the heat map to be processed as the target key point, and determine M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed, where M is a positive integer;

[0022] Determine the region representation value of each coding group area based on the thermal value corresponding to the pixel point contained in each coding group area of ​​the M coding group areas;

[0023] The heat map code corresponding to the heat map to be processed is determined according to the coordinate information of the target key point and the area representation value of each coding group area. The heat map code is used to decode to obtain a decoded heat map and perform the target image processing task.

[0024] In one aspect, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions. When the computer instructions are executed by a processor, the method provided in the above aspect can be implemented.

[0025] By adopting the embodiment of the present application, it is possible to obtain a heat map to be processed, and determine M coding grouping areas based on the pixel dimension of the heat map to be processed with the target key point as the center, and then determine the heat map code corresponding to the heat map to be processed based on the coordinate information of the target key point and the regional characterization value of each coding grouping area. The heat map code is used to decode to obtain a decoded heat map and perform the target image processing task. When determining the heat map code, the coding grouping area is determined based on a target key point with the largest heat value as the center. Each coding grouping area can determine a regional characterization value. The influence of the incorrectly predicted heat value on the regional characterization value is small. By processing the heat map to be processed as a heat map code, the area with incorrect thermal value prediction can be filtered out, which helps to suppress the area with incorrect thermal value prediction in the heat map and improve the accuracy of the heat map used for image processing tasks. In addition, the heat map encoding is a set of data after processing the image to be processed. Compared with the heat map to be processed itself, the amount of data is greatly reduced. Therefore, when executing the target image processing task, there is no need to directly transmit the heat map. Instead, the heat map encoding is determined based on the coordinate information of the target key points and the area representation value of each encoded grouping area, which helps to reduce the amount of transmission data of the heat map. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 This is a schematic diagram of the effect of a heat map provided in an embodiment of the present application;

[0028] Figure 2 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0029] Figure 3 This is a schematic diagram of the effect of a maximum grouping distance value provided in an embodiment of the present application;

[0030] Figure 4 This is a schematic diagram of the effect of a coding grouping area provided in an embodiment of the present application;

[0031] Figure 5 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0032] Figure 6 It is a schematic diagram of a process flow of a heat map processing process provided by an embodiment of the present application;

[0033] Figure 7 is a schematic diagram of the effect of an image cropping process provided by an embodiment of the present application;

[0034] Figure 8 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0035] Fig. 9 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0037] The embodiment of the present application proposes an image processing scheme, which can obtain a heat map to be processed, and determine M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed, and then determine the heat map code corresponding to the heat map to be processed according to the coordinate information of the target key point and the regional characterization value of each coding grouping area. The heat map code is used to decode to obtain a decoded heat map and perform the target image processing task. When determining the heat map code, the coding grouping area is determined based on a target key point with the largest heat value as the center. Each coding grouping area can determine a regional characterization value. The influence of the incorrectly predicted heat value on the regional characterization value is small. By processing the heat map to be processed as a heat map code, the area with incorrect thermal value prediction can be filtered out, which helps to suppress the area with incorrect thermal value prediction in the heat map and improve the accuracy of the heat map used for image processing tasks. In addition, the heat map encoding is a set of data after processing the image to be processed. Compared with the heat map to be processed itself, the amount of data is greatly reduced. Therefore, when executing the target image processing task, there is no need to directly transmit the heat map. Instead, the heat map encoding is determined based on the coordinate information of the target key points and the area representation value of each encoded grouping area, which helps to reduce the amount of transmission data of the heat map.

[0038] Among them, the heat map is the embodiment of the density, distribution and change of data on the page, and different colors are selected to correspond to different data intervals. The heat map involved in the present application can be generated based on the original image in the human posture estimation method. For example, the human posture estimation method based on the heat map uses the coordinate point annotation data of the character to generate K (K is the number of key points of interest, such as face, elbow, knee, etc.) heat maps based on Gaussian distribution, and uses K heat maps as the supervision of the network to train the human posture estimation network. The human posture estimation network generates K key point coordinates as the prediction results of the human posture estimation network through K heat maps. The heat map generated by the human posture estimation network is often used as a training for downstream tasks together with the original image. The heat map can include multiple pixels, also called pixel points, and each pixel point corresponds to a heat value. In the heat map, if the heat value of a pixel point is larger, it means that the probability that the pixel point is a key part that needs to be predicted is greater.

[0039] For example, see Figure 1 , Figure 1 Schematic diagram of the effect of a heat map provided in the embodiment of the present application. Figure 1As shown, the heat map 10a can be generated when the target key part is identified based on the target image, and the target key part can be the left or right elbow, face, left or right knee, etc., which is not limited here. The heat map can include multiple pixels, such as each small square in the heat map 10a is a pixel, and each pixel can also be represented as a pixel point at the center of the small square. Each pixel point (pixel) has a corresponding thermal value. The larger the thermal value, the greater the probability that the pixel point (pixel) is located at the target key part. Optionally, in the heat map, the thermal value of a pixel point can be represented as the gray value corresponding to the pixel point. The larger the thermal value, the larger the gray value corresponding to the pixel point, and the darker the color of the pixel corresponding to the pixel point in the heat map. For example, the pixel point with the largest thermal value in the heat map 10a can be pixel point 100a.

[0040] It should be noted that all user data collected by this application (such as image data containing human posture) are collected with the consent and authorization of the user, and the collection, use and processing of relevant user data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, before collecting relevant data of the user and during the process of collecting relevant data of the user, this application can display a prompt message to remind the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining relevant data of the user after obtaining the confirmation operation issued by the user for the prompt message, otherwise (that is, the confirmation operation issued by the user for the prompt message is not obtained), the relevant steps of obtaining relevant data of the user are terminated, that is, the relevant data of the user is not obtained.

[0041] The technical solution of the present application can be used in electronic devices, which can be terminals, servers, or other devices for image processing, and the present application does not limit this. Optionally, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminals include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, smart speakers, smart home appliances, etc.

[0042] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0043] Based on the above description, the present application embodiment proposes an image processing method. Figure 2 , Figure 2 1 is a flow chart of an image processing method provided in an embodiment of the present application. The method can be executed by the above-mentioned electronic device. The image processing method can include the following steps.

[0044] S201, obtaining a thermodynamic map to be processed, wherein the thermodynamic map to be processed includes a plurality of pixel points, and each pixel point corresponds to a thermodynamic value.

[0045] Among them, the relevant description of the heat map to be processed can refer to the relevant description of the above-mentioned heat map, which will not be repeated here.

[0046] It is understandable that the heat map to be processed can be generated when the target key parts are identified based on the target image. Optionally, the heat map to be processed can be generated based on the above-mentioned human posture estimation network, which will not be described in detail here.

[0047] S202, determining the pixel point with the largest thermal value in the heat map to be processed as the target key point, and determining M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed, where M is a positive integer.

[0048] The target key point may be a pixel point with the largest thermal value, that is, a pixel point with the highest probability of being a key part of the target.

[0049] In a possible implementation, M coded grouping areas are determined with the target key point as the center according to the pixel dimension of the heat map to be processed, which may include: obtaining M grouping distance values, wherein the maximum grouping distance value among the M grouping distance values ​​is determined based on the pixel dimension of the heat map to be processed; any one of the M grouping distance values ​​is represented as the i-th grouping distance value, where M is a positive integer and i is an integer from 0 to M-1; with the target key point as the center and each grouping distance value as the radius, a circle corresponding to each grouping distance value is determined respectively; the area between the circle corresponding to the i-th grouping distance value and the circle corresponding to the i+1-th grouping distance value is determined as the i-th coded grouping area, and M coded grouping areas are obtained.

[0050] Wherein, the grouping distance value (also referred to as grouping distance) is used to indicate how many pixels from the target key point are used as the radius for dividing the grouping area. Optionally, in one embodiment, the difference between the i-th grouping distance value and the i+1-th grouping distance value is the target difference, that is, the target difference can be the difference between two adjacent grouping distance values, for example, the target difference can be 1, 2, etc., which is not limited here.

[0051] The maximum group distance value among the M group distance values ​​can be determined according to the pixel dimension of the heat map to be processed. It is understandable that when a circle is drawn with any point as the center and the maximum group distance value as the radius in the heat map to be processed, the circle drawn can cover the entire heat map, that is, the maximum group distance value can be determined based on the distance between the pixel points indicated by the endpoints of the diagonal line of the heat map.

[0052] In one embodiment, in a coordinate system established with the target key point as the origin, the distance from any pixel point to the target key point can be calculated according to the following formula (Formula 1).

[0053]

[0054] Where d((x,y)) represents the distance between the pixel with coordinate information (x,y) and the origin of the target key point, and (x,y) represents the x-axis coordinate value and y-axis coordinate value of the pixel in the coordinate system established with the target key point as the origin. Optionally, formula 1 is only a special case of calculating distance, and there may be other distance formulas, such as Where scale represents the scale transformation coefficient.

[0055] It is understandable that, for example, see Figure 3 , Figure 3 This is a schematic diagram of the effect of the maximum grouping distance value provided by the embodiment of the present application. If the pixel dimension of the heat map is H×W, that is, the height of the heat map has H pixels and the width has W pixels, then the distance from the top pixel to the bottom pixel in the heat map is H-1 (such as Figure 3 31a in FIG), the distance from the leftmost pixel to the rightmost pixel is W-1 (as shown in FIG. Figure 3 32a in FIG. 32). Therefore, combined with the above distance formula 1, the maximum group distance value can be expressed as Equivalent to Figure 3 It can be understood that if the target difference between any two adjacent grouping distance values ​​is 1 pixel, the number M of the generated coding grouping areas can be d max +1.

[0056] Optionally, if the value directly calculated according to the pixel dimension of the heat map is not an integer, a value greater than the calculated value and the difference between the value and the previous group distance value is the target difference value can be selected as the maximum group distance value. For example, the target difference between any two adjacent group distance values ​​is 1 pixel, and the value calculated according to the pixel dimension of the heat map is 8.60. The group distance values ​​before 8.60 can be {0,1,2,3,4,5,6,7,8}. Then, a value greater than the calculated value and the difference between the value and the previous group distance value (i.e. 8) is the target difference value (i.e. 1) can be selected as the maximum group distance value, that is, the maximum group distance value is 9.

[0057] It can be understood that, in one embodiment, the size of the group distance value is positively correlated with the value of i, that is, the larger the group distance value, the larger the value of i. For example, when the group distance value is 0, i=0, that is, the value of the 0th group distance is 0; when the group distance value is 2, i=1, that is, the value of the 1st group distance is 2; when the group distance value is 4, i=2, that is, the value of the 2nd group distance is 4, and so on. It can be understood that if the target key point is used as the center of the circle and the corresponding circle is determined with a group distance value of 0, the determined circle is also the target key point.

[0058] Here, the process of determining the M coding grouping areas is explained with a diagram, see Figure 4 , Figure 4 Schematic diagram of the effect of a coding grouping area provided by an embodiment of the present application. Figure 4 As shown, first, M group distance values ​​can be obtained, and the difference between two adjacent group distance values ​​is 1. For example, the M group distance values ​​here can be {0, 1, 2, 3, 4...d max} Then, in the heat map to be processed, the target key point 41a can be taken as the center, and the corresponding circles can be determined by the distance values ​​of each group as the radius. It can be understood that, if Figure 4 The multiple circles determined in are only an example, and more circles may be drawn to divide M coding grouping areas. The value of M can be determined based on the pixel dimension of the heat map. For example, Figure 4 The circle shown in 42a in FIG. 1 is a circle determined by the group distance value of 12 pixels as the radius. Figure 4 The circle shown as 43a in FIG. 1 is a circle whose radius is determined by the grouping distance value of 11 pixels, and the area between the circle 42a and the circle 43a can be called the 11th coding grouping area.

[0059] S203 , determining a region representation value of each coding grouping region based on the thermal values ​​corresponding to the pixels included in each coding grouping region of the M coding grouping regions.

[0060] Among them, the region characterization value is used to indicate the heat value representing a coded packet region.

[0061] In one embodiment, determining the region characterization value of each coded packet region based on the heat values corresponding to the pixel points included in each coded packet region among the M coded packet regions may include: determining the pixel values corresponding to the coordinate points included in each coded packet region in the target coordinate system corresponding to the heat map to be processed; among them, the pixel value corresponding to the coordinate point belonging to the pixel points in the heat map to be processed is the heat value of the pixel point, and the pixel value corresponding to the coordinate point not belonging to the pixel points in the heat map to be processed is zero; determining the statistical value corresponding to each coded packet region according to the pixel values corresponding to the coordinate points included in each coded packet region, and using the statistical value corresponding to each coded packet region as the region characterization value of each coded packet region respectively; among them, the statistical value is any one of the following: median, average, mode.

[0062] Among them, the target coordinate system may be a coordinate system determined based on the heat map to be processed. For example, the target coordinate system is a coordinate system established with the corner point (such as the upper left corner) of the heat map to be processed as the origin and the straight lines parallel to the two sides of the heat map to be processed as the coordinate axes. If the coordinate information of a coordinate point in the target coordinate system is (x, y), then the first coordinate value indicated by the coordinate information of the coordinate point may be x, and the second coordinate value may be y. It can be understood that in this target coordinate system, the arrangement form of the coordinate points is the same as the arrangement form of the pixel points in the heat map, and each coded packet region may include at least one coordinate point, some of which belong to the pixel points in the heat map to be processed and some do not belong to the pixel points in the heat map to be processed.

[0063] It can be understood that the pixel values corresponding to the coordinate points included in each of the above coded packet regions can be represented by the following formula (Formula 2).

[0064]

[0065] Among them, φ((x, y)) represents the pixel value corresponding to the coordinate point (x, y), F((x, y)) represents taking the heat value at the position (x, y) in the heat map to be processed; H and W respectively represent the height and width of the heat map to be processed, then 0 <= x < H, 0 <= y < W indicates that the coordinate point (x, y) belongs to the pixel points in the heat map to be processed, and the pixel values corresponding to other coordinate points are 0.

[0066] Among them, the median can be used to indicate the data that is in the middle position when a group of data is arranged in order of size, such as the pixel value that is in the middle position when the pixel values ​​corresponding to all the coordinate points contained in the j-th coding grouping area are arranged in order of size, or the pixel value that is in the middle position when the pixel values ​​corresponding to the coordinate points of the pixels of the heat map to be processed contained in the j-th coding grouping area are arranged in order of size. The mean is used to indicate the average value of a group of data, such as the average value of the pixel values ​​corresponding to all the coordinate points contained in the j-th coding grouping area, or the average value of the pixel values ​​corresponding to the coordinate points of the pixels of the heat map to be processed contained in the j-th coding grouping area. The mode is used to indicate the data that appears most frequently in a group of data, such as the pixel value that appears most frequently among the pixel values ​​corresponding to all the coordinate points contained in the j-th coding grouping area, or the pixel value that appears most frequently among the pixel values ​​corresponding to the coordinate points of the pixels of the heat map to be processed contained in the j-th coding grouping area.

[0067] For example, if the statistical value is the median, the region representation value of the jth coding group region can be expressed as ζ(j) = median({φ((x,y))}for(x,y)∈ψ j ). Where median() means taking the median, φ((x,y)) means the pixel value corresponding to the coordinate point (x,y), and the coordinate point (x,y) belongs to the set of coordinate points of the jth coding group area ψ j .

[0068] It can be understood that the statistical value corresponding to each coding grouping area is determined according to the pixel values ​​corresponding to the coordinate points contained in each coding grouping area. The statistical value corresponding to each coding grouping area can be determined according to the pixel values ​​corresponding to all the coordinate points contained in each coding grouping area, or the statistical value corresponding to each coding grouping area can be determined according to the pixel values ​​corresponding to the coordinate points of the pixels in the thermodynamic map to be processed contained in each coding grouping area. There is no limitation here. Among them, the statistical value corresponding to each coding grouping area is determined according to the pixel values ​​corresponding to the coordinate points of the pixels in the thermodynamic map to be processed contained in each coding grouping area, that is, the statistical value corresponding to each coding grouping area is determined according to the thermal values ​​of the pixels in the thermodynamic map to be processed contained in each coding grouping area. It can be understood that, in other words, if the jth coding grouping area contains pixels in the heat map to be processed, the statistical value corresponding to the jth coding grouping area can be determined according to the thermal values ​​of the pixels in the heat map to be processed contained in the jth coding grouping area, and then the statistical value corresponding to the jth coding grouping area is used as the regional representation value of the jth coding grouping area; if the jth coding grouping area does not contain pixels in the heat map to be processed, the statistical value corresponding to the jth coding grouping area can be determined according to the pixel values ​​of the coordinate points contained in the jth coding grouping area. Since the pixel values ​​of the coordinate points that do not belong to the pixel points in the heat map to be processed are 0, when the jth coding grouping area does not contain the pixel points in the heat map to be processed, the statistical value of the jth coding grouping area is also 0, and then the regional representation value of the jth coding grouping area is also 0.

[0069] S204. Determine the heat map code corresponding to the heat map to be processed according to the coordinate information of the target key point and the area representation value of each coding group area. The heat map code is used to decode to obtain a decoded heat map and perform the target image processing task.

[0070] The heat map encoding may be a set of data determined according to the coordinate information of the target key point and the region characterization value of each encoding grouping region. It can be understood that if the heat map to be processed is directly transmitted, the data transmission amount required is H×W=O(max(H,W) 2 ), and when data is transmitted based on the heat map coding, the amount of data that needs to be transmitted is only a set of data indicated by the heat map coding, which greatly reduces the amount of data transmission and improves data transmission efficiency. Optionally, in a possible implementation, the heat map coding may include a first heat map coding and a second heat map coding.

[0071] In one embodiment, the heat map code includes a first heat map code; each of the M coding grouping areas is associated with a corresponding grouping number, and the value of the grouping number is an integer from 0 to M-1; the heat map code corresponding to the heat map to be processed is determined according to the coordinate information of the target key point and the area representation value of each coding grouping area, which may include: sorting the area representation value of each coding grouping area according to the grouping number of each coding grouping area to obtain a representation value sequence; combining the representation value sequence with the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point to obtain the first heat map code corresponding to the heat map to be processed.

[0072] Among them, it can be understood that the grouping number can be equivalent to the above-mentioned j, and the regional characterization value of each coding grouping area is sorted according to the grouping number of each coding grouping area. It can be sorted in order from large to small according to the grouping number, or it can be sorted in order from small to large according to the grouping number, or it can be sorted in other fixed orders, which are not limited here. The first coordinate value and the second coordinate value indicated by the coordinate information of the target key point can be the x-axis coordinate value and the y-axis coordinate value of the target key point. The coordinate information of the target key point can be the coordinate information in the target coordinate system. For example, the target coordinate system can be a coordinate system established with the corner point (such as the upper left corner) of the heat map to be processed as the origin and the straight lines parallel to the two sides of the heat map to be processed as the coordinate axes. If the coordinate information of the target key point in the target coordinate system is (x0, y0), the first coordinate value indicated by the coordinate information of the target key point can be x0, and the second coordinate value can be y0.

[0073] The characterization value sequence is combined with the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point. The combination can be performed by placing the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point in front of the characterization value sequence, or by placing the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point in the back of the characterization value sequence. There is no limitation here.

[0074] For example, the coordinate information of the target key point is (x0, y0), and the characterization value sequence is {ζ(0), ζ(1), ζ(2)...ζ(M)}, then the first thermal map encoding obtained by combining the coordinate information of the target key point with the characterization value sequence can be: {x0, y0, ζ(0), ζ(1), ζ(2)...ζ(M)}. Optionally, the first thermal map encoding can also be: {ζ(0), ζ(1), ζ(2)...ζ(M), x0, y0,}, which is not limited here. It can be understood that if the data transmission amount required for directly transmitting the thermal map to be processed is H×W=O(max(H, W) 2), and when data is transmitted based on the first heat map encoding, the amount of data required to be transmitted can be 2+M=Ο(max(H,W)), then the amount of data required to be transmitted is reduced from the square order of magnitude to the linear order of magnitude, which greatly reduces the amount of data transmission and improves data transmission efficiency.

[0075] In one embodiment, the heat map encoding includes a second heat map encoding; each of the M coding grouping areas is associated with a corresponding grouping number, and the value of the grouping number is an integer from 0 to M-1. The heat map encoding corresponding to the thermogram to be processed is determined according to the coordinate information of the target key point and the area representation value of each coding grouping area, which may include: standardizing the area representation value of each coding grouping area to obtain a standardized representation value corresponding to each coding grouping area; determining the target variance according to the standardized representation values ​​corresponding to the M coding grouping areas and the grouping number corresponding to the M coding grouping areas; combining the target variance, the thermal value of the target key point, and the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point to obtain the second heat map encoding corresponding to the thermogram to be processed.

[0076] Among them, it can be understood that the regional characterization value corresponding to each coding grouping area can approximately satisfy the Gaussian distribution, and the thermal value of each coding grouping area on the thermal map can be characterized by the Gaussian distribution corresponding to the variance determined based on the characterization value of each coding grouping area. It can be understood that the sum of the probabilities of each point in the Gaussian distribution is 1, so the regional characterization values ​​are standardized, and the sum of the regional characterization values ​​corresponding to 0-M coding grouping areas can be adjusted to 0.5. It can be understood that the target variance can be calculated based on the variance formula corresponding to the Gaussian distribution.

[0077] The target variance, the thermal value of the target key point, and the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point are combined. The target variance, the thermal value of the target key point, and the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point can be combined in any order, and there is no limitation here. For example, the coordinate information of the target key point is (x0, y0), the thermal value of the target key point is score, and the target variance is σ. Then the second thermal map encoding obtained by the combination can be: {x0, y0, score, σ}. Optionally, the second thermal map encoding can also be: {score, σ, x0, y0}, the second thermal map encoding can also be {σ, score, x0, y0}, and so on, which are not limited here. If the data transmission amount H×W required for directly transmitting data to the thermal map to be processed is = Ο(max(H, W) 2), and when data is transmitted based on the second heat map encoding, only four data {x0, y0, score, σ} need to be transmitted, and the amount of data to be transmitted is reduced to a constant order of magnitude, further reducing the amount of data transmission and improving data transmission efficiency.

[0078] In one embodiment, if the target difference between the distance values ​​of adjacent groups is 1, then according to the above formula for calculating the maximum group distance value, d max , then the characterization processing of each region characterization value can be performed according to the following formula (Formula 3).

[0079]

[0080] Among them, p j It can be a standardized characterization value, and the value of j can be -d max to d max , then the calculated standardized characterization value can include p j The value of p -j It can be understood that the calculation of j as a negative number here is to indicate that the heat map distribution satisfies the Gaussian distribution, so as to facilitate the calculation of the target variance. In the actual division of the coding grouping area, the grouping number corresponding to the coding grouping area where the target key point with the largest thermal value is located is used as the average value of the Gaussian distribution. The standardized representation values ​​corresponding to 0-(M-1) coding grouping areas only account for half of the Gaussian distribution.

[0081] Furthermore, the target variance may be calculated based on the standardized characterization value. For example, the target variance may be calculated according to the following formula (Formula 4).

[0082]

[0083] Among them, σ represents the target variance, p j It can be a standardized characterization value, and the value of j can be -d max to d max .

[0084] It is understandable that after the heat map encoding is determined, when an image processing task needs to be performed based on the heat map in the future, the heat map encoding can be decoded to obtain a decoded heat map, and then the image processing task is performed based on the decoded heat map. The image processing task can be an intelligent cropping task for the image, a video horizontal to vertical, a banner (banner advertisement) production and other downstream businesses. Etc., no restrictions are made here. It is understandable that in some scenarios, the human posture estimation network used to generate the heat map to be processed can be separated from the downstream image processing task, and then the heat map needs to be transmitted to the device for processing the downstream image processing task after the heat map is generated for task processing. Therefore, the heat map to be processed can be processed into a heat map encoding by the above method, and then the heat map encoding can be transmitted to the device for processing the downstream image processing task. When performing the image processing task later, the heat map can be easily determined based on the heat map encoding, which greatly reduces the amount of data transmission of the heat map and reduces the bandwidth consumption caused by data transmission.

[0085] See also Figure 5 , Figure 5 1 is a flow chart of an image processing method provided in an embodiment of the present application. The method can be executed by the above-mentioned electronic device. The image processing method can include the following steps.

[0086] S501, obtaining a heat map to be processed, wherein the heat map to be processed includes a plurality of pixels, and each pixel corresponds to a heat value.

[0087] S502: Determine the pixel point with the largest thermal value in the heat map to be processed as the target key point, and determine M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed.

[0088] S503 : Determine the region representation value of each coding grouping region based on the thermal value corresponding to the pixel points contained in each coding grouping region of the M coding grouping regions.

[0089] S504, determining the heat map code corresponding to the heat map to be processed according to the coordinate information of the target key point and the area representation value of each coding group area, and the heat map code is used to decode to obtain a decoded heat map and perform the target image processing task.

[0090] Among them, steps S501-S504 can refer to the relevant description of steps S201-S204, which will not be repeated here.

[0091] S505. Determine a decoded heat map corresponding to the heat map encoding according to the heat map encoding, and the decoded heat map has the same pixel dimension as the target image.

[0092] Among them, it can be understood that the decoding heat map can be a heat map obtained by decoding based on the heat map encoding. In one embodiment, the decoding heat map may include a first decoding heat map and a second decoding heat map. The first decoding heat map can be a decoding heat map obtained based on the first heat map encoding, and the second decoding heat map can be a decoding heat map obtained based on the second heat map encoding.

[0093] In one embodiment, the decoding heat map includes a first decoding heat map, and determining the decoding heat map corresponding to the heat map encoding according to the heat map encoding may include: when a decoding instruction for the first heat map encoding is detected, obtaining a first initial decoding heat map, the first initial decoding heat map includes a plurality of pixels, and the pixel dimension of the first initial decoding heat map is the same as the pixel dimension of the heat map to be processed; further, determining the decoding key point in the first initial decoding heat map according to the coordinate information of the target key point in the first heat map encoding, and centering the decoding key point according to the pixel dimension of the first initial decoding heat map. Determine M decoding group areas, the generation method of the M decoding group areas is the same as the generation method of the M encoding group areas, and one decoding group area corresponds to one encoding group area; further, the regional representation value of each encoding group area in the representation value sequence is respectively determined as the regional representation value of the corresponding decoding group area; further, the thermal value of the pixel point in the first initial decoding heat map is determined as the regional representation value corresponding to the decoding group area to which the pixel point in the first initial decoding heat map belongs, and the first initial decoding heat map with thermal values ​​determined for all pixels is determined as the first decoding heat map.

[0094] It can be understood that the decoding instruction is used to indicate an instruction for decoding the heat map encoding. The decoding instruction can be triggered by a business object when it is necessary to perform an image processing task based on a heat map. The first initial decoded heat map can be equivalent to an initialized image of an undetermined heat map value. The pixel dimensions of the first initial decoded heat map can be the same as the pixel dimensions of the heat map to be processed. For example, if the pixel dimensions of the heat map to be processed are H×W, the pixel dimensions of the first initial decoded heat map are also H×W.

[0095] It is understandable that when decoding the first heat map code, the decoding key point is determined in the first initial decoded heat map according to the coordinate information of the target key point in the first heat map code, and the decoding key point can be determined based on the coordinate information of the target key point in the target coordinate system corresponding to the first initial decoded heat map. The target coordinate system corresponding to the first initial decoded heat map is constructed in the same way as the target coordinate system corresponding to the above-mentioned heat map to be processed. For example, the target coordinate system corresponding to the first initial decoded heat map can also be a coordinate system established with the corner point (such as the upper left corner) of the first initial decoded heat map as the origin and the straight lines parallel to the two sides of the heat map to be processed as the coordinate axes. If the coordinate information of the target key point in the target coordinate system is (x0, y0), the pixel point with the coordinate information of (x0, y0) in the target coordinate system of the first initial decoded heat map is used as the decoding key point.

[0096] The method of determining M decoding grouping areas with the decoding key point as the center according to the pixel dimension of the first initial decoding heat map is the same as the above-mentioned method of determining M decoding grouping areas with the target key point as the center, and will not be repeated here. It can be understood that the decoding grouping areas correspond one-to-one to the encoding grouping areas. For example, the 0th decoding grouping area corresponds to the 0th encoding grouping area. If the area characterization value of the 0th encoding grouping area is ζ(0), then the area characterization value of the 0th decoding grouping area is ζ(0); the 1st decoding grouping area corresponds to the 1st encoding grouping area. If the area characterization value of the 1st encoding grouping area is ζ(1), then the area characterization value of the 1st decoding grouping area is ζ(1), and so on, the area characterization value corresponding to each decoding grouping area can be obtained. It can be understood that in the first decoding heat map, the thermal value of the pixel point in any decoding grouping area can be expressed as f dec ((x s ,y t ))=ζ(j)where((x s ,y t ))∈ψ j , where f dec ((x s ,y t )) indicates that the coordinate information in the first decoded heat map is (x s ,y t ), ζ(j) represents the region representation value of the jth decoding group area, ψ j represents the set of coordinate points belonging to the jth decoding group area, that is, the thermal value of the pixel point of the jth decoding group area is equal to the regional representation value of the jth decoding group area. It can be understood that if the pixel dimension of the first initial decoding heat map is H×W, then (x s ,y t )h0≤x s<H,0≤y t <W。

[0097] For example, the 0th decoding group area may include a decoding key point, and the thermal value of the decoding key point is ζ(0); the 1st decoding group area may include pixel points e1, e2, e3, and e4, and the thermal values ​​corresponding to the pixel points e1, e2, e3, and e4 are all ζ(1); the 2nd decoding group area may include pixel points e5, e6, e7, e8, e9, e10, e11, and e12, and the thermal values ​​corresponding to the pixel points e5, e6, e7, e8, e9, e10, e11, and e12 are all ζ(2), and so on. The corresponding thermal values ​​of each pixel point in the first initial decoding thermal map can be determined, and then the first initial decoding thermal map with determined thermal values ​​for all pixel points can be determined as the first decoding thermal map.

[0098] In one embodiment, the decoding heat map includes a second decoding heat map, and determining the decoding heat map corresponding to the heat map encoding according to the heat map encoding may include: when a decoding instruction for the second heat map encoding is detected, obtaining a second initial decoding heat map, the second initial decoding heat map includes a plurality of pixels, and the pixel dimension of the second initial decoding heat map is the same as the pixel dimension of the heat map to be processed; further, determining the thermal value of each pixel point in the second initial decoding heat map according to the thermal value and target variance of the target key point in the second heat map encoding, and determining the second initial decoding heat map with thermal values ​​determined for all pixels as the second decoding heat map.

[0099] Among them, the description related to the second initial decoding heat map can refer to the description related to the above-mentioned first initial decoding heat map, which will not be repeated here.

[0100] It is understandable that the regional representation value of each decoding group area is determined according to the thermal value and target variance of the target key point in the second thermal map encoding, and can be determined according to the expression of Gaussian distribution. For example, it can be based on the following formula (formula

[0101] Formula 5) determines the thermal value of a pixel point in any decoding group area in the second decoding thermal map.

[0102]

[0103] Among them, f dec ((x s ,y t )) indicates that the coordinate information in the second decoded heat map is (x s ,y t), score is the thermal value of the target key point, x0 and y0 represent the first coordinate value and the second coordinate value in the coordinate information of the target key point, and σ is the target variance. It can be understood that if the pixel dimension of the first initial decoded thermal map is H×W, then (x s ,y t )h0≤x s <H,0≤y t <W。

[0104] It can be understood that the decoded heat map determined based on the above process gradually diffuses outward with the target key point as the center, and the heat value gradually decreases. If other pixels with higher heat values ​​that are incorrectly responded appear in the heat map to be processed, then in the process of encoding the heat map to be processed into a heat map code, and then decoding the heat map code to obtain a decoded heat map, other pixels with higher heat values ​​that are incorrectly responded can be filtered. For example, in the generated first heat map code, the higher heat value of the wrong response is comprehensively adjusted to a more normal heat value by taking a statistical value, and the second decoded heat map obtained based on this decoding no longer contains the higher heat value of the wrong response. For another example, in decoding the second heat map code to obtain the second decoded heat map, the decoded heat map can be made to satisfy the Gaussian distribution centered on the target key point, thereby filtering the heat values ​​that do not satisfy the Gaussian distribution of the wrong response in the heat map to be processed, thereby greatly improving the accuracy of the heat map used for image processing tasks.

[0105] The entire heat map processing process is explained here with diagrams, see Figure 6 , Figure 6 is a schematic diagram of a heat map processing process provided by an embodiment of the present application. Figure 6As shown, first, a heat map 61 to be processed can be generated, such as that generated based on the above-mentioned human posture estimation network. Then the heat map to be processed can be divided into M coding grouping areas 62, and the process of determining the M coding grouping areas 62 can be determined with the target key point in the heat map to be processed as the center. The specific determination method can refer to the above description, which is not repeated here. Then, the heat map encoding 63 of the heat map to be processed can be determined based on the regional representation values ​​of the M coding grouping areas and the coordinate information of the target key point, and then when the downstream image processing task is required, the heat map encoding 63 can be decoded to obtain a decoded heat map 64. Among them, the heat map encoding 63 can include a first heat map encoding 63a and a second heat map encoding 63b, the first heat map encoding 63a can be obtained by combining the coordinate information of the target key point and the regional representative value of each coding grouping area, and the second heat map encoding 63b can be obtained by combining the target variance determined based on the regional representative value of each coding grouping area, the coordinate information of the target key point and the thermal value of the target key point, and the specific steps refer to the above description. Among them, the decoding heat map 64 may include a first decoding heat map 64a and a second decoding heat map 64b. The first decoding heat map may be obtained by decoding based on the first heat map code 63a, and the second decoding heat map may be obtained by decoding based on the second heat map code 63b.

[0106] S506. In response to an image cropping instruction for a target image, determine the pixels in the decoded thermal map whose thermal values ​​are greater than or equal to a threshold value, and crop the target image based on the pixels whose thermal values ​​are greater than or equal to the threshold value to obtain a cropped image, which includes key target parts.

[0107] It is understandable that the above-mentioned target image processing task includes an image cropping task, and the heat map to be processed is generated when the target key part is identified based on the target image, and the thermal value of the pixel point in the heat map to be processed is used to characterize the probability that the position of the pixel point is the target key part. The target key part can be the face, shoulders, knees and the like, which are not limited here. It is understandable that, as mentioned above, the larger the thermal value corresponding to the pixel point in the heat map, the greater the probability that the pixel point is the target key part, and the smaller the thermal value corresponding to the pixel point in the heat map, the smaller the probability that the pixel point is the target key part.

[0108] Among them, the image cropping instruction is used to instruct the intelligent cropping of the target image. It is understandable that the present application can fuse the key parts indicated in the decoded heat map and accurately locate the position of the key parts in the target image, so that the image of the key parts can be intelligently cropped based on this, thereby avoiding the failure examples of cropping resulting in the output image without a head or a damaged body in some scenes.

[0109] It can be understood that the pixel points in the decoded thermogram whose thermal value is greater than or equal to the threshold value can be the pixel points in the decoded thermogram whose probability of being the key part of the target is greater than a certain threshold value. Then, the target image is cropped based on the pixel points whose thermal value is greater than or equal to the threshold value, and a response area can be determined for the pixel points in the decoded thermogram whose thermal value is greater than or equal to the threshold value, and then a cropping area is determined in the target image based on the response area, and the cropping area needs to completely contain the response area, and then the target image is cropped based on the cropping area to obtain a cropped image.

[0110] For example, see Figure 7 , Figure 7 Schematic diagram of the effect of an image cropping process provided by an embodiment of the present application. Figure 7 As shown, first, a heat map to be processed corresponding to the head, the key target part, can be generated based on the target image 71, and then the heat map to be processed can be encoded to obtain a heat map encoding, and then the heat map encoding can be decoded to obtain a decoded heat map. The decoded heat map obtained is as shown in FIG. Figure 7 Then, the pixels whose thermal values ​​in the decoded thermal map are greater than or equal to the threshold value can be determined, and the response area formed by the pixels whose thermal values ​​are greater than or equal to the threshold value can be as shown in FIG. Figure 7 As shown in 73a in the figure, the cropping area 73b that completely includes the response area 73a can be determined based on the response area 73a, and then the target image can be cropped to obtain the image within the cropping area 73a, that is, the cropped image 74 can be obtained. It can be seen that the cropped image 74 can completely retain the key part of the person's head.

[0111] See also Figure 8 , Figure 8 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. Optionally, the image processing device may be provided in the above-mentioned electronic device. Figure 8 As shown, the image processing device described in this embodiment may include:

[0112] An acquisition unit 801 is used to acquire a heat map to be processed, wherein the heat map to be processed includes a plurality of pixels, and each pixel corresponds to a heat value;

[0113] The processing unit 802 is used to determine the pixel point with the largest thermal value in the heat map to be processed as the target key point, and determine M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed, where M is a positive integer;

[0114] The processing unit 802 is further configured to determine the region representation value of each coding grouping region based on the thermal value corresponding to the pixel point contained in each coding grouping region of the M coding grouping regions;

[0115] The processing unit 802 is also used to determine the heat map code corresponding to the heat map to be processed according to the coordinate information of the target key point and the area representation value of each coding group area, and the heat map code is used to decode to obtain a decoded heat map and perform the target image processing task.

[0116] In one implementation, the processing unit 802 is specifically configured to:

[0117] Obtain M group distance values, wherein the maximum group distance value among the M group distance values ​​is determined based on the pixel dimension of the heat map to be processed; any group distance value among the M group distance values ​​is represented as the i-th group distance value, where M is a positive integer and i is an integer from 0 to M-1;

[0118] Taking the target key point as the center of the circle and each group distance value as the radius, respectively determine the circle corresponding to each group distance value;

[0119] The area between the circle corresponding to the i-th grouping distance value and the circle corresponding to the (i+1)-th grouping distance value is determined as the i-th coding grouping area, and the M coding grouping areas are obtained.

[0120] In one implementation, the processing unit 802 is specifically configured to:

[0121] In the target coordinate system corresponding to the heat map to be processed, determine the pixel values ​​corresponding to the coordinate points included in each coding grouping area; wherein the pixel values ​​corresponding to the coordinate points of the pixel points in the heat map to be processed are the thermal values ​​of the pixel points, and the pixel values ​​corresponding to the coordinate points that do not belong to the pixel points in the heat map to be processed are zero;

[0122] The statistical values ​​corresponding to each coding grouping area are determined according to the pixel values ​​corresponding to the coordinate points contained in each coding grouping area, and the statistical values ​​corresponding to each coding grouping area are used as the area characterization values ​​of each coding grouping area; wherein the statistical value is any one of the following: median, mean, mode.

[0123] In one implementation, the heat map encoding includes a first heat map encoding; each of the M encoding grouping areas is associated with a corresponding grouping number, and the value of the grouping number is an integer from 0 to M-1; the processing unit 802 is specifically used to:

[0124] Sort the region representation value of each coding group region according to the group sequence number of each coding group region to obtain a representation value sequence;

[0125] The characterization value sequence is combined with the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point to obtain a first thermodynamic map encoding corresponding to the thermodynamic map to be processed.

[0126] In one implementation, the decoding heat map includes a first decoding heat map, and the processing unit 802 is further configured to:

[0127] When a decoding instruction for the first heat map encoding is detected, a first initial decoded heat map is obtained, wherein the first initial decoded heat map includes a plurality of pixel points, and the pixel dimension of the first initial decoded heat map is the same as the pixel dimension of the heat map to be processed;

[0128] Determine a decoding key point in the first initial decoding heat map according to the coordinate information of the target key point in the first heat map encoding, and determine M decoding grouping areas with the decoding key point as the center according to the pixel dimension of the first initial decoding heat map, wherein the M decoding grouping areas are generated in the same manner as the M encoding grouping areas, and one decoding grouping area corresponds to one encoding grouping area;

[0129] Determine the region representation value of each encoding group region in the representation value sequence as the region representation value of the corresponding decoding group region;

[0130] The thermal value of the pixel point in the first initial decoding heat map is determined as the area representation value corresponding to the decoding group area to which the pixel point in the first initial decoding heat map belongs, and the first initial decoding heat map with thermal values ​​determined for all pixels is determined as the first decoding heat map.

[0131] In one implementation, the heat map encoding includes a second heat map encoding; each of the M encoding grouping areas is associated with a corresponding grouping number, and the value of the grouping number is an integer from 0 to M-1; the processing unit 802 is specifically used to:

[0132] Performing standardization processing on the region representation value of each coding group region to obtain a standardized representation value corresponding to each coding group region;

[0133] Determine the target variance according to the standardized representation values ​​corresponding to the M coding grouping areas and the grouping numbers corresponding to the M coding grouping areas;

[0134] The target variance, the thermal value of the target key point, and the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point are combined to obtain a second thermal map encoding corresponding to the thermal map to be processed.

[0135] In one implementation, the decoding heat map includes a second decoding heat map, and the processing unit 802 is further configured to:

[0136] When a decoding instruction for the second heat map encoding is detected, a second initial decoded heat map is obtained, wherein the second initial decoded heat map includes a plurality of pixel points, and the pixel dimension of the second initial decoded heat map is the same as the pixel dimension of the heat map to be processed;

[0137] The thermal value of each pixel in the second initial decoded thermogram is determined according to the thermal value of the target key point in the second thermogram encoding and the target variance, and the second initial decoded thermogram with thermal values ​​determined for all pixels is determined as the second decoded thermogram.

[0138] In one implementation, the target image processing task includes an image cropping task, the heat map to be processed is generated when the target key part is identified based on the target image, and the thermal value of the pixel point in the heat map to be processed is used to characterize the probability that the position of the pixel point is the target key part; the processing unit 802 is also used to:

[0139] Determining a decoded heat map corresponding to the heat map encoding according to the heat map encoding, wherein the decoded heat map has the same pixel dimension as the target image;

[0140] In response to an image cropping instruction for the target image, the pixel points in the decoded thermal map whose thermal values ​​are greater than or equal to a threshold are determined, and the target image is cropped based on the pixel points whose thermal values ​​are greater than or equal to the threshold to obtain a cropped image, which includes the target key parts.

[0141] See also Fig. 9 , Fig. 9 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device described in this embodiment includes: a processor 901 and a memory 902. Optionally, the electronic device may also include a network interface or a power supply module. The processor 901 and the memory 902 may exchange data.

[0142] The processor 901 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0143] The above-mentioned network interface may include an input device and / or an output device. For example, the input device may be a control panel, a microphone, a receiver, etc., and the output device may be a display screen, a transmitter, etc., which are not listed here one by one.

[0144] The memory 902 may include a read-only memory and a random access memory, and provide program instructions and data to the processor 901. A portion of the memory 902 may also include a non-volatile random access memory. When the processor 901 calls the program instructions, it is used to execute:

[0145] Obtain a heat map to be processed, wherein the heat map to be processed includes a plurality of pixels, and each pixel corresponds to a heat value;

[0146] Determine the pixel point with the largest thermal value in the heat map to be processed as the target key point, and determine M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed, where M is a positive integer;

[0147] Determine the region representation value of each coding group area based on the thermal value corresponding to the pixel point contained in each coding group area of ​​the M coding group areas;

[0148] The heat map code corresponding to the heat map to be processed is determined according to the coordinate information of the target key point and the area representation value of each coding group area. The heat map code is used to decode to obtain a decoded heat map and perform the target image processing task.

[0149] In one implementation, the processor 901 is specifically configured to:

[0150] Obtain M group distance values, wherein the maximum group distance value among the M group distance values ​​is determined based on the pixel dimension of the heat map to be processed; any group distance value among the M group distance values ​​is represented as the i-th group distance value, where M is a positive integer and i is an integer from 0 to M-1;

[0151] Taking the target key point as the center of the circle and each group distance value as the radius, respectively determine the circle corresponding to each group distance value;

[0152] The area between the circle corresponding to the i-th grouping distance value and the circle corresponding to the (i+1)-th grouping distance value is determined as the i-th coding grouping area, and the M coding grouping areas are obtained.

[0153] In one implementation, the processor 901 is specifically configured to:

[0154] In the target coordinate system corresponding to the heat map to be processed, determine the pixel values ​​corresponding to the coordinate points included in each coding grouping area; wherein the pixel values ​​corresponding to the coordinate points of the pixel points in the heat map to be processed are the thermal values ​​of the pixel points, and the pixel values ​​corresponding to the coordinate points that do not belong to the pixel points in the heat map to be processed are zero;

[0155] According to the pixel values ​​corresponding to the coordinate points contained in each coding grouping area, the statistical values ​​corresponding to each coding grouping area are respectively determined, and the statistical values ​​corresponding to each coding grouping area are respectively used as the area characterization values ​​of each coding grouping area; wherein the statistical value is any one of the following: median, mean, mode. In one implementation, the heat map encoding includes a first heat map encoding; each of the M coding grouping areas is associated with a corresponding grouping number, and the value of the grouping number is an integer from 0 to M-1; the processor 901 is specifically used to:

[0156] Sort the region representation value of each coding group region according to the group sequence number of each coding group region to obtain a representation value sequence;

[0157] The characterization value sequence is combined with the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point to obtain a first thermodynamic map encoding corresponding to the thermodynamic map to be processed.

[0158] In one implementation, the decoding heat map includes a first decoding heat map, and the processor 901 is further configured to:

[0159] When a decoding instruction for the first heat map encoding is detected, a first initial decoded heat map is obtained, wherein the first initial decoded heat map includes a plurality of pixel points, and the pixel dimension of the first initial decoded heat map is the same as the pixel dimension of the heat map to be processed;

[0160] Determine a decoding key point in the first initial decoding heat map according to the coordinate information of the target key point in the first heat map encoding, and determine M decoding grouping areas with the decoding key point as the center according to the pixel dimension of the first initial decoding heat map, wherein the M decoding grouping areas are generated in the same manner as the M encoding grouping areas, and one decoding grouping area corresponds to one encoding grouping area;

[0161] Determine the region representation value of each encoding group region in the representation value sequence as the region representation value of the corresponding decoding group region;

[0162] The thermal value of the pixel point in the first initial decoding heat map is determined as the area representation value corresponding to the decoding group area to which the pixel point in the first initial decoding heat map belongs, and the first initial decoding heat map with thermal values ​​determined for all pixels is determined as the first decoding heat map.

[0163] In one implementation, the heat map encoding includes a second heat map encoding; each of the M encoding grouping areas is associated with a corresponding grouping number, and the value of the grouping number is an integer from 0 to M-1; the processor 901 is specifically configured to:

[0164] Performing standardization processing on the region representation value of each coding group region to obtain a standardized representation value corresponding to each coding group region;

[0165] Determine the target variance according to the standardized representation values ​​corresponding to the M coding grouping areas and the grouping numbers corresponding to the M coding grouping areas;

[0166] The target variance, the thermal value of the target key point, and the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point are combined to obtain a second thermal map encoding corresponding to the thermal map to be processed.

[0167] In one implementation, the decoding heat map includes a second decoding heat map, and the processor 901 is further configured to:

[0168] When a decoding instruction for the second heat map encoding is detected, a second initial decoded heat map is obtained, wherein the second initial decoded heat map includes a plurality of pixel points, and the pixel dimension of the second initial decoded heat map is the same as the pixel dimension of the heat map to be processed;

[0169] The thermal value of each pixel in the second initial decoded thermogram is determined according to the thermal value of the target key point in the second thermogram encoding and the target variance, and the second initial decoded thermogram with thermal values ​​determined for all pixels is determined as the second decoded thermogram.

[0170] In one implementation, the target image processing task includes an image cropping task, the heat map to be processed is generated when the target key part is identified based on the target image, and the thermal value of the pixel point in the heat map to be processed is used to characterize the probability that the position of the pixel point is the target key part; the processor 901 is further used to:

[0171] Determining a decoded heat map corresponding to the heat map encoding according to the heat map encoding, wherein the decoded heat map has the same pixel dimension as the target image;

[0172] In response to an image cropping instruction for the target image, the pixel points in the decoded thermal map whose thermal values ​​are greater than or equal to a threshold are determined, and the target image is cropped based on the pixel points whose thermal values ​​are greater than or equal to the threshold to obtain a cropped image, which includes the target key parts.

[0173] Optionally, when the program instructions are executed by the processor, other steps of the method in the above embodiment may also be implemented, which will not be repeated here.

[0174] The present application also provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the above method, such as executing the method executed by the above electronic device, which is not repeated here.

[0175] Optionally, the storage medium involved in the present application, such as computer-readable storage medium, may be non-volatile or volatile.

[0176] Optionally, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0177] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0178] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0179] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction, and when the computer instruction is executed by a processor, some or all of the steps in the above method can be implemented. For example, the computer instruction is stored in a computer-readable storage medium. The processor of the computer device (i.e., the above-mentioned electronic device) reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction so that the computer device executes the steps performed in the embodiments of the above-mentioned methods. For example, the computer device can be a terminal, or it can be a server.

[0180] The above is a detailed introduction to an image processing method, device, electronic device and medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Obtain a heat map to be processed, wherein the heat map to be processed includes a plurality of pixels, and each pixel corresponds to a heat value; Determine the pixel point with the largest thermal value in the heat map to be processed as the target key point, and determine M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed, where M is a positive integer; Determining the regional representation value of each coding grouping area based on the thermal value corresponding to the pixel points contained in each coding grouping area of ​​the M coding grouping areas; Determining the regional representation value of each coding grouping area based on the thermal value corresponding to the pixel points contained in each coding grouping area of ​​the M coding grouping areas includes: determining the statistical value corresponding to each coding grouping area according to the thermal value corresponding to the pixel points contained in each coding grouping area, and using the statistical value corresponding to each coding grouping area as the regional representation value of each coding grouping area; The heat map code corresponding to the heat map to be processed is determined according to the coordinate information of the target key point and the area representation value of each coding group area. The heat map code is used to decode to obtain a decoded heat map and perform the target image processing task.

2. The method according to claim 1, characterized in that The step of determining M encoding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed includes: Obtain M group distance values, wherein the maximum group distance value among the M group distance values ​​is determined based on the pixel dimension of the heat map to be processed; any group distance value among the M group distance values ​​is represented as the i-th group distance value, where M is a positive integer and i is an integer from 0 to M-1; Taking the target key point as the center of the circle and each group distance value as the radius, respectively determine the circle corresponding to each group distance value; The area between the circle corresponding to the i-th grouping distance value and the circle corresponding to the (i+1)-th grouping distance value is determined as the i-th coding grouping area, and the M coding grouping areas are obtained.

3. The method according to claim 1, characterized in that The determining of the statistical value corresponding to each coding grouping area according to the thermal value corresponding to the pixel points contained in each coding grouping area includes: In the target coordinate system corresponding to the heat map to be processed, determine the pixel values ​​corresponding to the coordinate points included in each coding grouping area; wherein the pixel values ​​corresponding to the coordinate points of the pixel points in the heat map to be processed are the thermal values ​​of the pixel points, and the pixel values ​​corresponding to the coordinate points that do not belong to the pixel points in the heat map to be processed are zero; The statistical value corresponding to each coding grouping area is determined according to the pixel values ​​corresponding to the coordinate points contained in each coding grouping area, wherein the statistical value is any one of the following: median, mean, mode.

4. The method according to claim 1, characterized in that: The heat map code includes a first heat map code; each of the M code grouping areas is associated with a corresponding grouping number, and the value of the grouping number is an integer from 0 to M-1; The step of determining the heat map code corresponding to the heat map to be processed according to the coordinate information of the target key point and the region representation value of each coding group region includes: Sort the region representation value of each coding group region according to the group sequence number of each coding group region to obtain a representation value sequence; The characterization value sequence is combined with the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point to obtain a first thermodynamic map encoding corresponding to the thermodynamic map to be processed.

5. The method according to claim 4, characterized in that The decoding heat map includes a first decoding heat map, and the method further includes: When a decoding instruction for the first heat map encoding is detected, a first initial decoded heat map is obtained, wherein the first initial decoded heat map includes a plurality of pixel points, and the pixel dimension of the first initial decoded heat map is the same as the pixel dimension of the heat map to be processed; Determine a decoding key point in the first initial decoding heat map according to the coordinate information of the target key point in the first heat map encoding, and determine M decoding grouping areas with the decoding key point as the center according to the pixel dimension of the first initial decoding heat map, wherein the M decoding grouping areas are generated in the same manner as the M encoding grouping areas, and one decoding grouping area corresponds to one encoding grouping area; Determine the region representation value of each encoding group region in the representation value sequence as the region representation value of the corresponding decoding group region; The thermal value of the pixel point in the first initial decoding heat map is determined as the area representation value corresponding to the decoding group area to which the pixel point in the first initial decoding heat map belongs, and the first initial decoding heat map with thermal values ​​determined for all pixels is determined as the first decoding heat map.

6. The method according to claim 1, characterized in that The heat map code includes a second heat map code; each of the M code grouping areas is associated with a corresponding grouping number, and the value of the grouping number is an integer from 0 to M-1; The step of determining the heat map code corresponding to the heat map to be processed according to the coordinate information of the target key point and the region representation value of each coding group region includes: Performing standardization processing on the region representation value of each coding group region to obtain a standardized representation value corresponding to each coding group region; Determine the target variance according to the standardized representation values ​​corresponding to the M coding grouping areas and the grouping numbers corresponding to the M coding grouping areas; The target variance, the thermal value of the target key point, and the first coordinate value and the second coordinate value indicated by the coordinate information of the target key point are combined to obtain a second thermal map encoding corresponding to the thermal map to be processed.

7. The method according to claim 6, characterized in that The decoding heat map includes a second decoding heat map, and the method further includes: When a decoding instruction for the second heat map encoding is detected, a second initial decoded heat map is obtained, wherein the second initial decoded heat map includes a plurality of pixel points, and the pixel dimension of the second initial decoded heat map is the same as the pixel dimension of the heat map to be processed; The thermal value of each pixel in the second initial decoded thermogram is determined according to the thermal value of the target key point in the second thermogram encoding and the target variance, and the second initial decoded thermogram with thermal values ​​determined for all pixels is determined as the second decoded thermogram.

8. The method according to claim 1, characterized in that: The target image processing task includes an image cropping task, the heat map to be processed is generated when the target key part is identified based on the target image, and the thermal value of the pixel point in the heat map to be processed is used to represent the probability that the position of the pixel point is the target key part; The method further comprises: Determining a decoded heat map corresponding to the heat map encoding according to the heat map encoding, wherein the decoded heat map has the same pixel dimension as the target image; In response to an image cropping instruction for the target image, the pixel points in the decoded thermal map whose thermal values ​​are greater than or equal to a threshold are determined, and the target image is cropped based on the pixel points whose thermal values ​​are greater than or equal to the threshold to obtain a cropped image, which includes the target key parts.

9. An image processing device, characterized in that: The device comprises: An acquisition unit, used for acquiring a heat map to be processed, wherein the heat map to be processed includes a plurality of pixels, each pixel corresponding to a heat value; A processing unit, used to determine the pixel point with the largest thermal value in the heat map to be processed as the target key point, and determine M coding grouping areas with the target key point as the center according to the pixel dimension of the heat map to be processed, where M is a positive integer; The processing unit is further used to determine the regional representation value of each coding group area based on the thermal value corresponding to the pixel points contained in each coding group area of ​​the M coding group areas; the determining the regional representation value of each coding group area based on the thermal value corresponding to the pixel points contained in each coding group area of ​​the M coding group areas includes: determining the statistical value corresponding to each coding group area according to the thermal value corresponding to the pixel points contained in each coding group area, and using the statistical value corresponding to each coding group area as the regional representation value of each coding group area; The processing unit is also used to determine the heat map code corresponding to the heat map to be processed according to the coordinate information of the target key point and the area representation value of each coding group area, and the heat map code is used to decode to obtain a decoded heat map and perform the target image processing task.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 8.

Images