Image processing method, device, equipment and storage medium

Abstract

The present invention relates to artificial intelligence and provides an image processing method, apparatus, device, and storage medium. The method, in response to an image processing request, obtains an image to be processed; identifies the requested task of the image processing request; crops a region of interest image corresponding to the requested task from the image to be processed based on a preset length; performs multiple transformations on the region of interest image to obtain information of interest in the image to be processed; extracts initial feature information from the information of interest; performs dimensionality reduction processing on the initial feature information to obtain low-dimensional information; and processes the low-dimensional information based on the requested task to obtain a processing result, thereby improving the accuracy of the processing result. Furthermore, the present invention also relates to blockchain technology, and the processing result can be stored in the blockchain.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] At present, when analyzing and processing images, rectangular detection frames are usually used to crop out images of interest from images. Since the images of interest are not cropped according to the characteristics of the image processing task, it is impossible to crop out reasonable images of interest. Secondly, the information of interest is only analyzed from a single dimension, which makes it impossible to accurately generate image processing results. Summary of the Invention

[0003] In view of the above, it is necessary to provide an image processing method, apparatus, device and storage medium that can solve the technical problem of being unable to accurately generate image processing results.

[0004] In one aspect, the present invention provides an image processing method, comprising:

[0005] Responding to an image processing request, obtaining an image to be processed;

[0006] identifying a request task for the image processing request;

[0007] Cutting out an image of a region of interest corresponding to the requested task from the image to be processed based on a preset length;

[0008] Performing multiple transformations on the image of the region of interest to obtain information of interest of the image to be processed;

[0009] extracting initial feature information from the information of interest;

[0010] Performing dimensionality reduction processing on the initial feature information to obtain low-dimensional information;

[0011] The low-dimensional information is processed based on the request task to obtain a processing result.

[0012] According to a preferred embodiment of the present invention, cropping the image of the region of interest corresponding to the requested task from the image to be processed based on a preset length includes:

[0013] Identifying key point information of the image to be processed;

[0014] Calculating the target center position of the image to be processed based on the key point information;

[0015] Acquire an image shape corresponding to the requested task, and select a target length from the preset lengths based on the image shape;

[0016] The region of interest image is cropped from the image to be processed based on the target center position and the target length.

[0017] According to a preferred embodiment of the present invention, performing multiple transformations on the image of the region of interest to obtain the information of interest of the image to be processed includes:

[0018] identifying a plurality of scanning angles based on a preset number of angles;

[0019] Performing pull transformation processing on the image of the region of interest based on each scanning angle to obtain image information corresponding to each scanning angle;

[0020] Encode each image information to obtain representation information;

[0021] The plurality of representation information are combined to obtain the information of interest.

[0022] According to a preferred embodiment of the present invention, extracting initial feature information from the information of interest includes:

[0023] Performing convolution processing on the information of interest to obtain convolution information;

[0024] Performing maximum pooling processing on the convolution information to obtain first pooled information;

[0025] Performing residual processing on the first pooled information to obtain residual information;

[0026] Performing average pooling processing on the residual information to obtain second pooling information;

[0027] Performing full connection processing on the second pooled information to obtain full connection information, where the full connection information includes first dimension information and second dimension information;

[0028] Counting the amount of information of the first dimension as a first amount, and counting the amount of information of the second dimension as a second amount;

[0029] If the first number is less than a first preset threshold, generating a first feature of the initial feature information according to the first dimension information;

[0030] If the second number is less than a second preset threshold, a second feature of the initial feature information is generated according to the second dimension information and the configuration value.

[0031] According to a preferred embodiment of the present invention, if the requested task is an offset prediction task, the processing of the low-dimensional information based on the requested task to obtain a processing result includes:

[0032] Reorganize the low-dimensional information to obtain first reorganized information, where the first reorganized information includes first information, second information, third information, and fourth information;

[0033] Calculating a rotation angle of a first image object in the image to be processed according to an initial angle generated by the first information and a correction angle generated by the second information;

[0034] generating offset information of the target center position according to the third information, the fourth information and the size information of the image of the region of interest;

[0035] The processing result is generated according to the rotation angle and the offset information.

[0036] According to a preferred embodiment of the present invention, if the requested task is a face recognition task, then the processing of the low-dimensional information based on the requested task to obtain a processing result further includes:

[0037] Performing a cutting and reorganization process on the low-dimensional information according to the cutting value identified by the low-dimensional information to obtain intermediate feature information;

[0038] Performing full connection processing on the intermediate feature information to obtain target feature information;

[0039] The target feature information is identified and matched based on a facial feature library to obtain the processing result.

[0040] According to a preferred embodiment of the present invention, if the requested task is a target detection task, then the processing of the low-dimensional information based on the requested task to obtain a processing result further includes:

[0041] Performing convolution processing on the low-dimensional information based on a first preset convolution layer to obtain first convolution information;

[0042] performing activation processing on each convolution element in the first convolution information to obtain a convolution activation value corresponding to each convolution element, and determining a position of a convolution element having a convolution activation value greater than a target preset threshold as an object center position of a second image object in the image to be processed;

[0043] Performing convolution processing on the low-dimensional information based on a second preset convolution layer to obtain second convolution information, and performing convolution processing on the low-dimensional information based on a third preset convolution layer to obtain third convolution information;

[0044] Identifying position information of the second image object in the image to be processed from the second convolution information according to the center position of the object;

[0045] predicting an object category to which the second image object belongs from the third convolution information according to the center position of the object;

[0046] The processing result is generated according to the object center position, the position information and the object category.

[0047] On the other hand, the present invention further provides an image processing device, comprising:

[0048] an acquiring unit, configured to acquire an image to be processed in response to an image processing request;

[0049] an identification unit, configured to identify a request task of the image processing request;

[0050] a cropping unit, configured to crop an image of a region of interest corresponding to the requested task from the image to be processed based on a preset length;

[0051] a transform unit, configured to perform multiple transform processes on the image of the region of interest to obtain information of interest of the image to be processed;

[0052] an extraction unit, configured to extract initial feature information from the information of interest;

[0053] A dimensionality reduction unit, configured to perform dimensionality reduction processing on the initial feature information to obtain low-dimensional information;

[0054] A processing unit is used to process the low-dimensional information based on the request task to obtain a processing result.

[0055] On the other hand, the present invention further provides an electronic device, comprising:

[0056] a memory storing computer-readable instructions; and

[0057] A processor executes the computer-readable instructions stored in the memory to implement the image processing method.

[0058] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the image processing method.

[0059] It can be seen from the above technical solution that the present invention crops the image to be processed based on the preset length and the requested task, and can reasonably crop the image of the region of interest to avoid analyzing redundant information in the image to be processed, thereby improving the generation efficiency of the processing result. At the same time, it can crop the image of the region of interest covering all features in the image to be processed to avoid the loss of features in the image to be processed, thereby improving the generation accuracy of the processing result. Furthermore, by performing multiple transformation processing on the image of the region of interest, information of interest containing multi-dimensional information can be generated, thereby further improving the generation accuracy of the processing result. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of a preferred embodiment of the image processing method of the present invention.

[0061] Figure 2 It is a functional module diagram of a preferred embodiment of the image processing device of the present invention.

[0062] Figure 3 It is a structural diagram of an electronic device according to a preferred embodiment of the present invention for implementing the image processing method. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0064] like Figure 1 FIG. 1 is a flow chart of a preferred embodiment of the image processing method of the present invention. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0065] The image processing method can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0066] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0067] The image processing method is applied to one or more electronic devices, which are devices that can automatically perform numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Their hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0068] The electronic device can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0069] The electronic device may include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network electronic device, a group of electronic devices consisting of multiple network electronic devices, or a cloud based on cloud computing consisting of a large number of hosts or network electronic devices.

[0070] The network where the electronic device is located includes, but is not limited to: the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0071] 101. In response to an image processing request, obtain an image to be processed.

[0072] In at least one embodiment of the present invention, the image processing request may be a request for predicting the offset of the image, wherein the offset may be the rotation angle of the object in the image and the offset of the center point of the object. The image processing request may also be a request for recognizing a face in the image. The image processing request may also be a request for predicting the position and category of the object in the image.

[0073] The image to be processed is an image acquired based on the image processing request.

[0074] In at least one embodiment of the present invention, the electronic device obtaining the image to be processed in response to the image processing request includes:

[0075] Parsing the request message of the image processing request to obtain data information carried in the request message;

[0076] extracting an image path from the data information;

[0077] The image to be processed is obtained from the image path.

[0078] By directly parsing the request message, the efficiency of generating the data information can be improved, thereby improving the efficiency of acquiring the image to be processed.

[0079] 102. Identify a request task of the image processing request.

[0080] In at least one embodiment of the present invention, the requested task may include, but is not limited to: an offset prediction task, a face recognition task, and the target detection task.

[0081] In at least one embodiment of the present invention, the electronic device identifying the request task of the image processing request includes:

[0082] extracting processing requirements from the image processing request;

[0083] The requested task is identified based on a degree of matching between the processing requirement and a preset task.

[0084] The matching degree may be the similarity between the processing requirement and the preset task.

[0085] Through the above implementation, the requested task can be accurately identified.

[0086] 103 : Crop an image of a region of interest corresponding to the requested task from the image to be processed based on a preset length.

[0087] In at least one embodiment of the present invention, if the request task of the image processing request is a task related to a face, the image shape of the area of ​​interest image is circular; if the request task of the image processing request is a task related to a car, the image shape of the area of ​​interest image is rectangular.

[0088] In at least one embodiment of the present invention, the electronic device cutting out the region of interest image corresponding to the requested task from the image to be processed based on a preset length includes:

[0089] Identifying key point information of the image to be processed;

[0090] Calculating the target center position of the image to be processed based on the key point information;

[0091] Acquire an image shape corresponding to the requested task, and select a target length from the preset lengths based on the image shape;

[0092] The region of interest image is cropped from the image to be processed based on the target center position and the target length.

[0093] The key point information is generated by analyzing the image to be processed based on a pre-trained key point recognition model, and the key point recognition model is generated by training annotated images in the same field as the image to be processed.

[0094] The target center position refers to the position information of the center key point in the key point information.

[0095] The target length is set according to the request task of the image processing request. For example, if the request task is a task related to the face, the target length is set to the radius of the circle, wherein the specific radius value can be determined according to the face length of the big data user. For example, the target length can be 5 cm.

[0096] The target center position can be accurately identified through the key point information, and the image of the region of interest can be accurately cropped in combination with the target center position and the preset length.

[0097] Specifically, the electronic device calculating the target center position of the image to be processed based on the key point information includes:

[0098] For each key point information, measure the key point distance from the remaining key point information to the key point information;

[0099] If the difference between any two distances among the plurality of key point distances is less than a preset difference, the key point information is determined as the central key point;

[0100] The coordinate information of the central key point in the image to be processed is identified as the target center position.

[0101] By performing pairwise measurements on any two key points in the key point information, the target center position can be accurately identified.

[0102] 104 , performing multiple transformation processes on the image of the region of interest to obtain information of interest of the image to be processed.

[0103] In at least one embodiment of the present invention, the information of interest includes information generated after the image of the region of interest is transformed multiple times.

[0104] In at least one embodiment of the present invention, the electronic device performs multiple transformations on the image of the region of interest to obtain the information of interest of the image to be processed, including:

[0105] identifying a plurality of scanning angles based on a preset number of angles;

[0106] Performing pull transformation processing on the image of the region of interest based on each scanning angle to obtain image information corresponding to each scanning angle;

[0107] Encode each image information to obtain representation information;

[0108] The plurality of representation information are combined to obtain the information of interest.

[0109] The number of preset angles can be set according to actual needs, for example, the number of preset angles can be 6. If the number of preset angles is 6, the expression form of the information of interest is: 6×M1×C1.

[0110] By performing pull transformation processing on the region of interest image at multiple scanning angles, information of interest corresponding to the region of interest images at multiple angles can be generated, thereby improving the comprehensiveness of the information of interest.

[0111] 105 : Extract initial feature information from the information of interest.

[0112] In at least one embodiment of the present invention, the initial feature information refers to feature information in the information of interest.

[0113] In at least one embodiment of the present invention, the electronic device extracting the initial feature information from the information of interest includes:

[0114] Performing convolution processing on the information of interest to obtain convolution information;

[0115] Performing maximum pooling processing on the convolution information to obtain first pooled information;

[0116] Performing residual processing on the first pooled information to obtain residual information;

[0117] Performing average pooling processing on the residual information to obtain second pooling information;

[0118] Performing full connection processing on the second pooled information to obtain full connection information, where the full connection information includes first dimension information and second dimension information;

[0119] Counting the amount of information of the first dimension as a first amount, and counting the amount of information of the second dimension as a second amount;

[0120] If the first number is less than a first preset threshold, generating a first feature of the initial feature information according to the first dimension information;

[0121] If the second number is less than a second preset threshold, a second feature of the initial feature information is generated according to the second dimension information and the configuration value.

[0122] The first preset threshold, the second preset threshold and the configuration value can be set according to actual needs.

[0123] Specifically, the electronic device generates the first feature of the initial feature information based on the first dimension information. For example, if the first dimension information is 012345, the first number is 6, and if the first preset threshold is 18, the first feature is 012345012345012345.

[0124] Specifically, the electronic device generates a second feature of the initial feature information based on the second dimension information and the configuration value. For example, if the second dimension information is 3456, the second number is 4, if the second preset threshold is 8, and the configuration value is 0, the second feature is 34560000.

[0125] By controlling the generation of the first feature by the first preset threshold and controlling the generation of the second feature by the second preset threshold, the comprehensiveness of the initial feature information can be ensured.

[0126] 106. Perform dimensionality reduction processing on the initial feature information to obtain low-dimensional information.

[0127] In at least one embodiment of the present invention, the low-dimensional information refers to information generated by performing convolution feature extraction on the initial feature information. If the image processing request is an object detection task, the amount of information in the row matrix of the low-dimensional information is the width of the image to be processed, and the amount of information in the vertical matrix of the low-dimensional information is the height of the image to be processed.

[0128] 107 : Process the low-dimensional information based on the requested task to obtain a processing result.

[0129] In at least one embodiment of the present invention, if the requested task is the offset prediction task, the processing result includes the rotation angle of the first image object in the image to be processed and the offset information of the target center position, wherein the offset information includes the offset of the target center position in the x-direction and the offset of the target center position in the y-direction. If the requested task is the face recognition task, the processing result includes the matching status of the image to be processed and the face feature library. If the requested task is the target detection task, the processing result includes the object center position of the second image object in the image to be processed, the position information of the second image object in the image to be processed, and the object category to which the second image object belongs.

[0130] In at least one embodiment of the present invention, if the requested task is an offset prediction task, then processing the low-dimensional information based on the requested task to obtain a processing result includes:

[0131] Reorganize the low-dimensional information to obtain first reorganized information, where the first reorganized information includes first information, second information, third information, and fourth information;

[0132] Calculating a rotation angle of a first image object in the image to be processed according to an initial angle generated by the first information and a correction angle generated by the second information;

[0133] generating offset information of the target center position according to the third information, the fourth information and the size information of the image of the region of interest;

[0134] The processing result is generated according to the rotation angle and the offset information.

[0135] The size information includes the region width and region height of the region of interest image.

[0136] By correcting the initial angle with the correction angle, the accuracy of the rotation angle can be improved. By predicting the offset information with the size information and analyzing it in combination with the size information, the accuracy of the offset information can be improved.

[0137] Specifically, the calculation formula of the initial angle is:

[0138]

[0139] z4=argmax(y3);

[0140] y3=softmax(y 21 );

[0141] Wherein, θ1 represents the initial angle, N represents the number of preset angles, and y 21 Represents the first information, and the value range of z4 is an integer in [0, N-1].

[0142] Specifically, the calculation formula of the correction angle is:

[0143]

[0144] Wherein, θ2 represents the correction angle, N represents the number of preset angles, and y 22 Represents the second information, softmax(y 22 )∈[-1,1].

[0145] Specifically, the calculation formula of the rotation angle is: θ final =θ1+θ2, where θ final represents the rotation angle.

[0146] Specifically, the calculation formula of the offset information is:

[0147] x final =y6×kdx;

[0148] y6=sigmoid(y 23 );

[0149]

[0150] y final =y7×kdy;

[0151] y7=sigmoid(y 24 );

[0152]

[0153] Among them, x final represents the offset information in the x direction, k is a preset constant, W represents the area width, N represents the number of preset angles, y final represents the offset information in the y direction, and H represents the height of the area.

[0154] In at least one embodiment of the present invention, if the requested task is a face recognition task, the electronic device processes the low-dimensional information based on the requested task, and obtaining the processing result further includes:

[0155] Performing a cutting and reorganization process on the low-dimensional information according to the cutting value identified by the low-dimensional information to obtain intermediate feature information;

[0156] Performing full connection processing on the intermediate feature information to obtain target feature information;

[0157] The target feature information is identified and matched based on a facial feature library to obtain the processing result.

[0158] By performing a cutting and reorganizing process on the low-dimensional information using the cutting value, it is possible to avoid neglecting key information in the image to be processed, thereby improving the recognition accuracy of the processing result.

[0159] Specifically, the electronic device performs a cutting and reassembly process on the low-dimensional information according to the cutting value identified by the low-dimensional information, and obtains intermediate feature information including:

[0160] extracting a target matrix from the low-dimensional information;

[0161] Performing activation processing on each matrix element in the target matrix to obtain multiple activation values;

[0162] Extracting the cut value from the plurality of activation values ​​based on an argmax function;

[0163] Cutting the low-dimensional information based on the cutting value to obtain a first matrix and a second matrix, wherein a matrix position of the first matrix in the low-dimensional information is smaller than a matrix position of the second matrix in the low-dimensional information;

[0164] The second matrix and the first matrix are concatenated to obtain the intermediate feature information.

[0165] Through the above implementation, it is possible to achieve the pre-positioning of key information in the image to be processed, thereby improving the rationality of generating the intermediate feature information.

[0166] In at least one embodiment of the present invention, if the requested task is a target detection task, then processing the low-dimensional information based on the requested task to obtain a processing result further includes:

[0167] Performing convolution processing on the low-dimensional information based on a first preset convolution layer to obtain first convolution information;

[0168] performing activation processing on each convolution element in the first convolution information to obtain a convolution activation value corresponding to each convolution element, and determining a position of a convolution element having a convolution activation value greater than a target preset threshold as an object center position of a second image object in the image to be processed;

[0169] Performing convolution processing on the low-dimensional information based on a second preset convolution layer to obtain second convolution information, and performing convolution processing on the low-dimensional information based on a third preset convolution layer to obtain third convolution information;

[0170] Identifying position information of the second image object in the image to be processed from the second convolution information according to the center position of the object;

[0171] predicting an object category to which the second image object belongs from the third convolution information according to the center position of the object;

[0172] The processing result is generated according to the object center position, the position information and the object category.

[0173] Among them, the first preset convolution layer, the second preset convolution layer, the third preset convolution layer and the target preset threshold can be set according to actual needs. For example, the first preset convolution layer can be 1*1*C3*1, the second preset convolution layer can be 1*1*C3*2, and the third preset convolution layer can be 1*1*C3*C4.

[0174] Specifically, the calculation formula of the position information is:

[0175]

[0176]

[0177] Among them, x ij 、y ij Represents the position information, i, j represents the center position of the object, y ij0 Represents the first row matrix information of the second convolution information, y ij1 The second row matrix information representing the second convolution information, a represents the width of the image to be processed, and b represents the height of the image to be processed.

[0178] Specifically, the width of the second image object is equal to the number of rows of the second convolution information, and the height of the second image object is equal to the number of columns of the second convolution information.

[0179] Specifically, the object category is generated by predicting the third convolution information based on a sigmoid function.

[0180] It should be emphasized that in order to further ensure the privacy and security of the above processing results, the above processing results can also be stored in a blockchain node.

[0181] It can be seen from the above technical solution that the present invention crops the image to be processed based on the preset length and the requested task, and can reasonably crop the image of the region of interest to avoid analyzing redundant information in the image to be processed, thereby improving the generation efficiency of the processing result. At the same time, it can crop the image of the region of interest covering all features in the image to be processed to avoid the loss of features in the image to be processed, thereby improving the generation accuracy of the processing result. Furthermore, by performing multiple transformation processing on the image of the region of interest, information of interest containing multi-dimensional information can be generated, thereby further improving the generation accuracy of the processing result.

[0182] like Figure 2, which is a functional block diagram of a preferred embodiment of an image processing device according to the present invention. The image processing device 11 includes an acquisition unit 110, a cropping unit 111, a transformation unit 112, an extraction unit 113, a dimensionality reduction unit 114, a recognition unit 115, and a processing unit 116. As used herein, a module / unit refers to a series of computer-readable instruction segments that can be acquired by the processor 13 and perform a fixed function, and is stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0183] The acquiring unit 110 acquires an image to be processed in response to an image processing request.

[0184] In at least one embodiment of the present invention, the image processing request may be a request for predicting the offset of the image, wherein the offset may be the rotation angle of the object in the image and the offset of the center point of the object. The image processing request may also be a request for recognizing a face in the image. The image processing request may also be a request for predicting the position and category of the object in the image.

[0185] The image to be processed is an image acquired based on the image processing request.

[0186] In at least one embodiment of the present invention, the acquiring unit 110 acquires the image to be processed in response to the image processing request, including:

[0187] Parsing the request message of the image processing request to obtain data information carried in the request message;

[0188] extracting an image path from the data information;

[0189] The image to be processed is obtained from the image path.

[0190] By directly parsing the request message, the efficiency of generating the data information can be improved, thereby improving the efficiency of acquiring the image to be processed.

[0191] The identification unit 115 identifies a request task of the image processing request.

[0192] In at least one embodiment of the present invention, the requested task may include, but is not limited to: an offset prediction task, a face recognition task, and the target detection task.

[0193] In at least one embodiment of the present invention, the identifying unit 115 identifies the request task of the image processing request including:

[0194] extracting processing requirements from the image processing request;

[0195] The requested task is identified based on a degree of matching between the processing requirement and a preset task.

[0196] The matching degree may be the similarity between the processing requirement and the preset task.

[0197] Through the above implementation, the requested task can be accurately identified.

[0198] The cropping unit 111 crops an image of a region of interest corresponding to the requested task from the image to be processed based on a preset length.

[0199] In at least one embodiment of the present invention, if the request task of the image processing request is a task related to a face, the image shape of the area of ​​interest image is circular; if the request task of the image processing request is a task related to a car, the image shape of the area of ​​interest image is rectangular.

[0200] In at least one embodiment of the present invention, the cropping unit 111 crops the image of the region of interest corresponding to the requested task from the image to be processed based on a preset length, including:

[0201] Identifying key point information of the image to be processed;

[0202] Calculating the target center position of the image to be processed based on the key point information;

[0203] Acquire an image shape corresponding to the requested task, and select a target length from the preset lengths based on the image shape;

[0204] The region of interest image is cropped from the image to be processed based on the target center position and the target length.

[0205] The key point information is generated by analyzing the image to be processed based on a pre-trained key point recognition model, and the key point recognition model is generated by training annotated images in the same field as the image to be processed.

[0206] The target center position refers to the position information of the center key point in the key point information.

[0207] The target length is set according to the request task of the image processing request. For example, if the request task is a task related to the face, the target length is set to the radius of the circle, wherein the specific radius value can be determined according to the face length of the big data user. For example, the target length can be 5 cm.

[0208] The target center position can be accurately identified through the key point information, and the image of the region of interest can be accurately cropped in combination with the target center position and the preset length.

[0209] Specifically, the cropping unit 111 calculates the target center position of the image to be processed based on the key point information, including:

[0210] For each key point information, measure the key point distance from the remaining key point information to the key point information;

[0211] If the difference between any two distances among the plurality of key point distances is less than a preset difference, the key point information is determined as the central key point;

[0212] The coordinate information of the central key point in the image to be processed is identified as the target center position.

[0213] By performing pairwise measurements on any two key points in the key point information, the target center position can be accurately identified.

[0214] The transformation unit 112 performs multiple transformation processes on the region of interest image to obtain the information of interest of the image to be processed.

[0215] In at least one embodiment of the present invention, the information of interest includes information generated after the image of the region of interest is transformed multiple times.

[0216] In at least one embodiment of the present invention, the transformation unit 112 performs multiple transformations on the region of interest image to obtain the information of interest of the image to be processed, including:

[0217] identifying a plurality of scanning angles based on a preset number of angles;

[0218] Performing pull transformation processing on the image of the region of interest based on each scanning angle to obtain image information corresponding to each scanning angle;

[0219] Encode each image information to obtain representation information;

[0220] The plurality of representation information are combined to obtain the information of interest.

[0221] The number of preset angles can be set according to actual needs, for example, the number of preset angles can be 6. If the number of preset angles is 6, the expression form of the information of interest is: 6×M1×C1.

[0222] By performing pull transformation processing on the region of interest image at multiple scanning angles, information of interest corresponding to the region of interest images at multiple angles can be generated, thereby improving the comprehensiveness of the information of interest.

[0223] The extraction unit 113 extracts initial feature information from the information of interest.

[0224] In at least one embodiment of the present invention, the initial feature information refers to feature information in the information of interest.

[0225] In at least one embodiment of the present invention, the extracting unit 113 extracts the initial feature information from the information of interest, including:

[0226] Performing convolution processing on the information of interest to obtain convolution information;

[0227] Performing maximum pooling processing on the convolution information to obtain first pooled information;

[0228] Performing residual processing on the first pooled information to obtain residual information;

[0229] Performing average pooling processing on the residual information to obtain second pooling information;

[0230] Performing full connection processing on the second pooled information to obtain full connection information, where the full connection information includes first dimension information and second dimension information;

[0231] Counting the amount of information of the first dimension as a first amount, and counting the amount of information of the second dimension as a second amount;

[0232] If the first number is less than a first preset threshold, generating a first feature of the initial feature information according to the first dimension information;

[0233] If the second number is less than a second preset threshold, a second feature of the initial feature information is generated according to the second dimension information and the configuration value.

[0234] The first preset threshold, the second preset threshold and the configuration value can be set according to actual needs.

[0235] Specifically, the extraction unit 113 generates the first feature of the initial feature information according to the first dimension information. For example, if the first dimension information is 012345, the first number is 6, and if the first preset threshold is 18, the first feature is 012345012345012345.

[0236] Specifically, the extraction unit 113 generates the second feature of the initial feature information based on the second dimension information and the configuration value. For example, if the second dimension information is 3456, the second number is 4, if the second preset threshold is 8, and the configuration value is 0, the second feature is 34560000.

[0237] By controlling the generation of the first feature by the first preset threshold and controlling the generation of the second feature by the second preset threshold, the comprehensiveness of the initial feature information can be ensured.

[0238] The dimensionality reduction unit 114 performs dimensionality reduction processing on the initial feature information to obtain low-dimensional information.

[0239] In at least one embodiment of the present invention, the low-dimensional information refers to information generated by performing convolution feature extraction on the initial feature information. If the image processing request is an object detection task, the amount of information in the row matrix of the low-dimensional information is the width of the image to be processed, and the amount of information in the vertical matrix of the low-dimensional information is the height of the image to be processed.

[0240] The processing unit 116 processes the low-dimensional information based on the request task to obtain a processing result.

[0241] In at least one embodiment of the present invention, if the requested task is the offset prediction task, the processing result includes the rotation angle of the first image object in the image to be processed and the offset information of the target center position, wherein the offset information includes the offset of the target center position in the x-direction and the offset of the target center position in the y-direction. If the requested task is the face recognition task, the processing result includes the matching status of the image to be processed and the face feature library. If the requested task is the target detection task, the processing result includes the object center position of the second image object in the image to be processed, the position information of the second image object in the image to be processed, and the object category to which the second image object belongs.

[0242] In at least one embodiment of the present invention, if the requested task is an offset prediction task, the processing unit 116 processes the low-dimensional information based on the requested task, and the processing result obtained includes:

[0243] Reorganize the low-dimensional information to obtain first reorganized information, where the first reorganized information includes first information, second information, third information, and fourth information;

[0244] Calculating a rotation angle of a first image object in the image to be processed according to an initial angle generated by the first information and a correction angle generated by the second information;

[0245] generating offset information of the target center position according to the third information, the fourth information and the size information of the image of the region of interest;

[0246] The processing result is generated according to the rotation angle and the offset information.

[0247] The size information includes the region width and region height of the region of interest image.

[0248] By correcting the initial angle with the correction angle, the accuracy of the rotation angle can be improved. By predicting the offset information with the size information and analyzing it in combination with the size information, the accuracy of the offset information can be improved.

[0249] Specifically, the calculation formula of the initial angle is:

[0250]

[0251] z4=argmax(y3);

[0252] y3=softmax(y 21 );

[0253] Wherein, θ1 represents the initial angle, N represents the number of preset angles, and y 21 Represents the first information, and the value range of z4 is an integer in [0, N-1].

[0254] Specifically, the calculation formula of the correction angle is:

[0255]

[0256] Wherein, θ2 represents the correction angle, N represents the number of preset angles, and y 22 Represents the second information, softmax(y 22 )∈[-1,1].

[0257] Specifically, the calculation formula of the rotation angle is: θ final =θ1+θ2, where θ final represents the rotation angle.

[0258] Specifically, the calculation formula of the offset information is:

[0259] x final =y6×kdx;

[0260] y6=sigmoid(y 23 );

[0261]

[0262] y final =y7×kdy;

[0263] y7=sigmoid(y 24 );

[0264]

[0265] Among them, x final represents the offset information in the x direction, k is a preset constant, W represents the area width, N represents the number of preset angles, y final represents the offset information in the y direction, and H represents the height of the area.

[0266] In at least one embodiment of the present invention, if the requested task is a face recognition task, the processing unit 116 processes the low-dimensional information based on the requested task, and obtaining the processing result further includes:

[0267] Performing a cutting and reorganization process on the low-dimensional information according to the cutting value identified by the low-dimensional information to obtain intermediate feature information;

[0268] Performing full connection processing on the intermediate feature information to obtain target feature information;

[0269] The target feature information is identified and matched based on a facial feature library to obtain the processing result.

[0270] By performing a cutting and reorganizing process on the low-dimensional information using the cutting value, it is possible to avoid neglecting key information in the image to be processed, thereby improving the recognition accuracy of the processing result.

[0271] Specifically, the processing unit 116 performs a cutting and reassembly process on the low-dimensional information according to the cutting value identified by the low-dimensional information, and obtains intermediate feature information including:

[0272] extracting a target matrix from the low-dimensional information;

[0273] Performing activation processing on each matrix element in the target matrix to obtain multiple activation values;

[0274] Extracting the cut value from the plurality of activation values ​​based on an argmax function;

[0275] Cutting the low-dimensional information based on the cutting value to obtain a first matrix and a second matrix, wherein a matrix position of the first matrix in the low-dimensional information is smaller than a matrix position of the second matrix in the low-dimensional information;

[0276] The second matrix and the first matrix are concatenated to obtain the intermediate feature information.

[0277] Through the above implementation, it is possible to achieve the pre-positioning of key information in the image to be processed, thereby improving the rationality of generating the intermediate feature information.

[0278] In at least one embodiment of the present invention, if the requested task is a target detection task, the processing unit 116 processes the low-dimensional information based on the requested task, and obtaining the processing result further includes:

[0279] Performing convolution processing on the low-dimensional information based on a first preset convolution layer to obtain first convolution information;

[0280] performing activation processing on each convolution element in the first convolution information to obtain a convolution activation value corresponding to each convolution element, and determining a position of a convolution element having a convolution activation value greater than a target preset threshold as an object center position of a second image object in the image to be processed;

[0281] Performing convolution processing on the low-dimensional information based on a second preset convolution layer to obtain second convolution information, and performing convolution processing on the low-dimensional information based on a third preset convolution layer to obtain third convolution information;

[0282] Identifying position information of the second image object in the image to be processed from the second convolution information according to the center position of the object;

[0283] predicting an object category to which the second image object belongs from the third convolution information according to the center position of the object;

[0284] The processing result is generated according to the object center position, the position information and the object category.

[0285] Among them, the first preset convolution layer, the second preset convolution layer, the third preset convolution layer and the target preset threshold can be set according to actual needs. For example, the first preset convolution layer can be 1*1*C3*1, the second preset convolution layer can be 1*1*C3*2, and the third preset convolution layer can be 1*1*C3*C4.

[0286] Specifically, the calculation formula of the position information is:

[0287]

[0288]

[0289] Among them, x ij 、y ijRepresents the position information, i, j represents the center position of the object, y ij0 Represents the first row matrix information of the second convolution information, y ij1 The second row matrix information representing the second convolution information, a represents the width of the image to be processed, and b represents the height of the image to be processed.

[0290] Specifically, the width of the second image object is equal to the number of rows of the second convolution information, and the height of the second image object is equal to the number of columns of the second convolution information.

[0291] Specifically, the object category is generated by predicting the third convolution information based on a sigmoid function.

[0292] It should be emphasized that in order to further ensure the privacy and security of the above processing results, the above processing results can also be stored in a blockchain node.

[0293] It can be seen from the above technical solution that the present invention crops the image to be processed based on the preset length and the requested task, and can reasonably crop the image of the region of interest to avoid analyzing redundant information in the image to be processed, thereby improving the generation efficiency of the processing result. At the same time, it can crop the image of the region of interest covering all features in the image to be processed to avoid the loss of features in the image to be processed, thereby improving the generation accuracy of the processing result. Furthermore, by performing multiple transformation processing on the image of the region of interest, information of interest containing multi-dimensional information can be generated, thereby further improving the generation accuracy of the processing result.

[0294] like Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device according to a preferred embodiment of the present invention for implementing the image processing method.

[0295] In one embodiment of the present invention, the electronic device 1 includes, but is not limited to, a memory 12 , a processor 13 , and computer-readable instructions stored in the memory 12 and executable on the processor 13 , such as an image processing program.

[0296] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. The electronic device 1 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 1 may also include input and output devices, network access devices, buses, etc.

[0297] The processor 13 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 13 is the computing core and control center of the electronic device 1, connecting various parts of the entire electronic device 1 using various interfaces and lines, and executing the operating system of the electronic device 1 as well as various installed applications, program codes, etc.

[0298] Exemplarily, the computer-readable instructions may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to implement the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which describe the execution process of the computer-readable instructions in the electronic device 1. For example, the computer-readable instructions may be divided into an acquisition unit 110, a cropping unit 111, a transformation unit 112, an extraction unit 113, a dimensionality reduction unit 114, a recognition unit 115, and a processing unit 116.

[0299] The memory 12 can be used to store the computer-readable instructions and / or modules. The processor 13 implements the various functions of the electronic device 1 by running or executing the computer-readable instructions and / or modules stored in the memory 12 and calling the data stored in the memory 12. The memory 12 can mainly include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the electronic device. The memory 12 can include non-volatile and volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, at least one disk storage device, a flash memory device, or other storage devices.

[0300] The memory 12 may be an external memory and / or an internal memory of the electronic device 1. Furthermore, the memory 12 may be a physical memory, such as a memory stick, a TF card (Trans-flash Card), and the like.

[0301] If the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the processes in the above-mentioned embodiment methods by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed by a processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0302] The computer-readable instructions include computer-readable instruction codes, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer-readable instruction codes, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), or a random access memory (RAM).

[0303] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0304] Combine Figure 1 The memory 12 in the electronic device 1 stores computer-readable instructions to implement an image processing method, and the processor 13 can execute the computer-readable instructions to implement:

[0305] Responding to an image processing request, obtaining an image to be processed;

[0306] identifying a request task for the image processing request;

[0307] Cutting out an image of a region of interest corresponding to the requested task from the image to be processed based on a preset length;

[0308] Performing multiple transformations on the image of the region of interest to obtain information of interest of the image to be processed;

[0309] extracting initial feature information from the information of interest;

[0310] Performing dimensionality reduction processing on the initial feature information to obtain low-dimensional information;

[0311] The low-dimensional information is processed based on the request task to obtain a processing result.

[0312] Specifically, the specific implementation method of the processor 13 for the above computer readable instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0313] In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical functional division, and actual implementation may employ other division methods.

[0314] The computer-readable storage medium stores computer-readable instructions, wherein the computer-readable instructions, when executed by the processor 13, are used to implement the following steps:

[0315] Responding to an image processing request, obtaining an image to be processed;

[0316] identifying a request task for the image processing request;

[0317] Cutting out an image of a region of interest corresponding to the requested task from the image to be processed based on a preset length;

[0318] Performing multiple transformations on the image of the region of interest to obtain information of interest of the image to be processed;

[0319] extracting initial feature information from the information of interest;

[0320] Performing dimensionality reduction processing on the initial feature information to obtain low-dimensional information;

[0321] The low-dimensional information is processed based on the request task to obtain a processing result.

[0322] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0323] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0324] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0325] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices described may also be implemented by a single unit or device through software or hardware. Terms such as first and second are used to indicate names and do not imply any particular order.

[0326] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An image processing method, characterized in that: The image processing method comprises: Responding to an image processing request, obtaining an image to be processed; identifying a request task for the image processing request; Cutting out an image of a region of interest corresponding to the requested task from the image to be processed based on a preset length; Performing multiple transformations on the image of the region of interest to obtain information of interest of the image to be processed; extracting initial feature information from the information of interest; Performing dimensionality reduction processing on the initial feature information to obtain low-dimensional information; The low-dimensional information is processed based on the requested task to obtain a processing result, including: if the requested task is an offset prediction task, the low-dimensional information is reorganized to obtain first reorganized information, where the first reorganized information includes first information, second information, third information, and fourth information; and a rotation angle of a first image object in the image to be processed is calculated based on an initial angle generated by the first information and a correction angle generated by the second information, where a calculation formula for the correction angle is: in, represents the correction angle, Indicates the number of preset angles, represents the second information, ; generating the offset information of the target center position according to the third information, the fourth information and the size information of the image of the region of interest; generating the processing result according to the rotation angle and the offset information.

2. The image processing method according to claim 1, wherein: The step of cutting out the region of interest image corresponding to the requested task from the image to be processed based on the preset length includes: Identifying key point information of the image to be processed; Calculating the target center position of the image to be processed based on the key point information; Acquire an image shape corresponding to the requested task, and select a target length from the preset lengths based on the image shape; The region of interest image is cropped from the image to be processed based on the target center position and the target length.

3. The image processing method according to claim 1, wherein: The performing multiple transformation processes on the image of the region of interest to obtain the information of interest of the image to be processed includes: identifying a plurality of scanning angles based on a preset number of angles; Performing pull transformation processing on the image of the region of interest based on each scanning angle to obtain image information corresponding to each scanning angle; Encode each image information to obtain representation information; The plurality of representation information are combined to obtain the information of interest.

4. The image processing method according to claim 1, wherein: The extracting initial feature information from the information of interest includes: Performing convolution processing on the information of interest to obtain convolution information; Performing maximum pooling processing on the convolution information to obtain first pooled information; Performing residual processing on the first pooled information to obtain residual information; Performing average pooling processing on the residual information to obtain second pooling information; Performing full connection processing on the second pooled information to obtain full connection information, where the full connection information includes first dimension information and second dimension information; Counting the amount of information of the first dimension as a first amount, and counting the amount of information of the second dimension as a second amount; If the first number is less than a first preset threshold, generating a first feature of the initial feature information according to the first dimension information; If the second number is less than a second preset threshold, a second feature of the initial feature information is generated according to the second dimension information and the configuration value.

5. The image processing method according to claim 1, wherein: If the requested task is a face recognition task, then the processing of the low-dimensional information based on the requested task to obtain a processing result further includes: Performing a cutting and reorganization process on the low-dimensional information according to the cutting value identified by the low-dimensional information to obtain intermediate feature information; Performing full connection processing on the intermediate feature information to obtain target feature information; The target feature information is identified and matched based on a facial feature library to obtain the processing result.

6. The image processing method according to claim 1, wherein: If the requested task is a target detection task, then the processing of the low-dimensional information based on the requested task to obtain a processing result further includes: Performing convolution processing on the low-dimensional information based on a first preset convolution layer to obtain first convolution information; performing activation processing on each convolution element in the first convolution information to obtain a convolution activation value corresponding to each convolution element, and determining a position of a convolution element having a convolution activation value greater than a target preset threshold as an object center position of a second image object in the image to be processed; Performing convolution processing on the low-dimensional information based on a second preset convolution layer to obtain second convolution information, and performing convolution processing on the low-dimensional information based on a third preset convolution layer to obtain third convolution information; Identifying position information of the second image object in the image to be processed from the second convolution information according to the center position of the object; predicting an object category to which the second image object belongs from the third convolution information according to the center position of the object; The processing result is generated according to the object center position, the position information and the object category.

7. An image processing device, characterized in that: The image processing device comprises: an acquiring unit, configured to acquire an image to be processed in response to an image processing request; an identification unit, configured to identify a request task of the image processing request; a cropping unit, configured to crop an image of a region of interest corresponding to the requested task from the image to be processed based on a preset length; a transform unit, configured to perform multiple transform processes on the image of the region of interest to obtain information of interest of the image to be processed; an extraction unit, configured to extract initial feature information from the information of interest; A dimensionality reduction unit, configured to perform dimensionality reduction processing on the initial feature information to obtain low-dimensional information; A processing unit is configured to process the low-dimensional information based on the requested task to obtain a processing result, including: if the requested task is an offset prediction task, reorganizing the low-dimensional information to obtain first reorganized information, where the first reorganized information includes first information, second information, third information, and fourth information; and calculating a rotation angle of a first image object in the image to be processed based on an initial angle generated by the first information and a correction angle generated by the second information, where the calculation formula for the correction angle is: in, represents the correction angle, Indicates the number of preset angles, represents the second information, ; generating the offset information of the target center position according to the third information, the fourth information and the size information of the image of the region of interest; generating the processing result according to the rotation angle and the offset information.

8. An electronic device, characterized in that: The electronic device comprises: a memory storing computer-readable instructions; and A processor is configured to execute computer-readable instructions stored in the memory to implement the image processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the image processing method according to any one of claims 1 to 6.

Images