Image processing method and device

Abstract

The invention at least provides an image processing method and device. The image processing method comprises the steps of obtaining at least two to-be-recognized images; the at least two to-be-recognized images respectively comprise image information obtained by acquiring the same coded image at different positions by two acquisition units; if the at least two to-be-recognized images do not meet the target condition, obtaining a target image based on the at least two to-be-recognized images; the target condition representation can be identified to obtain an analysis result corresponding to the coded image; wherein the target image meets the target condition.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of image processing technology, and in particular to an image processing method and device. Background Art

[0002] Currently, when image recognition is performed on a captured coded image, the captured image cannot be effectively recognized because the captured image does not meet the recognition requirements. Summary of the invention

[0003] In view of this, the embodiments of the present application at least provide an image processing method and device.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising: obtaining at least two images to be identified; the at least two images to be identified respectively include image information obtained by two acquisition units acquiring the same encoded image at different positions; if the at least two images to be identified do not meet the target condition, obtaining a target image based on the at least two images to be identified; the target condition representation can be identified to obtain a parsing result corresponding to the encoded image; wherein the target image meets the target condition.

[0006] In a second aspect, an embodiment of the present application provides an image processing device, comprising: a first acquisition module, used to obtain at least two images to be identified; the at least two images to be identified respectively include image information obtained by two acquisition units acquiring the same encoded image at different positions; a second acquisition module, used to obtain a target image based on the at least two images to be identified if the at least two images to be identified do not meet the target condition; the target condition representation can be identified to obtain a parsing result corresponding to the encoded image; wherein the target image meets the target condition.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising two acquisition units, a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements some or all of the steps in the above method when executed by a processor.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement some or all of the steps in the above method.

[0010] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0012] Figure 1 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0013] Figure 2 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0014] Figure 3 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0015] Figure 4 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0016] Figure 5 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0017] Figure 6 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0018] Figure 7 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0019] Figure 8 A schematic diagram of a shooting scene based on a binocular camera provided in an embodiment of the present application;

[0020] Fig. 9 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application;

[0021] Fig.10 A schematic diagram of epipolar correction is provided for an embodiment of the present application;

[0022] Fig.11 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0023] Fig.12 A hardware entity schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0025] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific order for the objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.

[0027] Currently, when recognizing an image captured after a coded image is captured, the captured image cannot be effectively recognized because the captured image does not meet the recognition requirements.

[0028] The embodiment of the present application provides an image processing method, comprising obtaining at least two images to be identified; the at least two images to be identified respectively include image information obtained by two acquisition units acquiring the same coded image at different positions; if the at least two images to be identified do not meet the target condition, a target image is obtained based on the at least two images to be identified; the target condition representation can be identified to obtain the analysis result corresponding to the coded image; wherein the target image meets the target condition. The method can be executed by a processor of an electronic device. The electronic device may refer to a server, a laptop, a tablet computer, a desktop computer, a smart TV, a set-top box, a mobile device (such as a mobile phone, a portable video player, a personal digital assistant, a dedicated messaging device, a portable gaming device), a wearable device, and other devices with data processing capabilities.

[0029] Figure 1 The following is a schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application, which can be executed by a processor of an electronic device. Figure 1 As shown, the method includes the following steps S101 to S102, combining Figure 1 The steps shown are explained.

[0030] Step S101: Obtain at least two images to be recognized.

[0031] The at least two images to be identified respectively include image information obtained by two acquisition units acquiring the same coded image at different positions.

[0032] In some embodiments, the two acquisition units may be two photographing devices disposed at different positions or at different angles. For example, the two acquisition units may be two cameras disposed at different positions and at different angles.

[0033] In some embodiments, the two acquisition units may also be two shooting components with different viewing angles arranged in the same shooting device. Exemplarily, the two acquisition units may be two cameras on a binocular vision device, wherein the two cameras have different viewing angles, such as two cameras arranged in a virtual reality head mounted display device.

[0034] In some embodiments, the encoded image may be a one-dimensional code (eg, a barcode), a two-dimensional code, or the like.

[0035] In some embodiments, the image information of the coded image represents the surface texture features of the coded image, wherein the image to be identified includes the surface texture features of the coded image, such as the surface texture features of a two-dimensional code.

[0036] In some embodiments, at least two images to be identified are obtained. It can be understood that based on two shooting devices set at different positions or at different angles, shooting is performed on the same coded image to obtain multiple images at different angles and different positions that include surface texture features of the coded image.

[0037] Exemplarily, two cameras are respectively set at point A and point B. Based on the first camera set at point A and the second camera set at point B, the QR code image set at point C is photographed to obtain multiple images including surface texture features of the QR code image.

[0038] In some embodiments, the same coded image is photographed by two cameras at different positions and different viewing angles in a binocular vision device to obtain multiple images at different positions and different viewing angles that include surface texture features of the coded image.

[0039] Exemplarily, based on a camera disposed at a left eye position and a camera disposed at a right eye position in a virtual reality head mounted display device, the same QR code is photographed to obtain multiple images including surface texture features of the QR code at different viewing angles and different positions.

[0040] Step S102: If the at least two images to be recognized do not meet the target condition, obtain a target image based on the at least two images to be recognized.

[0041] The target condition representation can be identified to obtain a parsing result corresponding to the encoded image.

[0042] Wherein, the target image satisfies the target condition.

[0043] In some embodiments, the analysis result includes the information stored in the encoded image; illustratively, the QR code is identified to obtain the analysis result, and the analysis result includes the stored text information, image information, video information, product information, etc.

[0044] In some embodiments, the at least two images to be identified do not satisfy the target condition, indicating that both images to be identified do not satisfy the target condition. It can be understood that when the two images to be identified are identified, no information stored in the corresponding encoded image can be obtained.

[0045] In some embodiments, the target image satisfies the target condition. It can be understood that by identifying the target image, a parsing result of the information stored in the encoded image can be obtained.

[0046] In some embodiments, a target image is obtained based on at least two images to be identified. It can be understood that an image including surface texture features of the encoded image is processed, and images in the image to be identified that are irrelevant to the surface texture features of the encoded image are cropped to obtain multiple cropped images. Feature extraction is performed on the cropped images to obtain eigenvalues ​​and eigenvectors related to the expressive texture features of the encoded image, and the eigenvalues ​​and eigenvectors in the multiple cropped images are fused to obtain the target image, thereby identifying the target image and obtaining a parsing result including the information stored in the encoded image.

[0047] In an embodiment of the present application, the same coded image is collected by collecting units arranged at different positions to obtain multiple images to be identified including image information of the coded image. Based on the multiple images to be identified that cannot identify the analysis results of the coded image, a target image that can identify the analysis results of the coded image is obtained to improve the accuracy of the images collected by the recognition collection unit, thereby improving the user experience.

[0048] In some embodiments, in the above step S102, if the at least two images to be recognized do not meet the target condition, it can be determined in the following manner:

[0049] If the information completeness of the encoded image included in the image to be identified is less than the target threshold, it is determined that the image to be identified does not meet the target condition.

[0050] In some embodiments, the information completeness represents the image completeness of the encoded image included in the object to be identified. It can be understood that if the image completeness of the encoded object included in the image to be identified is less than the target threshold, it represents that the image to be identified does not meet the target condition.

[0051] In some embodiments, the target threshold represents that when the integrity of the encoded image reaches the target threshold, the encoded image can be recognized; illustratively, the encoded image in the image to be recognized lacks the image in the upper right corner, resulting in its image integrity of 80%, and the target threshold is 90%, then the image integrity of the image to be recognized is less than the target threshold, indicating that the image to be recognized cannot be recognized, thereby determining that the image to be recognized cannot recognize the parsing result of the encoded image.

[0052] If the information completeness of the encoded image included in the image to be identified is greater than the target threshold and the image to be identified cannot be identified after the first processing, it is determined that the image to be identified does not meet the target condition.

[0053] In some embodiments, the information completeness of the encoded image included in the image to be identified is greater than the target threshold, indicating that the image to be identified can be identified; exemplarily, the image completeness of the encoded image in the image to be identified is 95%, and the target threshold is 90%, then the image completeness of the encoded image in the image to be identified is greater than the target threshold, indicating that the image to be identified can be identified.

[0054] In some embodiments, the first processing may be to perform anti-distortion processing on the image to be identified, including: calibrating the image to be identified based on a calibration tool (such as a checkerboard) to obtain a calibrated image, and extracting corner points in the calibrated image through an image processing algorithm; using the corner point coordinates and the actual size of the calibration tool to calculate the intrinsic parameter matrix (including focal length, principal point coordinates, etc.) and distortion coefficient of the acquisition unit; according to the result of the calibration of the acquisition unit, using the anti-distortion formula to map each pixel in the calibration image; calculating the grayscale value of the mapped pixel through an interpolation algorithm (such as bilinear interpolation) to obtain the anti-distorted image to be identified.

[0055] In some embodiments, the first processing may be image enhancement processing of the image to be identified, including adjusting parameters such as contrast, brightness, and sharpening of the image to be identified to improve the quality of the adjusted image to be identified.

[0056] In some embodiments, the image to be identified after the first processing is identified. If the parsing result of the coded image in the image to be identified cannot be obtained, it indicates that the image to be identified does not meet the target condition.

[0057] In an embodiment of the present application, by judging the information completeness of the image to be identified and the size of the target threshold, it is determined whether the image to be identified meets the target condition. If the information completeness of the image to be identified is less than the target threshold, it is determined that the image to be identified cannot be identified, that is, it does not meet the target condition; if the information completeness of the image to be identified is greater than the target threshold, and the image to be identified cannot be identified after the first processing, it is determined that the image to be identified does not meet the target condition, thereby improving the accuracy of identifying the image to be identified.

[0058] Figure 2 The following is a schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application. The method can be executed by a processor of an electronic device. The method includes steps S201 and S202. Figure 2 The steps shown are explained.

[0059] Step S201: Determine a recognition order based on the image effect of each image to be recognized.

[0060] The image effect is used to characterize the quality parameter of the encoded image included in the image to be identified.

[0061] In some embodiments, the quality parameters may include information integrity, size, clarity, resolution, contrast, etc. of the encoded image.

[0062] In some embodiments, at least two images to be identified are sorted from high to low based on quality parameters of the encoded images included in the images to be identified, and the order is determined as the recognition order for the images to be identified; in other embodiments, at least two images to be identified are sorted from low to high based on quality parameters of the encoded images included in the images to be identified, and the order is determined as the recognition order for the images to be identified.

[0063] Exemplarily, the clarity of the QR code images in the image to be recognized is sorted from high to low, and the order is determined as the recognition order; or the clarity of the QR code images in the image to be recognized is sorted from low to high, and the order is determined as the recognition order.

[0064] Step S202: performing recognition based on the recognition order until an image to be recognized that meets the target condition is recognized, and determining the image to be recognized as the target image.

[0065] In some embodiments, the images to be identified are identified based on the order of quality parameters of the encoded images from high to low.

[0066] In some embodiments, the objects to be identified are identified based on the order of quality parameters of the encoded images from low to high.

[0067] In some embodiments, each image to be identified is identified based on the identification order until the parsing result of the encoded image in the image to be identified can be identified, and the image to be identified is determined as the target image; it can be understood that based on the decoding algorithm corresponding to the type of the encoded image included in the image to be identified, the encoded image is decoded until the parsing result including the information stored in the encoded image can be obtained, and the image to be identified is determined as the target image.

[0068] In some embodiments, when a target image of a parsing result capable of identifying a coded image is obtained, identification of the remaining images to be identified among the at least two images to be identified is stopped.

[0069] Exemplarily, the QR code images in at least two images to be identified are identified in descending order based on the clarity of the QR code images in the images to be identified, so that the QR code image in each image to be identified is decoded by a QR code decoding algorithm until the analysis information including the link, text, image, etc. stored in the QR code image is obtained, and the image to be identified is determined as the target image; and the identification of the remaining images to be identified is stopped.

[0070] In an embodiment of the present application, at least two images to be identified are sorted by quality parameters of the encoded image in each image to be identified and the order is determined as the recognition order, and the images to be identified are identified based on the recognition order until a target image that can identify the parsing result of the encoded image is obtained, thereby improving the recognition efficiency.

[0071] Figure 3 The following is a schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application, which can be executed by a processor of an electronic device. Figure 1 , Figure 1 Step S102 in can be updated to step S301 to step S303, combining Figure 3 The steps shown are explained.

[0072] Step S301: Obtain feature points of each image to be identified and description information of the feature points.

[0073] In some embodiments, the feature points of the image to be identified are obvious points in the image. It can be understood that the feature points refer to points where the grayscale value of the image changes dramatically or points with larger curvature on the edge of the image (i.e., the intersection of two edges). These points are also called interest points or corner points; for example, prominent corner points, edge endpoints, extreme points, etc.

[0074] In some embodiments, the feature points have advantages such as rotation invariance, illumination invariance, and viewing angle invariance, so that the feature points can still be accurately detected under different shooting angles and illumination conditions.

[0075] In some embodiments, each image to be identified is detected based on a feature point detection algorithm to obtain feature points in the image to be identified; wherein the feature point detection algorithm may include a corner point detection algorithm, features from accelerated segment test (FAST), oriented fast and rotated brief (ORB), etc.

[0076] In some embodiments, the description information of the feature points includes the position, scale, and direction of each feature point.

[0077] In some embodiments, obtaining the feature points of each image to be identified includes: selecting a point P from the image to be identified, drawing a circle with a radius of 3 pixels with P as the center; comparing the grayscale values ​​of the pixels on the circumference, and finding pixels with grayscale values ​​exceeding l(P)+h and below l(P)-h, where l(P) is the grayscale of point P, and h is a given threshold; if there are n consecutive pixels that meet the conditions (generally n is set to 9), P is considered to be a feature point; and obtaining the position, scale, and direction of each feature point.

[0078] Step S302: In at least two images to be identified, obtain at least one feature point group based on the description information.

[0079] The feature point group includes two feature points respectively located in at least two images to be identified.

[0080] In some embodiments, at least one feature point group is obtained based on the description information. It can be understood that the feature points between at least two images to be identified are matched based on the description information to obtain at least one feature point group, including: determining the feature vector of each feature point based on the position, scale, and direction of each feature point, calculating the hash value between the feature vectors of the feature points between at least two images to be identified, and when the hash value between the feature vectors of the feature points between at least two images to be identified is less than a preset threshold, at least two feature points corresponding to the at least two images to be identified are determined as a feature point group.

[0081] Exemplarily, a first image to be identified and a second image to be identified are included, a feature vector of the first feature point is determined based on description information of the first feature point in the first image to be identified, a feature vector of the second feature point is determined based on description information of the second feature point in the second image to be identified, a hash value of the feature vector of the first feature point and the feature vector of the second feature point is calculated, and when the hash value is less than a preset threshold, the first feature point in the first image to be identified and the second feature point in the second image to be identified are determined as a feature point group.

[0082] Step S303: Generate a target image based on the at least one feature point group and the at least two images to be recognized.

[0083] In some embodiments, a transformation matrix is ​​generated based on at least one feature point group, and a target image is obtained based on the changed transformation matrix and at least two images to be identified; it can be understood that the transformation matrix of at least one feature point group is calculated based on a data processing method, and the transformation matrix and at least two images to be identified are fused to obtain the target image.

[0084] Exemplarily, an affine transformation matrix or a perspective transformation matrix of at least one feature point group is calculated based on the least squares method, a transformation relationship between at least two images to be identified is determined based on the radial transformation matrix or the perspective transformation matrix, and based on the transformation relationship, at least two images to be identified are fused to obtain a target image.

[0085] In an embodiment of the present application, based on feature points of at least two images to be identified and description information corresponding to the feature points, at least one feature point group corresponding to the at least two images to be identified is determined, and based on the at least one feature point group, a transformation relationship between the at least two images to be identified is obtained, so that the at least two portraits to be identified are fused to obtain a target image that can identify the analysis result of the encoded image, thereby improving the recognition accuracy.

[0086] In some embodiments, the description information includes scale and direction; the scale and direction of each feature point in the feature point group in the corresponding image to be identified are the same.

[0087] In some embodiments, the description information of the feature point includes the scale, direction and position. In the process of obtaining the feature points of the image to be identified, it is necessary to construct a multi-scale pyramid (such as a Gaussian pyramid) to detect multiple extreme points in the scale space, and determine the target extreme point as the feature point among the multiple extreme points, so as to determine the scale of the target extreme point as the scale of the feature point, determine the gradient direction of the target extreme point as the direction of the feature point, and determine the position of the target extreme point as the position of the feature point.

[0088] In some embodiments, a feature vector of each feature point is determined based on its direction, scale, and position, a hash value of the feature vector of the feature point between at least two images to be identified is calculated, and at least two feature points whose hash values ​​are less than a threshold are determined as a feature point group.

[0089] Figure 4 The following is a schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application, which can be executed by a processor of an electronic device. Figure 3 , the description information includes the location of the feature point, Figure 3Step S303 in can be updated to step S401 and step S402, combining Figure 4 The steps shown are explained.

[0090] Step S401: based on the position of each feature point in each feature point group in the corresponding image to be identified, determine the regional image of each feature point in each feature point group in the corresponding image to be identified, and obtain at least two regional images corresponding to each feature point group.

[0091] In some embodiments, the at least two region images represent overlapping region images in the at least two images to be identified.

[0092] In some embodiments, multiple feature points belonging to the same image to be identified are obtained from at least one feature point group; based on the positions of the multiple feature point groups in the corresponding image to be identified, the overlapping area image of the image to be identified is determined to obtain at least two overlapping area images corresponding to each feature point group.

[0093] Exemplarily, at least two images to be identified include a first image to be identified and a second image to be identified; a plurality of first feature points belonging to the first image to be identified in at least one feature point group are obtained; a first overlapping area image in the first image to be identified is obtained based on the positions of the plurality of first feature points; a plurality of second feature points belonging to the second image to be identified in at least one feature point group are obtained; a second overlapping area image in the second image to be identified is obtained based on the positions of the plurality of second feature points; thereby obtaining the first overlapping area image and the second overlapping area image.

[0094] Step S402: Generate the target image based on at least two area images corresponding to each of the feature point groups and the at least two images to be recognized.

[0095] In some embodiments, the at least two regional images are fused to obtain a fused regional image; the corresponding regional image in each image to be identified is cropped to obtain at least two cropped images; the fused regional image and the two cropped images are spliced ​​to obtain a target image.

[0096] Exemplarily, at least two images to be identified include a first image to be identified and a second image to be identified; a first overlapping area image in the first image to be identified and a second overlapping area image in the second image to be identified are fused to obtain a fused overlapping area image; the first overlapping area image in the first image to be identified is cropped to obtain a first cropped image; the second overlapping area image in the second image to be identified is cropped to obtain a second cropped image, and a target image is obtained based on the fused overlapping area image, the first cropped image, and the second cropped image.

[0097] In the embodiment of the present application, the overlapping area images corresponding to the images to be identified are obtained through the feature points of the corresponding images to be identified in the feature point group, and the target image is obtained based on the overlapping area image and the images of the images to be identified except the overlapping area, thereby improving the recognition accuracy.

[0098] Figure 5 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application, which method can be executed by a processor of an electronic device. Figure 4 Step S401 in can be updated to step S501 to step S503, combining Figure 5 The steps shown are explained.

[0099] Step S501: Obtain the clarity of the regional image corresponding to each feature point in each feature point group.

[0100] In some embodiments, the clarity of the regional image in the image to be identified corresponding to each feature point in the feature point group is obtained based on the image clarity evaluation algorithm. It can be understood that the edge information in the image is obtained based on the image clarity algorithm, and the image clarity is evaluated by calculating the gradient change of the edge information in the image, wherein the more obvious the gradient change, the higher the image clarity.

[0101] Exemplarily, the image of the area to be identified is converted into a grayscale image, and the grayscale image is converted into a two-dimensional discrete matrix, wherein each element in the two-dimensional discrete matrix represents the grayscale value (pixel value) in the image; the grayscale gradient of the image is obtained based on the gradient function to obtain the grayscale gradient of different parts in the area image, thereby calculating the root mean square sum of the grayscale gradients to obtain the clarity of the image; the root mean square sum of the grayscale gradients of different parts is calculated to obtain the corresponding clarity of different parts.

[0102] Step S502: Generate a target area image corresponding to each feature point group based on the clarity of the area image corresponding to each feature point in each feature point group.

[0103] In some embodiments, the clarity of each part in the regional image is obtained, and the regional images corresponding to each feature point are fused based on the clarity of each part in each regional image to obtain a target regional image.

[0104] Exemplarily, the clarity of the same part in the regional image corresponding to each feature point is compared to obtain the target part with high clarity in the same part in the regional image corresponding to each feature point; based on multiple target parts with high clarity in the regional image corresponding to each feature point, a target regional image is obtained.

[0105] Step S503: generating the target image based on the target area image corresponding to each of the feature point groups and the at least two images to be recognized.

[0106] In some embodiments, an image to be identified corresponding to each feature point is obtained, and a region image corresponding to each feature point in the image to be identified is cropped to obtain a cropped image, and the cropped image of each image to be identified and the target region image are fused to obtain a target image.

[0107] Exemplarily, at least two images to be identified include a first image to be identified and a second image to be identified; the first image to be identified corresponding to the first feature point in the feature point group is obtained, and the first area image corresponding to the first feature point in the first image to be identified is cropped to obtain a first cropped image; the second area image corresponding to the second feature point in the second image to be identified is cropped to obtain a second cropped image, and the target image is obtained based on the target area image, the first cropped image and the second cropped image.

[0108] In an embodiment of the present application, by acquiring the clarity of the area image corresponding to each feature point, the area images are fused based on the clarity of each area image to obtain a target area image, and splicing is performed based on the target area image and at least two images to be identified to obtain a target image that can identify the analysis result of the encoded image, thereby improving the recognition accuracy.

[0109] Figure 6 The following is a schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application, which can be executed by a processor of an electronic device. Figure 5 , the regional image includes at least two sub-region images; the clarity of the regional image includes the clarity of each sub-region image; Figure 5 Step S502 in can be updated to step S601 and step S602, combining Figure 6 The steps shown are explained.

[0110] Step S601: for each sub-region in the region image corresponding to the feature point group, obtain the clarity of the sub-region image of each feature point in the sub-region, and determine the sub-region image with the highest clarity as the target sub-region image of the sub-region.

[0111] In some embodiments, each region image is divided into multiple sub-regions based on a preset division rule to obtain multiple sub-region images, the clarity of each sub-region image in each region image is obtained, the clarity of sub-region images at the same position in the multiple region images is compared, and the sub-region image with high clarity is determined as the target sub-region image. The preset division rule may be division based on a preset size.

[0112] Exemplarily, the area image corresponding to the feature point group includes a first area image and a second area image; the first area image and the second area image are divided into a plurality of first sub-areas and a plurality of second sub-areas, respectively, according to preset sizes to obtain a plurality of first sub-area images and a plurality of second sub-areas images and obtain the clarity of each first sub-area image and each second sub-area image; the clarity of the first sub-area image and the second sub-area image at the same position in the first area image and the second area image is compared to determine the sub-area image with high clarity as the target sub-area image.

[0113] Step S602: Based on the target sub-region image of each of the sub-regions, a target region image corresponding to the feature point group is generated.

[0114] In some embodiments, the clarity of sub-region images at multiple positions in multiple region images is compared to obtain target sub-region images at multiple positions, and the target sub-region images at multiple positions are then fused to obtain the target region image.

[0115] Exemplarily, multiple groups of first sub-region images and second sub-region images corresponding to multiple identical positions in the first region image and the second region image are obtained; the clarity of each group of first sub-region images and second sub-region images corresponding to the identical position is compared to obtain a target sub-region image corresponding to the identical position; and multiple target sub-region images corresponding to the identical positions are fused to obtain a target sub-region image.

[0116] In an embodiment of the present application, by dividing each area image into multiple sub-area images and obtaining the clarity of each sub-area image, a sub-area image with high clarity among the sub-area images at the same position in the multiple area images is determined as a target sub-area image, and the multiple target sub-area images are fused into a target area image to improve the image recognition accuracy.

[0117] Figure 7 A schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application, which method can be executed by a processor of an electronic device. Figure 6 Step S602 in can be updated to step S701 and step S702, combining Figure 7 The steps shown are explained.

[0118] Step S701: cropping a corresponding region image in each of the images to be identified to obtain at least two images to be merged.

[0119] In some embodiments, an image to be identified corresponding to each feature point is obtained, and a region image corresponding to each feature point in the image to be identified is cropped to obtain at least two images to be merged.

[0120] Exemplarily, at least two images to be identified include a first image to be identified and a second image to be identified; the first image to be identified corresponding to the first feature point in the feature point group is obtained, and the first area image corresponding to the first feature point in the first image to be identified is cropped to obtain a first image to be merged; and the second area image corresponding to the second feature point in the second image to be identified is cropped to obtain a second image to be merged.

[0121] Step S702: Merge the at least two images to be merged and the target area image to generate the target image.

[0122] In some embodiments, the position information of each image to be merged in the corresponding image to be identified is obtained; and each image to be merged and the target area image are fused based on the first position information corresponding to each image to be merged to obtain the target image.

[0123] Exemplarily, at least two images to be identified include a first image to be identified and a second image to be identified; at least two images to be merged include a first image to be merged and a second image to be merged corresponding to at least the first image to be identified and the second image to be identified; wherein the first image to be merged is on the left side of the first image to be identified, and the second image to be merged is on the right side of the second image to be identified, and the first image to be merged is spliced ​​on the left side of the target area image, and the second image to be merged is spliced ​​on the right side of the target area image to obtain a target image.

[0124] In an embodiment of the present application, the corresponding area image is cropped out in each image to be identified to obtain the corresponding area image to be merged, and the area image to be merged and the target area image are spliced ​​based on the position of the area to be merged in the corresponding area to be identified to obtain a target image that can identify the parsing result of the encoded image, thereby improving the accuracy of identifying the image to be identified.

[0125] An exemplary application of an image processing method provided in an embodiment of the present application in a practical scenario is described below.

[0126] Currently, when scanning a QR code on a virtual reality (VR) device, only a single camera on the VR device can be used to scan the code, which will result in an inability to accurately focus when scanning based on the VR device, resulting in a problem of scan and recognition failure.

[0127] In some embodiments, when a user uses a binocular camera to scan a QR code, the left and right eyes see images extracted from the left and right cameras respectively, without synthesis, and the corresponding image is recognized by one of the predetermined left or right cameras. Among them, when the scanning frame is located in the center of the two images, only a part of the QR code can be extracted in each frame. The QR code image at this time cannot be used to complete the recognition, so the two QR codes need to be combined. Since the combined image does not need to be displayed to the user, it does not need to be displayed using the same merging algorithm as the binocular camera.

[0128] Figure 8 A schematic diagram of a shooting scene based on a binocular camera is provided for an embodiment of the present application, including a first perspective image 801 and a second perspective image 802, wherein the first perspective image 801 and the second perspective image 802 are respectively obtained based on two cameras equipped on a VR device, and the first perspective image 801 and the second perspective image 802 both include the same QR code, which has different positions in the first perspective image 801 and the second perspective image 802, which will cause the corresponding camera to recognize the QR code. The recognition area of ​​the QR code in the two perspective images is different, thereby reducing the recognition accuracy.

[0129] In some embodiments, the present application detects and matches the QR codes and their position information in the first perspective image 801 and the second perspective image 802 through machine learning to merge the QR codes in the first perspective image 801 and the second perspective image 802 and identify the merged QR code.

[0130] Fig. 9 The following is a schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application. The process can be executed by a processor of an electronic device. The process includes steps S901 to S905. Fig. 9 The steps shown are explained.

[0131] Step S901: Dedistort the two images acquired by the binocular device.

[0132] In some embodiments, two images acquired by a binocular vision device are dedistorted based on the model parameters thereof, including: using a specific calibration tool (such as a chessboard) to calibrate the two images to obtain a calibrated image, and extracting corner points in the calibration image through an image processing algorithm (such as the findChessboardCorners function in the Open Source Computer Vision Library (OpenCV)); using the corner point coordinates and the actual size of the calibration tool to calculate the camera's intrinsic parameter matrix (including focal length, principal point coordinates, etc.) and distortion coefficient; according to the camera calibration result, using the dedistortion formula to map each pixel in the image; and calculating the grayscale value of the mapped pixel through an interpolation algorithm (such as bilinear interpolation) to obtain a dedistorted image.

[0133] Step S902: intercept the dedistorted image based on a preset interception rule to obtain a intercepted image.

[0134] In some embodiments, the center of the dedistorted image is used as the center point, and half the length and half the width of the dedistorted image are used to crop the image to obtain a cropped image; wherein the size of the cropped image is approximately one-fourth of the size of the original image.

[0135] Step S903: performing epipolar correction on the captured image to obtain two corrected images.

[0136] In some embodiments, Fig.10 A schematic diagram of epipolar correction is provided for an embodiment of the present application, wherein 1001 is a shooting point of a binocular device, 1002 is a first image taken by a camera of the binocular device, and 1003 is a second image taken by a camera of the binocular device; epipolar correction is performed on the first image 1002 and the second image 1003 to unify the first image 1002 and the second image 1003 into the same plane to obtain a first corrected image 1004 and a second corrected image 1005.

[0137] Step S904: Acquire feature points in the two corrected images for comparison to obtain a comparison result, and determine an overlapping image of the two corrected images based on the comparison result.

[0138] In some embodiments, feature points in two corrected images are extracted based on a feature extraction algorithm (e.g., Features from Accelerated Segment Test (FAST), Oriented FAST and Rotated BRIEF (ORB)), including: selecting a point P from the image, drawing a circle with a radius of 3 pixels with P as the center; comparing the grayscale values ​​of the pixels on the circumference, and finding pixels with grayscale values ​​exceeding l(P)+h and below l(P)-h, where l(P) is the grayscale of point P, and h is a given threshold; if there are n consecutive pixels that meet the conditions (generally n is set to 9), P is considered to be a feature point, and additional information is added to each feature point, including the position, scale, and direction of each feature point.

[0139] In some embodiments, feature points in two corrected images are compared, including: obtaining a feature vector of each feature point based on the position, scale, and direction of each feature point; if the Euclidean distance between the feature vectors of the feature points in the two corrected images reaches a preset threshold, the corresponding feature points are matched.

[0140] In some embodiments, based on the matched feature points in the two corrected images, the overlapping areas in the two corrected images are obtained to obtain the overlapping image.

[0141] Step S905: Divide the two overlapping images corresponding to the two corrected images into a plurality of blocks, and fuse the target blocks in the plurality of blocks to obtain a target overlapping image.

[0142] In some embodiments, two overlapping images are divided in the same way to obtain N blocks respectively, and the clarity of each block is obtained based on an image clarity algorithm. The clarity of the blocks at the same position in the two overlapping images is compared to obtain N target blocks with high clarity at each position in the two overlapping images, and the N target blocks with high clarity are fused to obtain a target overlapping image.

[0143] In some embodiments, a remaining image other than the cropped image in the dedistorted image is obtained, and the remaining image and the target overlapping image are merged to obtain a target image with high accuracy so that the target image can be identified.

[0144] Based on the foregoing embodiments, an embodiment of the present application provides an image processing device, which includes the units included and the modules included in the units, which can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0145] Fig.11 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the image processing device 1100 includes: a first obtaining module 1101 and a second obtaining module 1102, wherein: the first obtaining module 1101 is used to obtain at least two images to be identified; the at least two images to be identified respectively include image information obtained by two acquisition units acquiring the same encoded image at different positions; the second obtaining module 1102 is used to obtain a target image based on the at least two images to be identified if the at least two images to be identified do not meet the target condition; the target condition representation can be identified to obtain the analysis result corresponding to the encoded image; wherein the target image meets the target condition.

[0146] In some embodiments, the image processing device 1100 also includes a determination module (not shown in the figure), which is used to determine that the image to be identified does not meet the target condition if the information completeness of the encoded image included in the image to be identified is less than the target threshold; if the information completeness of the encoded image included in the image to be identified is greater than the target threshold and cannot be identified after the first processing, it is determined that the image to be identified does not meet the target condition.

[0147] In some embodiments, the determination module (not shown in the figure) is also used to determine the recognition order based on the image effect of each image to be recognized; the image effect is used to characterize the quality parameters of the encoded image included in the image to be recognized; recognition is performed based on the recognition order until an image to be recognized that meets the target conditions is recognized; and the image to be recognized is determined as the target image.

[0148] In some embodiments, the first acquisition module 1101 is further used to obtain feature points of each image to be identified and description information of the feature points; in at least two images to be identified, at least one feature point group is obtained based on the description information; the feature point group includes two feature points located in at least two images to be identified respectively; a target image is generated based on the at least one feature point group and the at least two images to be identified. The description information includes scale and direction; the scale and direction of each feature point in the feature point group are the same in the corresponding image to be identified.

[0149] In some embodiments, the description information includes the position of the feature point; the image processing device 1100 also includes a generation module (not shown in the figure), which is used to determine the regional image of each feature point in each feature point group in the corresponding image to be identified based on the position of each feature point in each feature point group in the corresponding image to be identified, and obtain at least two regional images corresponding to each feature point group; generate the target image based on the at least two regional images corresponding to each feature point group and the at least two images to be identified.

[0150] In some embodiments, the generation module is also used to obtain the clarity of the area image corresponding to each feature point in each feature point group; based on the clarity of the area image corresponding to each feature point in each feature point group, generate the target area image corresponding to each feature point group; based on the target area image corresponding to each feature point group and the at least two images to be identified, generate the target image.

[0151] In some embodiments, the area image includes at least two sub-area images; the clarity of the area image includes the clarity of each sub-area image; the generation module is also used to obtain the clarity of the sub-area image of each feature point in the sub-area for each sub-area in the area image corresponding to the feature point group, and determine the sub-area image with the highest clarity as the target sub-area image of the sub-area; based on the target sub-area image of each sub-area, generate a target area image corresponding to the feature point group.

[0152] In some embodiments, the generating module is further used to crop the corresponding area image in each of the images to be identified to obtain at least two images to be merged; and merge the at least two images to be merged and the target area image to generate the target image.

[0153] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0154] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0155] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0156] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0157] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in an electronic device, a processor in the electronic device executes some or all of the steps for implementing the above method.

[0158] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0159] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0160] Fig.12 A hardware entity diagram of an electronic device provided in an embodiment of the present application, such as Fig.12 As shown, the hardware entity of the electronic device 1200 includes: a processor 1201, a memory 1202 and two acquisition devices 1203, wherein the two acquisition devices 1203 are used to obtain at least two images to be identified; the memory 1202 stores a computer program that can be run on the processor 1201, and the processor 1201 implements the steps in the method of any of the above embodiments when executing the program.

[0161] The memory 1202 stores computer programs that can be run on the processor. The memory 1202 is configured to store instructions and applications executable by the processor 1201. It can also cache data to be processed or processed by the processor 1201 and various modules in the electronic device 1200 (for example, image data, audio data, voice communication data, and video communication data). This can be achieved through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0162] When the processor 1201 executes the program, the steps of any of the above methods are implemented. The processor 1201 generally controls the overall operation of the electronic device 1200.

[0163] An embodiment of the present application provides a computer storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the method of any of the above embodiments.

[0164] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0165] The processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understandable that the electronic device that implements the functions of the processor may also be other, and the embodiments of the present application are not specifically limited.

[0166] The above-mentioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0167] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0168] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0169] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0170] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0171] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art can understand that all or part of the steps of the above method embodiments can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above method embodiments are executed; and the above storage medium includes: mobile storage devices, read-only memory (ROM), disks or optical disks, etc. Various media that can store program codes.

[0172] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0173] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An image processing method, the method comprising: Obtain at least two images to be recognized; The at least two images to be identified respectively include image information obtained by two acquisition units acquiring the same coded image at different positions; If the at least two images to be recognized do not meet the target condition, obtaining a target image based on the at least two images to be recognized; The target condition representation can be identified to obtain a parsing result corresponding to the encoded image; Wherein, the target image satisfies the target condition.

2. The method according to claim 1, further comprising any of the following: If the information integrity of the coded image included in the image to be identified is less than the target threshold, it is determined that the image to be identified does not meet the target condition; If the information completeness of the encoded image included in the image to be identified is greater than the target threshold and the image to be identified cannot be identified after the first processing, it is determined that the image to be identified does not meet the target condition.

3. The method according to claim 1, further comprising: Determine the recognition order based on the image effect of each image to be recognized; the image effect is used to characterize the quality parameter of the encoded image included in the image to be recognized; Recognition is performed based on the recognition sequence until an image to be recognized that meets the target condition is recognized; and the image to be recognized is determined as the target image.

4. The method according to any one of claims 1 to 3, wherein obtaining a target image based on the at least two images to be identified comprises: Obtaining feature points of each of the images to be identified and description information of the feature points; In at least two images to be identified, obtaining at least one feature point group based on the description information; the feature point group includes two feature points respectively located in the at least two images to be identified; A target image is generated based on the at least one feature point group and the at least two images to be recognized.

5. According to the method of claim 4, the description information includes scale and direction; the scale and direction of each feature point in the feature point group are the same in the corresponding image to be identified.

6. The method according to claim 4, wherein the description information includes the location of the feature points; and the step of generating the target image based on the at least one feature point group and the at least two images to be identified comprises: Based on the position of each feature point in each feature point group in the corresponding image to be identified, determine the regional image of each feature point in each feature point group in the corresponding image to be identified, and obtain at least two regional images corresponding to each feature point group; The target image is generated based on at least two area images corresponding to each of the feature point groups and the at least two images to be recognized.

7. The method according to claim 6, wherein generating the target image based on the at least two area images corresponding to each of the feature point groups and the at least two images to be identified comprises: Obtaining the clarity of the regional image corresponding to each feature point in each of the feature point groups; Generate a target area image corresponding to each feature point group based on the clarity of the area image corresponding to each feature point in each feature point group; The target image is generated based on the target area image corresponding to each of the feature point groups and the at least two images to be recognized.

8. The method according to claim 7, wherein the regional image comprises at least two sub-region images; the definition of the regional image comprises the definition of each sub-region image; The generating of the target area image corresponding to each feature point group based on the clarity of the area image corresponding to each feature point in each feature point group includes: For each sub-region in the region image corresponding to the feature point group, obtaining the clarity of the sub-region image of each feature point in the sub-region, and determining the sub-region image with the highest clarity as the target sub-region image of the sub-region; Based on the target sub-region image of each of the sub-regions, a target region image corresponding to the feature point group is generated.

9. The method according to claim 7, wherein generating the target image based on the target area image and the image to be identified comprises: In each of the images to be identified, a corresponding region image is cropped to obtain at least two images to be merged; The at least two images to be merged and the target area image are merged to generate the target image.

10. An electronic device comprising two acquisition units, a processor and a memory, The two acquisition units are used to obtain at least two images to be identified; the at least two images to be identified respectively include image information obtained by two acquisition units acquiring the same coded image at different positions; The memory stores a computer program that can be run on a processor, and the processor executes a program that obtains a target image based on the at least two images to be identified if the at least two images to be identified do not meet a target condition; the target condition representation can be identified to obtain a parsing result corresponding to the encoded image.

Images