Data processing method and electronic equipment

Abstract

The invention discloses a data processing method and electronic equipment, and the method comprises the steps: determining a target recognition region corresponding to a target object based on an input image; based on the target recognition area, processing the first image feature of the target recognition area according to a target processing strategy to obtain a target result image; outputting the target result image, so that the target result image is displayed in the target recognition area; wherein the first image feature at least comprises character information and pattern information, and the target result image at least comprises pose data of the first image feature.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and electronic equipment. Background Art

[0002] In areas such as online meetings or online teaching, due to factors such as shooting distance, angle, spatial reflection, and unclear handwriting, users are often unable to recognize what text or patterns are written on the writing board. Summary of the invention

[0003] In view of this, an embodiment of the present application provides a data processing method and an electronic device.

[0004] According to the first aspect of the present application, an embodiment of the present application provides a data processing method, including:

[0005] Based on the input image, determining a target recognition area corresponding to the target object;

[0006] Based on the target recognition area, the first image feature of the target recognition area is processed according to the target processing strategy to obtain a target result image;

[0007] Outputting a target result image so that the target result image is displayed in the target recognition area;

[0008] The first image feature includes at least text information and pattern information, and the target result image includes at least position data of the first image feature.

[0009] Optionally, determining a target recognition area corresponding to the target object based on the input image includes:

[0010] Based on the input image, determining a first recognition area including a target object;

[0011] Determining vertex coordinates of the target object based on image features of the first recognition area;

[0012] Determining a second recognition area based on vertex coordinates of the target object;

[0013] The second recognition area is subjected to trapezoidal correction to obtain a target recognition area corresponding to the target object, and the area of ​​the first recognition area is larger than that of the second recognition area.

[0014] Optionally, after determining the target recognition area corresponding to the target object, the data processing method further includes:

[0015] Based on the target recognition area, the image features of the target object are enhanced;

[0016] The enhancement process includes at least one of the following:

[0017] De-noising the image features of the target object;

[0018] Perform contrast enhancement on the image features of the target object;

[0019] Perform edge enhancement processing on the image features of the target object;

[0020] Binarize the image features of the target object;

[0021] Perform image correction on the image features of the target object.

[0022] Optionally, based on the target recognition area, the first image feature of the target recognition area is processed according to the target processing strategy to obtain a target result image, including:

[0023] Processing the target recognition area based on the target processing strategy to obtain a target detection area corresponding to the first image feature;

[0024] The first image feature in the target detection area is processed to obtain a target result image.

[0025] Optionally, processing the target recognition area based on the target processing strategy to obtain a target detection area corresponding to the first image feature includes:

[0026] Performing feature extraction on the first image feature multiple times to obtain multiple first feature maps with decreasing image sizes;

[0027] Weighting the features related to the first image features in each first feature map to obtain a second feature map;

[0028] Based on the second feature map, an object detection area corresponding to the first image feature is obtained.

[0029] Optionally, obtaining a target detection area corresponding to the first image feature based on the second feature map includes:

[0030] Performing pooling processing on at least one first feature map in a vertical direction and a horizontal direction to obtain a third feature map;

[0031] The third feature map is transformed by a channel attention mechanism to obtain a fourth feature map; the channel includes at least a long text channel, a short text channel and a pattern channel, and the channel attention mechanism includes increasing the weight of the long text channel;

[0032] Based on the second feature map and the fourth feature map, an object detection area corresponding to the first image feature is obtained.

[0033] Optionally, obtaining a target detection area corresponding to the first image feature based on the second feature map and the fourth feature map includes:

[0034] The second feature map is merged with the fourth feature map to obtain a fifth feature map;

[0035] Classification detection is performed on the fifth feature map to obtain a target detection area corresponding to the first image feature, where the target detection area at least includes position information of the detection frame and angle information of the detection frame.

[0036] Optionally, the data processing method further includes:

[0037] Determine the aspect ratio of the detection frame and the distance between the center points based on the position information of the detection frame and the angle information of the detection frame;

[0038] Determine the confidence information of the target detection area based on the aspect ratio of the detection frame and the distance of the center point;

[0039] If the confidence information is greater than the threshold, the target detection area is output.

[0040] Optionally, processing the first image feature in the target detection area to obtain a target result image includes:

[0041] Identify content information corresponding to the first image feature in the target detection area to obtain a recognition result;

[0042] Determine the pose data corresponding to the target detection area corresponding to the recognition result in the target recognition area, where the pose data includes at least one of the position information of the detection frame and the angle information of the detection frame;

[0043] Based on the corresponding position data of the target detection area corresponding to the recognition result in the target recognition area, the recognition result is arranged to form a target result image.

[0044] According to a second aspect of the present application, an embodiment of the present application provides an electronic device, including:

[0045] A determination module, the determination module is used to determine a target recognition area corresponding to a target object based on an input image;

[0046] A processing module, the processing module is used to process the first image feature of the target recognition area according to the target processing strategy based on the target recognition area to obtain a target result image;

[0047] An output module, the output module is used to output a target result image so that the target result image is displayed in the target recognition area;

[0048] The first image feature includes at least text information and pattern information, and the target result image includes at least position data of the first image feature.

[0049] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a data processing method in an embodiment of the present application;

[0051] Figure 2 This is a schematic diagram of the structure of the third detection model in the embodiment of the present application;

[0052] Figure 3 This is a schematic diagram of the structure of the PPM module and the SPM module in the embodiment of the present application;

[0053] Figure 4 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application;

[0054] Figure 5 Schematic diagram of the hardware structure of another electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0056] The present application embodiment provides a data processing method, such as Figure 1 As shown, including:

[0057] S101, determining a target recognition area corresponding to a target object based on an input image.

[0058] In this embodiment, the input image may include not only the target object but also a background image other than the target object. The target object may be an object that needs to regenerate or organize the layout content, such as a writing board. The target recognition area may be the smallest rectangular area including the target object.

[0059] In this embodiment, the input image can be an image including a blackboard or a whiteboard captured during teaching; the input image can be an image including a whiteboard captured during a meeting; the input image can be an experiment record image including handwritten data and / or charts, a scheme design image, an equipment debugging record image, an artistic creation image, a project plan image, etc.

[0060] In some embodiments, the target object recognition area corresponding to the target object can be determined by performing target object recognition on the input image.

[0061] S102, based on the target recognition area, processing the first image feature of the target recognition area according to the target processing strategy to obtain a target result image; wherein the first image feature at least includes text information and pattern information, and the target result image at least includes posture data of the first image feature;

[0062] In this embodiment, the target processing strategy may include at least identifying the posture data of text information and pattern information in the target recognition area, and regenerating the target object layout content based on the posture data of text information and pattern information in the target recognition area to obtain a target result image.

[0063] S103: outputting a target result image so as to display the target result image in the target recognition area.

[0064] In this embodiment, after the target result image is output, the target result image can be used to replace the original image of the target recognition area in the input image, so that the target result image is displayed in the target recognition area of ​​the input image.

[0065] The data processing method provided in the embodiment of the present application determines a target recognition area corresponding to a target object based on an input image; based on the target recognition area, processes a first image feature of the target recognition area according to a target processing strategy to obtain a target result image; wherein the first image feature includes at least text information and pattern information, and the target result image includes at least posture data of the first image feature; outputs the target result image to display the target result image in the target recognition area; in this way, not only the posture data of the text information in the target recognition area can be obtained, but also the posture data of the pattern information in the target recognition area can be obtained, thereby displaying the posture data of the pattern information and the posture data of the text information in the target recognition area of ​​the input image, so that the user can identify the text information and the pattern information, thereby providing the user with more intuitive and clear information, and improving communication efficiency.

[0066] In an optional embodiment, step S101, determining a target recognition area corresponding to a target object based on an input image, includes:

[0067] Based on the input image, a first recognition area including a target object is determined; based on the image features of the first recognition area, the vertex coordinates of the target object are determined; based on the vertex coordinates of the target object, a second recognition area is determined; and trapezoidal correction is performed on the second recognition area to obtain a target recognition area corresponding to the target object, wherein the area of ​​the first recognition area is larger than that of the second recognition area.

[0068] In this embodiment, the target object in the input image can be identified by the first detection model to obtain a first identification area including the target object. The first detection model can be a target object identification model.

[0069] In this embodiment, the input image may be preprocessed first, including resizing, normalization, etc., to meet the input requirements of the first detection model. Then, the preprocessed input image may be input into the first detection model to obtain a first recognition area including the target object. For example, an input image with a whiteboard is input into the whiteboard detection model to detect the whiteboard in the input image and obtain the frame coordinates of the whiteboard.

[0070] In this embodiment, a second detection model may be used to further detect the first recognition area including the target object and identify the vertex coordinates of the target object. The second detection model may be a target object key point detection model.

[0071] In this embodiment, the image output by the first detection model can be input into the second detection model to obtain the vertex coordinates of the target object. For example, the input image with the coordinates of the whiteboard border is input into the second detection model to detect the coordinates of the four corner points of the whiteboard to obtain the vertex coordinates of the whiteboard.

[0072] In this embodiment, the vertex coordinates of the target object are detected, and a rectangular area can be formed based on the vertex coordinates of the target object to obtain a second recognition area.

[0073] In this embodiment, since the second recognition area may not be a standard rectangular area, the second recognition area may be subjected to trapezoidal correction to obtain a target recognition area corresponding to the target object, so that the target recognition area including the target object is a standard rectangular area.

[0074] In this embodiment, based on the input image, a first recognition area including the target object is determined; based on the image features of the first recognition area, the vertex coordinates of the target object are determined; based on the vertex coordinates of the target object, a second recognition area is determined; and the second recognition area is trapezoidally corrected to obtain a target recognition area corresponding to the target object; in this way, a minimum target recognition area including the target object can be obtained, and the target recognition area is a standard rectangular area, which is convenient for subsequent processing.

[0075] In an optional embodiment, after determining the target recognition area corresponding to the target object, the data processing method further includes:

[0076] Based on the target recognition area, the image features of the target object are enhanced.

[0077] The enhancement process includes at least one of the following:

[0078] The image features of the target object are subjected to denoising processing; the image features of the target object are subjected to contrast enhancement processing; the image features of the target object are subjected to edge enhancement processing; the image features of the target object are subjected to binarization processing; and the image features of the target object are subjected to image correction processing.

[0079] In this embodiment, after the target recognition area is determined, the target recognition area image can be segmented from the input image, and then the image features of the target object in the target recognition area image are enhanced.

[0080] In this embodiment, since there may be various noises in the text or pattern in the target object, such as stains, scratches, particles, etc., which will interfere with recognition, algorithms such as median filtering and Gaussian filtering can be used to denoise the image features of the target object.

[0081] In this embodiment, if the contrast of the text or pattern in the target object is low, the edge of the text or pattern will become unclear, affecting recognition. The difference between the text or pattern and the background can be made more obvious by adjusting parameters such as brightness and contrast of the text or pattern in the target object.

[0082] In this embodiment, edge detection algorithms, such as the Canny algorithm, can be used to highlight the edge information of the text or pattern in the target object, making the lines of the text or pattern clearer. After edge enhancement processing, the contour features of the text or pattern can be better extracted, thereby improving the recognition accuracy.

[0083] In this embodiment, the color or grayscale image corresponding to the target recognition area can be converted into a binary image, that is, an image with only black and white colors, in which the text or pattern is black and the background is white. This can simplify the image, reduce the amount of calculation for subsequent processing, and also highlight the shape information of the text or pattern.

[0084] In this embodiment, if the text or pattern in the target object is tilted, distorted, or deformed, correction is required. For example, for tilted text, the tilt angle of the text can be detected by algorithms such as Hough transform, and then rotation correction is performed to make the text horizontal or vertical.

[0085] In this embodiment, based on the target recognition area, the image features of the target object are enhanced, which can make the image features corresponding to the target object more obvious, thereby facilitating the subsequent recognition of the posture data of the text information and pattern information in the target object, and / or the text information and pattern information.

[0086] In an optional embodiment, step S102, based on the target recognition area, the first image feature of the target recognition area is processed according to the target processing strategy to obtain a target result image, including:

[0087] The target recognition area is processed based on the target processing strategy to obtain a target detection area corresponding to the first image feature; the first image feature in the target detection area is processed to obtain a target result image.

[0088] In this embodiment, the specific area of ​​the text and pattern in the target recognition area can be identified by the third detection model to obtain the target detection area corresponding to the first image feature. For example, the whiteboard area image is input into the third detection model for text and pattern detection, and the detection result of the target detection area corresponding to the text and pattern can be obtained. Then, the text recognition model can be used to recognize the text in the target detection area, and the image classification or image recognition model can be used to recognize the image in the target detection area to obtain the final text and pattern. Then, the text and pattern can be displayed in the target result image at the position corresponding to the target detection area.

[0089] In this embodiment, the target recognition area is processed based on the target processing strategy to obtain a target detection area corresponding to the first image feature; the first image feature in the target detection area is processed to obtain a target result image; in this way, not only the posture data of the pattern information and text information in the target recognition area can be identified, but also the specific text information and pattern information can be identified, so that the target result image includes not only the posture data corresponding to the image information and text information, but also the specific pattern information and text information, so that clear and complete content information corresponding to the target object can be formed.

[0090] In an optional embodiment, the target recognition area is processed based on the target processing strategy to obtain the target detection area corresponding to the first image feature, including:

[0091] Perform feature extraction on the first image feature multiple times to obtain multiple first feature maps with decreasing image sizes; weight the features related to the first image feature in each first feature map to obtain a second feature map; and obtain a target detection area corresponding to the first image feature based on the second feature map.

[0092] In specific implementation, the third detection model structure is as follows: Figure 2 As shown in the figure, the commonly used VGG16 network can be used to extract multiple first feature maps from the four stages of convolution layers from top to bottom. The size of the convolution layer is halved successively, but the number of convolution kernels is doubled successively. In this way, first feature maps of different scales can be extracted to achieve detection of text and patterns of different scales.

[0093] like Figure 2 As shown in the figure, during the feature extraction process, from Conv1 to Conv5, each layer needs to be downsampled. Downsampling is an operation that reduces the image resolution and can be used to reduce the image. In downsampling, pixels in the image are merged or discarded to reduce the amount of information in the image. This causes the details in the image to become blurred or lost, and smaller details cannot be retained at a lower resolution. To solve this problem, Figure 2 As shown in the figure, a fully connected layer can be added to the shallow network to implement the SENet attention mechanism, which is used to emphasize the details. The shallow network has not been downsampled many times and still retains a lot of detail information. The SENet attention mechanism can assign higher weights to these channels containing detail information, and weight the features related to the first image features in each first feature map to obtain the second feature map. Then, based on the second feature map, the target detection area corresponding to the first image feature is obtained.

[0094] The features related to the first image features in each first feature map are weighted to obtain the second feature map. The specific process is as follows: Figure 2 As shown, it can be:

[0095] Step 1: Enlargement of the last layer feature map: The first feature map (f1) of the last layer extracted from the feature extraction layer is sent to the unpooling layer of the feature fusion layer. In the unpooling layer, f1 is enlarged by 1 times to obtain the enlarged first feature map.

[0096] Step 2: Combine SENet channel attention weights: The first feature map of the previous layer (assuming it is f0) is processed through the SENet channel attention mechanism to obtain a weighted feature map (f2). The enlarged first feature map is combined with the weighted feature map through a concatenation operation to form a new feature map (f11).

[0097] Step 3: Convolution processing: Perform two convolution operations on the new feature map (f11): 1x1 convolution: process it with a convolution kernel size of 1x1; 3x3 convolution: process it with a convolution kernel size of 3x3 to obtain the processed feature map (f111).

[0098] Step 4: Repeat the process: The processed feature map (f111) is enlarged more than twice, combined with the SENet channel attention weight and convolution process. The number of convolution kernels in each layer of the feature fusion layer decreases layer by layer, and the number of convolution kernels is 128, 64, and 32 respectively.

[0099] Step 5: Final convolution processing: The feature map output by the last feature fusion layer is finally processed by 32-core, 3x3 convolution to obtain the second feature map.

[0100] In this embodiment, feature extraction is performed on the first image feature multiple times to obtain multiple first feature maps with decreasing image sizes; the features related to the first image feature in each first feature map are weighted to obtain a second feature map; thereby, the second feature map includes detail features related to the first image feature, and when detecting text based on the second feature map, short text can be detected, thereby improving the problem of missing short text.

[0101] In an optional embodiment, obtaining the target detection area corresponding to the first image feature based on the second feature map includes:

[0102] Performing pooling processing on at least one first feature map in vertical and horizontal directions to obtain a third feature map; performing channel attention mechanism transformation on the third feature map to obtain a fourth feature map; the channel includes at least a long text channel, a short text channel and a pattern channel, and the channel attention mechanism includes increasing the weight of the long text channel; based on the second feature map and the fourth feature map, obtaining a target detection area corresponding to the first image feature.

[0103] When implementing it, Figure 2 As shown in the figure, a hybrid pooling module (MPM) can be introduced, and a 1xN or Nx1 pooling kernel can be used to pool the first feature map output by the last feature extraction layer in the vertical and horizontal directions. While increasing the receptive field of the first feature map, it can avoid mixing interference information from irrelevant areas when capturing text, especially long text. At the same time, a Coordinate Attention module (channel attention mechanism module) is added to strengthen the part of interest by embedding the position information into the channel attention. This module still uses a (1xN or Nx1) pooling kernel to enable the network to more accurately locate the long text part of interest. Then the second feature map and the fourth feature map are input to the output layer to obtain the target detection area corresponding to the first image feature.

[0104] In some embodiments, the hybrid pooling module MPM may be composed of a strip pooling module (SPM) and a pyramid pooling module (PPM). Figure 3 As shown in (a) of Figure 3As shown in (b) in the figure. The SPM module encodes along the horizontal or vertical spatial dimension, and then merges each spatial position in the two dimensions to refine the features. This long and narrow pooling kernel easily establishes long-range dependencies between discrete regions. The PPM pyramid pooling module (PPM) uses a square pooling kernel to further expand the receptive field, which is conducive to capturing short-range dependencies.

[0105] The Coordinate Attention module can decompose channel attention into two one-dimensional features for encoding, aggregate features along the horizontal or vertical spatial direction respectively, assign greater weights to channels with long texts, which helps the final classification decision, and can also capture direction-aware and position-sensitive information to help accurately locate long texts.

[0106] The Coordinate Attention module mainly consists of two steps: Squeeze (global information embedding) and Excitation (adaptive recalibration). In the Squeeze stage, a global average pooling operation is used to convert the input features from a three-dimensional tensor to a one-dimensional vector, which can reduce the amount of calculation and extract the global information of the features so that each unit can use the contextual information outside the area to calculate the importance of each channel. In the Excitation stage, feature weighting is mainly performed by learning the correlation between the channels of the features. Specifically, the Excitation stage first resizes the one-dimensional vector into a two-dimensional tensor, and then performs feature mapping and channel relationship modeling through two fully connected layers to obtain the importance weight of each channel. Finally, these weights are applied to the input features to strengthen useful features and weaken irrelevant features, thereby improving the representation ability of the network.

[0107] The specific process of determining the fourth characteristic map is:

[0108] Step 1: Through the MPM module: The first feature map (f1) is processed by two mixed pooling modules (MPM) to obtain a feature map (f1') processed by MPM, that is, the third feature map.

[0109] Step 2: Through the Coordinate Attention module: The feature map (f1') processed by the MPM is processed by the Coordinate Attention module. The Coordinate Attention module embeds the position information into the channel attention and decomposes it into two one-dimensional features for encoding. The feature map (f1") processed by the Coordinate Attention module is obtained.

[0110] Step 3: Unpooling and enlargement: Send f1'' to the unpooling layer and enlarge it by 8 times to obtain the enlarged feature map (f1'').

[0111] Step 4: Final convolution processing: The enlarged feature map (f1””) is processed by 32-core, 1x1 convolution to obtain the fourth feature map.

[0112] In this embodiment, multiple convolutions and downsampling may lead to a problem of too small a receptive field, resulting in broken detection frames and missed detections when detecting long texts. A third feature map is obtained by performing vertical and horizontal pooling processing on at least one first feature map; the third feature map is transformed by a channel attention mechanism to obtain a fourth feature map; the channel includes at least a long text channel, a short text channel, and a pattern channel, and the channel attention mechanism includes increasing the weight of the long text channel, so that the fourth feature map has a larger receptive field and strengthens the weight of the long text channel of interest. Therefore, based on the second feature map and the fourth feature map, when performing first image feature detection, not only the problem of missed detection of short texts can be improved, but also the problem of inadequate detection of long texts can be improved.

[0113] In an optional embodiment, obtaining the target detection area corresponding to the first image feature based on the second feature map and the fourth feature map includes:

[0114] The second feature map is fused with the fourth feature map to obtain a fifth feature map; the fifth feature map is classified and detected to obtain a target detection area corresponding to the first image feature, and the target detection area at least includes position information of the detection frame and angle information of the detection frame.

[0115] When implementing it, Figure 2 As shown, the second feature map and the fourth feature map can be fused to obtain a fifth feature map; then the fifth feature map is input to the output layer, and the output layer detects text and pattern on the fifth feature respectively to obtain the corresponding text box position and rotation angle, pattern box position and rotation angle.

[0116] In this embodiment, the position information of the detection frame may include four parameters (a, b, c, d), which respectively represent the values ​​of the four distances from the pixel 0 point position to the top, right, bottom, and left boundaries of the rectangular frame, which can more accurately describe the position and size of the text area. The angle information of the detection frame may include a parameter θ, which represents the rotation angle of the rectangular frame.

[0117] In some embodiments, the loss function of the third detection model during training may be:

[0118]

[0119] Among them, ρ2 (b,b gt ) is the square of the Euclidean distance, b and b gt are the center coordinates of the detection box and the label box respectively, c is the diagonal length of the smallest rectangle that just contains the detection box and the label box, and IoU is the traditional intersection over union ratio; A is the detection box, B is the label box, and IoU calculates the ratio of the intersection and union of the detection box and the label box. v is the distance of the aspect ratio of the detection box and the label box: w gt 、h gt , w, h are the width and height of the label box and the detection box respectively; α is the weight coefficient;

[0120] In this embodiment, the loss function not only considers the intersection-and-union ratio of the rectangular box, but also considers the aspect ratio of the rectangular box and the distance of the center point, so that the third detection model is sensitive to the shape of the text. The addition of the center point distance enables short texts to be effectively detected, which can improve the third detection model's ability to detect short texts.

[0121] In this embodiment, the fifth feature map is obtained by fusing the second feature map with the fourth feature map; the fifth feature map is classified and detected to obtain a target detection area corresponding to the first image feature, so that the fifth feature map has a larger receptive field, and the weight of the long text channel of interest is strengthened, and the detailed features of the short text are retained. Short text and long text can be accurately detected, and the accurate position and rotation angle of the text box and the position and rotation angle of the pattern box can be obtained.

[0122] In an optional embodiment, the data processing method further includes:

[0123] Based on the position information and angle information of the detection frame, determine the aspect ratio of the detection frame and the distance from the center point; based on the aspect ratio and the distance from the center point of the detection frame, determine the confidence information of the target detection area; if the confidence information is greater than the threshold, output the target detection area.

[0124] In this embodiment, based on the position information and angle information of the detection frame, the aspect ratio of the detection frame and the distance of the center point can be calculated, and then the aspect ratio of the detection frame and the distance of the center point are compared with a preset value to determine the confidence information of the target detection area. If the confidence information is greater than a threshold, the target detection area is output; if the confidence information is less than the threshold, it is determined that the target detection area is incorrect.

[0125] In this embodiment, the confidence information of the target detection area is determined by calculating the aspect ratio of the detection frame and the distance between the center points, so that the output target detection area can be more reasonable and accurate.

[0126] In an optional embodiment, processing the first image feature in the target detection area to obtain a target result image includes:

[0127] Identify content information corresponding to the first image feature in the target detection area to obtain a recognition result; determine the corresponding posture data of the target detection area corresponding to the recognition result in the target recognition area, the posture data including at least one of the position information of the detection frame and the angle information of the detection frame; based on the corresponding posture data of the target detection area corresponding to the recognition result in the target recognition area, typeset the recognition result to form a target result image.

[0128] In this embodiment, the text in the target detection area can be recognized by a text recognition model, and the image classification or image recognition model can be used to recognize the image in the target detection area to obtain the final text and pattern. Then, according to the corresponding posture data of the text and pattern in the target recognition area, typesetting and layout are performed to restore the text and pattern in the target object, generate the target object after layout restoration, and obtain the target result image.

[0129] In this embodiment, the recognition result is obtained by identifying the content information corresponding to the first image feature in the target detection area; the posture data corresponding to the target detection area corresponding to the recognition result in the target recognition area is determined, and the posture data includes at least one of the position information of the detection frame and the angle information of the detection frame; based on the posture data corresponding to the target detection area corresponding to the recognition result in the target recognition area, the recognition result is typeset to form a target result image; in this way, effective detection and recognition of text and patterns in the target object can be achieved, and a target result image with clear and complete content and pattern can be regenerated for user viewing.

[0130] The present application also provides an electronic device, such as Figure 4 As shown, including:

[0131] A determination module 51, which is used to determine a target recognition area corresponding to a target object based on an input image;

[0132] A processing module 52, the processing module is used to process the first image feature of the target recognition area according to the target processing strategy based on the target recognition area to obtain a target result image;

[0133] An output module 53, the output module is used to output a target result image so that the target result image is displayed in the target recognition area;

[0134] The first image feature includes at least text information and pattern information, and the target result image includes at least position data of the first image feature.

[0135] In some embodiments, Figure 5 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0136] like Figure 5 As shown, electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The computing unit 801 may include a determination module, a processing module, and an output module. In RAM 803, various programs and data required for the operation of the electronic device 800 may also be stored. The computing unit 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0137] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0138] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0139] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0141] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0143] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0144] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0145] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0146] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0147] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, comprising: Based on the input image, determining a target recognition area corresponding to the target object; Based on the target recognition area, the first image feature of the target recognition area is processed according to the target processing strategy to obtain a target result image; Outputting the target result image so as to display the target result image in the target recognition area; The first image feature includes at least text information and pattern information, and the target result image includes at least position data of the first image feature.

2. The data processing method according to claim 1, wherein determining the target recognition area corresponding to the target object based on the input image comprises: Based on the input image, determining a first recognition area including the target object; Determining vertex coordinates of the target object based on image features of the first recognition area; Determining a second recognition area based on vertex coordinates of the target object; The second recognition area is subjected to trapezoidal correction to obtain a target recognition area corresponding to the target object, wherein the area of ​​the first recognition area is larger than that of the second recognition area.

3. The data processing method according to claim 2, after determining the target recognition area corresponding to the target object, further comprising: Based on the target recognition area, enhancing the image features of the target object; The enhancement process includes at least one of the following: Performing denoising processing on the image features of the target object; Performing contrast enhancement processing on the image features of the target object; Performing edge enhancement processing on the image features of the target object; Performing binarization processing on the image features of the target object; Perform image correction processing on the image features of the target object.

4. The data processing method according to claim 1, wherein the first image feature of the target recognition area is processed according to the target processing strategy to obtain the target result image, comprising: Processing the target recognition area based on the target processing strategy to obtain a target detection area corresponding to the first image feature; The first image feature in the target detection area is processed to obtain the target result image.

5. The data processing method according to claim 4, wherein the step of processing the target recognition area based on the target processing strategy to obtain the target detection area corresponding to the first image feature comprises: Performing feature extraction on the first image feature multiple times to obtain multiple first feature maps with decreasing image sizes; Weighting the features related to the first image features in each first feature map to obtain a second feature map; Based on the second feature map, an object detection area corresponding to the first image feature is obtained.

6. The data processing method according to claim 5, wherein obtaining the target detection area corresponding to the first image feature based on the second feature map comprises: Performing pooling processing on at least one of the first feature maps in vertical and horizontal directions to obtain a third feature map; Performing a channel attention mechanism transformation on the third feature map to obtain a fourth feature map; the channel includes at least a long text channel, a short text channel and a pattern channel, and the channel attention mechanism includes increasing the weight of the long text channel; Based on the second feature map and the fourth feature map, a target detection area corresponding to the first image feature is obtained.

7. The data processing method according to claim 6, wherein obtaining the target detection area corresponding to the first image feature based on the second feature map and the fourth feature map comprises: Fusing the second feature map with the fourth feature map to obtain a fifth feature map; Classification detection is performed on the fifth feature map to obtain a target detection area corresponding to the first image feature, where the target detection area at least includes position information of a detection frame and angle information of the detection frame.

8. The data processing method according to claim 7, further comprising: Determine the aspect ratio of the detection frame and the distance between the center points based on the position information of the detection frame and the angle information of the detection frame; Determining confidence information of the target detection area based on the aspect ratio of the detection frame and the distance between the center points; If the confidence information is greater than a threshold, the target detection area is output.

9. The data processing method according to claim 5, processing the first image feature in the target detection area to obtain the target result image, comprising: Identify content information corresponding to the first image feature in the target detection area to obtain a recognition result; Determine the pose data corresponding to the target detection area corresponding to the recognition result in the target recognition area, wherein the pose data includes at least one of the position information of the detection frame and the angle information of the detection frame; Based on the posture data corresponding to the target detection area corresponding to the recognition result in the target recognition area, the recognition result is arranged to form the target result image.

10. An electronic device, comprising: A determination module, the determination module is used to determine a target recognition area corresponding to a target object based on an input image; A processing module, the processing module is used to process the first image feature of the target recognition area according to the target processing strategy based on the target recognition area to obtain a target result image; an output module, the output module being used to output the target result image so as to display the target result image in the target recognition area; The first image feature includes at least text information and pattern information, and the target result image includes at least position data of the first image feature.

Images